Work

Effects of Text Release Features on Communication and Problem-Solving in Teams with a Deaf/Hard of Hearing and a Hearing Member Using Automatic Speech Recognition

Year: 2025

Type: conference-paper

Abstract: This study examined effects of three text release features used with Automatic Speech Recognition (ASR)-based automatic captioning and of hearing status on quality of performance on problem-solving tasks, number of words two team members exchanged with each other, and on member's judgments regarding the extent the ASR generated text contained errors. Each of 36 teams, consisting of one deaf/hard of hearing (DHH) and one hearing member, worked online to complete three mapping tasks. These mapping tasks required naming of map locations that were marked by numbers along a pathway indicated by a line with numerous turns. For each of three mapping tasks, the hearing member used a different ASR-based text release feature to produce captions for the DHH member to read, either: (a) controlled release; (b) automatic release with no editing; or (c) automatic release with editing. Results suggested that for DHH and hearing members to solve a problem using automatic captioning with ASR synchronously, it is important for the captioning technology to facilitate rapid exchange of words between members and, at the same time, facilitate dealing with the errors produced by the ASR. Consistent with this proposition, problem-solving performance was highest under the automatic release with editing feature.

Source: International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

DOI: 10.1145/3663547.3759694

Authors: Michael Stinson, Rebecca L Carpenter, Donna Easton

Institution: Rochester Institute of Technology

Language: English

Topic: Hearing Impairment and Communication

Subfield: Developmental and Educational Psychology

Field: Psychology

Domain: Social Sciences

SDG: Reduced inequalities

Open Access status: bronze

Funder: National Science Foundation