Work

Repurposing Visual Input Modalities for Blind Users: A Case Study of Word Processors

Year: 2020

Type: conference-paper

Abstract: Visual `point-and-click' interaction artifacts such as mouse and touchpad are tangible input modalities, which are essential for sighted users to conveniently interact with computer applications. In contrast, blind users are unable to leverage these visual input modalities and are thus limited while interacting with computers using a sequentially narrating screen-reader assistive technology that is coupled to keyboards. As a consequence, blind users generally require significantly more time and effort to do even simple application tasks (e.g., applying a style to text in a word processor) using only keyboard, compared to their sighted peers who can effortlessly accomplish the same tasks using a point-and-click mouse. This paper explores the idea of repurposing visual input modalities for non-visual interaction so that blind users too can draw the benefits of simple and efficient access from these modalities. Specifically, with word processing applications as the representative case study, we designed and developed NVMouse as a concrete manifestation of this repurposing idea, in which the spatially distributed word-processor controls are mapped to a virtual hierarchical `Feature Menu' that is easily traversable non-visually using simple scroll and click input actions. Furthermore, NVMouse enhances the efficiency of accessing frequently-used application commands by leveraging a data-driven prediction model that can determine what commands the user will most likely access next, given the current `local' screen-reader context in the document. A user study with 14 blind participants comparing keyboard-based screen readers with NVMouse, showed that the latter significantly reduced both the task-completion times and user effort (i.e., number of user actions) for different word-processing activities.

Source: Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics

DOI: 10.1109/smc42975.2020.9283015

Authors: Hae-Na Lee, Vikas Ashok, I. V. Ramakrishnan

Institutions: Stony Brook University, Old Dominion University

Language: English

Topic: Digital Accessibility for Disabilities

Subfield: Human Factors and Ergonomics

Field: Social Sciences

Domain: Social Sciences

Open Access status: green

Funders: National Eye Institute, Eunice Kennedy Shriver National Institute of Child Health and Human Development

Awards: Semantics-Preserving Virtualization: A Computing System Framework to Run Any Screen Reader on Any Device with Easy Customization, Rethinking Electronic Fetal Monitoring to Improve Perinatal Outcomes and Reduce Frequency of Operative Vaginal and Cesarean Deliveries, Space-Compacting Magnification Augmented with Natural Gestures and Keyboardless Text Entry for Low Vision Smartphone Interaction