Work

Tactile Emotions: Multimodal Affective Captioning with Haptics Improves Narrative Engagement for d/Deaf and Hard-of-Hearing Viewers

Year: 2025

Type: conference-paper

Abstract: Figure 1: Multimodal afective captions, combining visual cues and vibrations felt via a wrist-worn device, enrich the viewing experience for d/Deaf or Hard-of-Hearing individuals by portraying speaker emotions, improving engagement.

Source: CHI Conference on Human Factors in Computing Systems

DOI: 10.1145/3706598.3713304

Authors: Caluã de Lacerda Pataca, Saad Hassan, Lloyd May, Michelle M Olson, Toni D'aurio, Roshan L Peiris, Matt Huenerfauth

Institutions: Rochester Institute of Technology, Tulane University, Stanford University

Language: English

Topic: Tactile and Sensory Interactions

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

Open Access status: hybrid

Funders: National Science Foundation, U.S. Department of Health and Human Services, US-UK Fulbright Commission

Awards: CHS: Medium: Critical Factors for Automatic Speech Recognition in Supporting Small Group Communication Between People who are Deaf or Hard of Hearing and Hearing Colleagues, NSF Convergence Accelerator Track H: AI-based Tools to Enhance Access and Opportunities for the Deaf, Collaborative Research: HCC: Medium: Linguistically-Driven Sign Recognition from Continuous Signing for American Sign Language (ASL), NRT-AI: AWARE-AI: AWAREness for Sensing Humans Responsibly with AI

OpenAlex ID: W4409886290

FWCI: 37.35

Cites: 82

Cited by: 16

Related to: 10