Work

Fuzzy Feelings: Arousal’s Interpretive Noise and the Case for Acoustic-Based Haptics

Year: 2026

Type: conference-paper

Abstract: Captions rarely convey emotional nuances in speech, leaving Deaf and Hard-of-Hearing (dhh) viewers without access to tonal and affective information. We present a two-part mixed-methods study on how haptic feedback can communicate vocal emotion without adding visual load. In Part 1, we replicated an arousal-driven captioning approach using speech-emotion-recognition to modulate typographic weight and vibration intensity. Participants showed divergent mental models and often mapped “more vibration” to loudness rather than emotional arousal, underscoring the construct’s conceptual fuzziness. In Part 2, we evaluated five acoustic-to-haptic mappings that bypass affective inference and translate pitch, rhythm, and waveform cues into vibration patterns. No single pattern dominated, but participants associated options such as pulse or sawtooth with high-arousal emotions, and pitch-normalized signals with calmer states. We derive design guidelines emphasizing contrastive, acoustically grounded mappings and user control for integrating emotional haptics into short-form, captioned media.

Source: CHI Conference on Human Factors in Computing Systems

DOI: 10.1145/3772318.3793421

Authors: Caluã de Lacerda Pataca, Stephanie Patterson, Roshan L Peiris, Matt Huenerfauth

Institutions: Birmingham City University, Rochester Institute of Technology

Language: English

Topic: Tactile and Sensory Interactions

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

Open Access status: hybrid

Funder: National Science Foundation

Award: Collaborative Research: HCC: Medium: Linguistically-Driven Sign Recognition from Continuous Signing for American Sign Language (ASL)