Work

Audition controls the flow of visual time during multisensory perception

Year: 2022

Type: article

Abstract: Previous work addressing the influence of audition on visual perception has mainly been assessed using non-speech stimuli. Herein, we introduce the Audiovisual Time-Flow Illusion in spoken language, underscoring the role of audition in multisensory processing. When brief pauses were inserted into or brief portions were removed from an acoustic speech stream, individuals perceived the corresponding visual speech as "pausing" or " skipping ", respectively—even though the visual stimulus was intact. When the stimulus manipulation was reversed—brief pauses were inserted into, or brief portions were removed from the visual speech stream—individuals failed to perceive the illusion in the corresponding intact auditory stream. Our findings demonstrate that in the context of spoken language, people continually realign the pace of their visual perception based on that of the auditory input. In short, the auditory modality sets the pace of the visual modality during audiovisual speech processing.

Source: iScience

DOI: 10.1016/j.isci.2022.104671

Authors: Mariel G. Gonzales, Kristina C. Backer, Yueqi Yan, Lee M. Miller, Heather Bortfeld, Antoine J. Shahin

Institutions: University of California, Merced, University of California Davis Medical Center, University of California, Davis

Language: English

Topic: Multisensory perception and integration

Subfield: Experimental and Cognitive Psychology

Field: Psychology

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $3,000

Funders: National Institutes of Health, National Institute on Deafness and Other Communication Disorders