Understanding Audiovisual Affective Processing: Behavioral Effects and Neural Underpinnings
Year: 2026
Type: article
Abstract: Emotional signals are typically conveyed jointly through vision and audition. This article reviews behavioral, electrophysiological, and neuroimaging evidence on audiovisual affective processing, addressing what its core effects are, when they unfold, and where they occur in the brain. Congruent audiovisual cues often facilitate emotion recognition, although benefits vary across emotions, tasks, measures, and unimodal comparisons. Audiovisual input can also shape affective appraisals and subjective emotional experience. Modality dominance is context-dependent, varying with cue reliability and the affective dimension judged. Cross-modal influences can persist when one modality is task-irrelevant, suggesting limited goal dependence. Electrophysiological findings reveal audiovisual affective interactions at both early and later stages of neural processing. Neuroimaging work implicates posterior superior temporal cortex (pSTC) within a distributed network of sensory, subcortical, and frontal regions. Building on this evidence, we consider Anderson's information integration theory as a computational framework for characterizing modality-related and value-dependent contributions to affective judgments. We propose a hypothesis-generating dual-pathway framework predicting that reliability-related changes in cue influence will covary with sensory-pSTC interactions, whereas value-dependent changes will additionally involve amygdala and prefrontal systems. These hypotheses concern relative rather than exclusive network contributions and require direct testing through independent manipulations of cue reliability and affective value.
DOI: 10.1093/scan/nsag090
Institutions: Nanjing Normal University, University of South Carolina
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,379
Award: G3752145653
OpenAlex ID: W7225303204
