Work

Semantic-based audiovisual integration enhances active storage in visual working memory

Year: 2025

Type: article

Abstract: • Definitive evidence for the benefit of multisensory integration to working memory is still lacking • Congruent-sound-induced increase in visual memory was accompanied by an enhanced CDA • ERP correlate of semantic-based audiovisual interaction predicted both CDA and memory increases • Cross-modal semantic priming could not account for these neural and behavioral effects • Multisensory semantic integration can facilitate active working memory storage Presenting a semantically congruent sound during visual target encoding can facilitate subsequent short-term memory retrieval, which is generally considered as a beneficial effect of multisensory integration on working memory. However, there remains a fundamental doubt about whether this benefit in performance is truly due to semantic-based multisensory integration rendering information about the target object stored more precisely in working memory, or simply due to better activation of the object’s archetypal template preexisting in long-term memory. The present event-related potential (ERP) study investigated this issue in humans with a fine-grained visual object recognition task to prevent any stronger activation of the object’s internal template from affecting performance. Meanwhile, a retro-cue was introduced to enable the dissociation of the ERPs reflecting audiovisual interactions and the contralateral delay activity (CDA) indexing active visual working memory maintenance. Our Experiment 1 unveiled that the congruent-sound-induced benefit to recognition performance was accompanied by an increased CDA, both of which could be predicted by an early ERP correlate of semantic-specific audiovisual interaction. Experiment 2 further revealed that these behavioral and electrophysiological effects were present only when the congruent sound was delivered synchronously with, but not prior to, the memory items, invalidating cross-modal semantic priming as an alternative account. These findings offer the first direct evidence that multisensory semantic integration enables details about the target object itself to be stored more precisely in working memory, hence the improved short-term memory performance, which establishes a theoretical foundation for improving working memory with multisensory integration.

Source: NeuroImage

DOI: 10.1016/j.neuroimage.2025.121577

Authors: Song Zhao, Shilong Hui, Minran Chen, Chengzhi Feng, Mengdie Zhai, Wenfeng Feng

Institution: Soochow University

Language: English

Topic: Multisensory perception and integration

Subfield: Experimental and Cognitive Psychology

Field: Psychology

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $3,540

Funders: National Natural Science Foundation of China, National Key Research and Development Program of China