Year: 2026
Type: conference-paper
Abstract: Existing referring Audio-Visual Segmentation (Ref-AVS) benchmarks primarily focus on explicit expressions whose referents can be resolved through direct visual, acoustic, or linguistic correspondence.Consequently, they provide limited coverage of implicit references that require multi-step reasoning over multimodal evidence.To bridge this gap, we introduce Reasoning Ref-AVS, a new task that requires models to infer the referred object from audio-visual-language cues before generating its segmentation mask.We further construct ReAVS, a comprehensive dataset with manually annotated referring expressions, multimodal cues, object masks, and reasoning chains, covering diverse reasoning spanning diverse reasoning patterns.Evaluations of representative Ref-AVS models on ReAVS reveal substantial performance degradation when direct cross-modal correspondence is insufficient, exposing a clear gap between multimodal recognition and referent reasoning.Motivated by these challenges, we propose Re2Seg, an omni-modal framework that performs object-centric evidence encoding, multi-hop reasoning, and reasoningconditioned mask decoding.Experiments demonstrate that Re2Seg achieves stronger segmentation performance under reasoningintensive references and provides interpretable object-inference traces.These results highlight the importance of learning to reason before masking for bridging the gap between multimodal recognition and referring segmentation.
Authors: Zhixuan Wu, Ye Ni, Yiming Chen, Wenyuan Gu, Bo Cheng
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Awards: G2218334274, G482571221
OpenAlex ID: W7224414723