Work

Unbiased Embodied Visual Representation Learning with Causal Inference and Cross-Modality Alignment

Year: 2025

Type: article

Abstract: Object Goal Navigation (ObjectNav) in novel environments relies on comprehensive scene understanding, including precise visual perception and accurate modeling of spatial-semantic regularities. However, excessive attention to the hand-crafted scene representation in prevailing approaches leads to the neglect of the negative influence of the perception bias hidden in the visual observations. The hand-crafted semantic distribution in domestic environments causes the spurious association bias, while the semantic conflict bias arises due to the dynamic perspective changes. Biased visual perception significantly limits the generalization of the navigation strategy. In this article, we propose the U nbiased E mbodied V isual R epresentation ( UEVR ), which overcomes the perception biases using causal inference and cross-modality alignment. Specifically, we establish reasonable assumptions about confounders for multi-object features through our proposed Unbiased Causal R-CNN framework and eliminate the spurious associations bias through B ackdoor I ntervention C ausal A djustment ( BICA ) module during navigation. To overcome the dynamic-view bias hidden in 2D image features, we propose to employ the cross-modality alignment mechanism with the Geometric Consistency ( GeoCon ) to encode 3D geometry prior into the 2D representations. Finally, we design a modular ObjectNav framework integrated with UEVR named Causal-ObjectNav , which consists of the corner-based scene exploration module and target object discrimination module. Extensive experiments on the MP3D and HM3D datasets demonstrate the superiority of the unbiased navigation model over existing ObjectNav methods.

Source: ACM Transactions on Multimedia Computing Communications and Applications

DOI: 10.1145/3760261

Authors: Jiaxu Kang, Bolei Chen, Ping Zhong, Yifei Wang, Haonan Yang, Yu Sheng

Institution: Central South University

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: National Natural Science Foundation of China, Central South University

Awards: G1737521470, G5305027673