Work

Map-based Modular Approach for Zero-shot Embodied Question Answering

Year: 2024

Type: conference-paper

Abstract: Embodied Question Answering (EQA) serves as a benchmark task to evaluate the capability of robots to navigate within novel environments and identify objects in response to human queries. However, existing EQA methods often rely on simulated environments and operate with limited vocabularies. This paper presents a map-based modular approach to EQA, enabling real-world robots to explore and map unknown environments. By leveraging foundation models, our method facilitates answering a diverse range of questions using natural language. We conducted extensive experiments in both virtual and real-world settings, demonstrating the robustness of our approach in navigating and comprehending queries within unknown environments.

Source: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

DOI: 10.1109/iros58592.2024.10801314

Authors: Koya Sakamoto, Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, Motoaki Kawanabe

Institutions: Kyoto University, Sony Semiconductor Solutions Corporation (Japan), The University of Tokyo, National Institute of Informatics, Advanced Telecommunications Research Institute International

Language: English

Topic: Robot Manipulation and Learning

Subfield: Control and Systems Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: green

Funders: Japan Society for the Promotion of Science, Precursory Research for Embryonic Science and Technology

Awards: Developing a 3D Spatially-Grounded Question Answering System for Real-World Environments, Quantification of Brain States and Individual Traits Based on Spatiotemporal Feature Learning of Resting-State EEG, Creation of datasets that combine 3D environmental and textual information