Work

Answerability Fields: Answerable Location Estimation via Diffusion Models

Year: 2024

Type: conference-paper

Abstract: We propose Answerability Fields (AnsFields), a novel approach for predicting the answerability of questions at different locations within indoor environments. AnsFields is represented as a map, where each grid’s score reflects how well a question can be answered using the panoramic image at that location. Using a 3D question-answering dataset, we construct comprehensive AnsFields covering diverse scenes from ScanNet. Additionally, we employ a diffusion model to infer AnsFields from a scene’s top-down view image and the question. We then conduct 3D question-answering using these predicted AnsFields and achieve a 24% improvement in accuracy over the standard 3D-QA method. Our results demonstrate the importance of object locations for answering questions in the environment, highlighting the potential of AnsFields for applications in robotics, augmented reality, and human-robot interaction.

Source: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

DOI: 10.1109/iros58592.2024.10802662

Authors: Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, Koya Sakamoto, Motoaki Kawanabe

Institutions: Sony Semiconductor Solutions Corporation (Japan), The University of Tokyo, National Institute of Informatics, Kyoto University, Advanced Telecommunications Research Institute International

Language: English

Topic: Multimodal Machine Learning Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: Japan Society for the Promotion of Science

Award: Developing a 3D Spatially-Grounded Question Answering System for Real-World Environments