Work

Empowering Robot Path Planning with Large Language Models: osmAG Map Topology & Hierarchy Comprehension with LLMs

Year: 2024

Type: conference-paper

Abstract: Large Language Models (LLMs) have demonstrated great potential in robotic applications by providing essential general knowledge. Mobile robots rely on map comprehension for tasks like localization and navigation. In this paper, we explore enabling LLMs to comprehend the topology and hierarchy of Area Graph, a text-based hierarchical, topometric semantic map representation utilizing polygons to demark areas such as rooms or buildings. Our experiments demonstrate that with the right map representation, LLMs can effectively comprehend Area Graph's topology and hierarchy. After straightforward fine-tuning, the LLaMA2 models exceeded ChatGPT-3.5 in mastering these aspects. Our dataset, dataset generation code, fine-tuned LoRA adapters can be accessed at https://github.com/xiefujing/LLM-osmAG-Comprehension.

Source: IEEE International Conference on Robotics and Biomimetics (ROBIO)

DOI: 10.1109/robio64047.2024.10907658

Authors: Fujing Xie, Sören Schwertfeger

Institution: ShanghaiTech University

Language: English

Topic: Robotics and Sensor-Based Localization

Subfield: Aerospace Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: green

Funders: Science and Technology Commission of Shanghai Municipality, ShanghaiTech University

Award: G1299713578