Work

Hybrid Visual Perception and LLM Framework for Service Robot Planning and Navigation

Year: 2026
Type: preprint
Abstract: In the last decade, service robots have gained significant attention for their role in supporting humans across various applications. To operate effectively and adaptively, service robots require environmental perception, natural-language understanding, and planning and navigation capabilities. However, traditional planning frameworks struggle to provide the required flexibility. In contrast, end-to-end multimodal systems based on vision-language models (VLMs) require substantial computational resources, making them unsuitable for deployment on embedded robotic platforms. This paper proposes a hybrid framework that combines fast, deep-learning-based visual perception with the reasoning capabilities of a large language model (LLM) to enhance task comprehension, planning, and navigation for service robots. By decoupling vision from language reasoning, we avoid the complexity of a unified vision-language model, allowing the proposed framework to be deployed on a lightweight embedded platform with substantially lower latency than the VLM baseline. Experimental results show a 96\% task completion rate in structured mission tests and 84\% overall success in hybrid perception tasks. Moreover, the proposed approach achieves 3× to 20× faster inference than VLM-based baselines under both API-based and local deployment settings. Real-robot deployment confirmed the successful transfer of the full perception-to-action pipeline to the physical platform.
Language: English
Open Access status: green
OpenAlex ID: W7225876787
Cites: 9
Cited by:
Related to: 10