A Model-Class Comparison of Large Language Models for Translating Chinese Route Instructions into Robot Command Sequences
Year: 2026
Type: article
Abstract: Advances in large language models (LLMs) have accelerated the real-world deployment of embodied intelligent robots, supporting intuitive human–robot interaction without specialized robotics expertise. However, few systematic assessments examine how well LLMs translate natural-language route instructions into executable robot command sequences, particularly for non-English instructions and across deployment-relevant model classes. In this paper, a Chinese route-instruction corpus was constructed, which comprised 212 instructions collected from six native speakers across 50 procedurally generated town maps. Each was annotated with a ground-truth command sequence verified by simulation to reach its goal, and a 100% performance ceiling was established. Using this corpus, 12 LLMs spanning three capability tiers—cloud proprietary models, mid-sized open-weight models, and small open-weight models—were evaluated under three system prompts differing only in the number of worked examples (zero, one, and three). Open-weight models in the 3–9B range reached at most 9.2% of goals, while proprietary cloud models reached 80.0%, a substantial gap that increasing the number of worked examples did not measurably narrow. A paired analysis across the three prompt conditions detected no reliable effect of demonstration count: pooled path_reach rose modestly from 25.0% under zero-shot prompting to 27.2% with one example and 27.7% with three examples (three versus zero examples: odds ratio: 1.15; 95% CI: 0.98–1.34). None of these differences was significant after multiple-comparison correction (Holm-adjusted p ≥ 0.247). Low-Rank Adaptation (LoRA) fine-tuning on 126 examples raised mid-sized open-weight models to 27.7–35.4%, and the gain spanned both output-format compliance and route competence.
DOI: 10.3390/jsan15060091
Authors: Ruizhe Yang(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing), Yuki Nishimura(Conceptualization, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Writing – review & editing)
Institution: Hangzhou Dianzi University
Language: English
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,413
OpenAlex ID: W7236081145
