Work

Beyond V2X: A Review of Large Language Models in In-Vehicle Autonomous Driving Systems

Year: 2025
Type: preprint
Abstract: Large Language Models (LLMs) are emerging as key enablers of intelligent in-vehicle systems, supporting conversational interfaces, diagnostics, and decision support in autonomous driving. While prior surveys have focused on LLMs in vehicular infrastructure, little attention has been paid to their integration within the in-vehicle environment-where safetycritical decisions occur. This paper offers a comprehensive review of LLM applications inside vehicles, presenting a structured taxonomy and analyzing recent developments across academia and industry. We identify key deployment challenges, including latency, determinism, real-time reliability, explainability, and compatibility with legacy automotive protocols. In parallel, we highlight critical security and privacy threats such as prompt injection, adversarial manipulation, data leakage, and model inversion, which are amplified in over-the-air and safety-critical contexts. We further argue that unsafe in-vehicle LLM behavior can propagate into the broader Intelligent Transportation System (ITS), increasing systemic risk. To mitigate these concerns, we propose a targeted research agenda and evaluation framework for LLMs in vehicular settings. Our goal is to bridge a key gap in existing literature and guide the secure, robust, and humanaligned deployment of LLMs in automotive systems.
Author: Ayan Roy
Language: English
Open Access status: gold
OpenAlex ID: W4413358182
Cites:
Cited by:
Related to: 10