LATTE: Language-Driven Construction of 3D Driving Scenes for Closed-Loop AD Simulation
Year: 2026
Type: conference-paper
Abstract: Closed-loop simulation is central to autonomous driving (AD) safety evaluation, yet it remains difficult to turn natural-language requests into valid 3D test scenarios. Existing language-driven methods either operate in pixel space without explicit scene instantiation, or rely on fixed asset banks and coarse object placement that are insufficient for open-world, test-oriented scene construction. We present LATTE, a language-driven framework for constructing explicit 3D driving scenes from open-world scene requests. LATTE decomposes the problem into two coupled parts: Image-Grounded Asset Construction (IGAC), which acquires insertion-ready 3D assets for missing objects from visual evidence, and Structurally Coherent Layout Optimization (SCLO), which turns language-guided initial placements into simulation-suitable layouts under physical and semantic constraints. The resulting edits are written back to a real-scene-reconstructed simulator and evaluated through closed-loop AD rollouts. Experiments show that LATTE not only supports open-world asset insertion and more coherent scene layout, but also produces behaviorally effective test cases that measurably alter downstream AD performance. The project page is at here.
Authors: Zaiyi Hu, Haifeng Wu, Yiyi Liao, Lixin Duan, Wen Li
Language: English
Subfield: Automotive Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7225426822