VISTA+: Mesh-based single-view novel view synthesis and CAD-based foreground composition for synthetic driving-scene generation
Year: 2026
Type: article
Abstract: Synthetic data has emerged as a promising solution for alleviating data scarcity in autonomous driving, especially in cold-start and corner-case scenarios. However, existing synthetic data generation methods still face a trade-off between photorealistic fidelity and engineering scalability. This paper proposes VISTA+, a framework for synthetic driving-scene generation that combines mesh-based single-view novel view synthesis (NVS) of static backgrounds with computer-aided design (CAD)-based foreground rendering and foreground-background composition. Unlike multi-view neural reconstruction methods, VISTA+ avoids large-scale multi-view scene reconstruction, while still requiring camera calibration and source-to-target relative poses, thereby reducing engineering complexity. Specifically, we improve mesh-based background NVS by incorporating foreground inpainting, dense depth supervision, structural pretraining, and joint depth-offset prediction. Dynamic foreground objects are rendered using CAD assets under estimated environmental lighting and are then integrated with generated backgrounds through image harmonization. Experiments on KITTI and nuScenes demonstrate that VISTA+ outperforms representative single-view NVS baselines in terms of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Furthermore, synthetic data generated by VISTA+ consistently improves the performance of multiple downstream three-dimensional (3D) object detection models, demonstrating the downstream utility and functional realism of the generated driving scenes.
Authors: Song Zhang(Conceptualization, Data curation, Methodology, Software, Validation, Writing – original draft), Ruibo Wang(Conceptualization, Data curation, Methodology, Software, Validation, Writing – original draft), Jian Li(Methodology, Writing – review & editing), Wenxiu Shi(Methodology, Writing – review & editing), Huiping Zhu(Methodology, Writing – review & editing), Ruo Jia(Methodology, Writing – review & editing), Ziyu Lin(Methodology, Writing – review & editing), Haiquan Liu(Funding acquisition, Supervision)
Institutions: Chalmers University of Technology, Tsinghua University
Language: English
Topic: Advanced Vision and Imaging
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Awards: G440879909, G8529277840
OpenAlex ID: W7236747913
