Joint Camera-LiDAR Scene Synthesis and Perception for Autonomous Driving
Year: 2025
Type: article
Abstract: The advancement of autonomous driving and embedded AI systems has intensified the need for large-scale, richly annotated multimodal datasets encompassing RGB images, semantic labels, and 3D LiDAR data. Manual collection and annotation of such datasets remain costly and time-consuming, especially when temporal and cross-modal consistency is required. The proposed method introduces Joint Camera-LiDAR Scene Synthesis and Perception (JCLSP), a unified generative framework that simultaneously synthesizes photorealistic RGB images, semantic segmentation maps, and LiDAR range images through a compact and optimized diffusion process. Unlike prior approaches that employ separate diffusion branches, JCLSP fuses image and LiDAR modalities early in the pipeline and leverages a shared latent space for coherent multimodal generation. The architecture integrates three key elements: BKSDM, which streamlines the diffusion process by eliminating redundant blocks, a joint image-LiDAR diffusion module that applies the BKSDM framework to enable depth-aware synthesis with geometric fidelity, and modality-specific decoders that extract semantic masks, LiDAR range images, and image scenes from the shared latent representation. Experimental results on synthetic datasets indicate that JCLSP captures meaningful cross-modal correlations and preserves spatial features. By generating joint representations from camera and LiDAR views along with semantic segmentation annotations, the method demonstrates promising potential for cross-modal representation learning with labeled data.
Source: IEEE Access
Institution: Manipal Academy of Higher Education
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,075
OpenAlex ID: W4414404722