Generalizable 3D Gaussian Splatting Enabled Immersive Video Communications with Semantic Coding
Year: 2025
Type: conference-paper
Abstract: A core challenge in immersive video communication is to achieve real-time 3D reconstruction and rendering while ensuring efficient multi-view source coding. Recent advances in feed-forward 3D Gaussian Splatting (3DGS) have enabled efficient 3D reconstruction from sparse stereo views, while semantic communication offers higher compression efficiency by leveraging semantic priors. In this work, we present SC-GSNet, a unified end-to-end framework that jointly performs semantic-aware multi-view video compression and generalizable 3D Gaussian Splatting. Our approach introduces a disparity-guided semantic codec capable of parallel encoding for stereo views with cross-view contextual interaction, significantly improving compression efficiency. Furthermore, real-time estimation of 3D splatting parameters from compressed semantic features is challenging due to the computational constraints of immersive applications. To address this, we propose a lightweight Gaussian parameter predictor that directly estimates 3D splatting attributes from decoded semantic features. Extensive experiments on real-world human datasets demonstrate that SC-GSNet achieves state-of-the-art rate-rendering performance, reducing BD-rate by up to 75.2% compared with multi-view video coding standard, while maintaining real-time inference at about 19 fps. The model also exhibits strong generalizability across unseen scenes, offering a scalable, low-latency solution for immersive communications.
Authors: Dingxi Yang, Zhijin Qin
Institution: Tsinghua University
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Awards: G2415068989, G3421198822
OpenAlex ID: W7090463929