Work

HyVDAR: A semi-supervised framework for high-fidelity 3D object reconstruction from sparse multi-view point clouds in virtual reality applications

Year: 2026

Type: article

Abstract: In virtual reality (VR) applications, high fidelity 3D meshes are needed for immersive experiences, and this calls for high fidelity reconstruction of 3D objects from sparse multi-view point clouds. There are several approaches, like PointMVSNet and NeRF, which require dense supervision or are computationally expensive and are not very suitable for the use of semi-supervised frameworks with limited labeled data. To address the above gaps, in this research study, a novel semi-supervised approach, Hybrid View-Consistent Diffusion-Augmented Reconstruction (HyVDAR), is proposed for the reconstruction of 3D meshes for VR from sparse multi-viewpoint clouds with limited labeled data while ensuring the highest geometric fidelity and rendering efficiency. HyVDAR has three novel components to the above pipeline: (1) self-supervised cross-view geometric consistency trained with transformers is used for alignment, denoted as View-Consistent Feature Propagation (VCFP); (2) conditional denoising diffusion models are used for patching the missing parts in sparse point clouds, denoted as Diffusion-Augmented Point Cloud Completion (APC); (3) graph neural networks with pseudo-labeling are used for refining the coarse meshes with both labeled and unlabeled data through semi-supervised learning, denoted as Semi-supervised Mesh Refinement (SMR). Trained on a combination of ShapeNet (labeled) and Pix3D (unlabeled) dataset, HyVDAR improves over the best competitive model (PointMVSNet) on sparse inputs by decreasing Chamfer Distance (CD) from 0.013 to 0.010 (25% improvement), decreases the labeled data requirements by 60% compared to supervised baseline models and achieves real-time VR rendering at 65 FPS. HyVDAR shows a good reconstruction quality and computational efficiency compared to the benchmark methods, and it can be used as a general-purpose tool for high-fidelity 3D modeling in virtual reality, which will be beneficial for gaming, digital avatar and immersive experience content creation.

Source: Egyptian Informatics Journal

DOI: 10.1016/j.eij.2026.100906

Authors: Shanglin Li, Juan Xiao

Institutions: Hunan University, Xiangnan University

Language: English

Topic: 3D Shape Modeling and Analysis

Subfield: Computational Mechanics

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,200

Funder: Education Department of Hunan Province