Reg3D: Reconstructive Geometry Instruction Tuning for 3D Scene Understanding
Year: 2026
Type: conference-paper
Abstract: The rapid development of Large Multimodal Models (LMMs) has led to notable advances in 2D visual understanding, however, extending these capabilities to 3D scene comprehension remains a significant challenge. Existing methods primarily rely on text-only supervision, which lacks the geometric constraints essential for learning accurate and robust 3D spatial representations. To address this limitation, we introduce Reg3D, a novel Reconstructive Geometry Instruction Tuning framework that integrates geometry-aware supervision directly into model training. Unlike previous approaches that inject 3D information solely at the input level, Reg3D adopts a dual-supervision paradigm, utilizing 3D geometry both as input and as explicit training targets. We design complementary object-level and frame-level reconstruction tasks within a dual-encoder architecture to enforce geometric consistency and enhance spatial reasoning. Extensive experiments on four benchmark datasets: ScanQA, Scan2Cap, ScanRefer, and SQA3D demonstrate that with only RGB input, Reg3D outperforms existing methods, establishing a new paradigm for spatially grounded multimodal learning.
Institution: University of Manchester
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7155065398