Limited-data transfer learning for semantic segmentation and pre-labeling of 3D shell construction LiDAR scans
Year: 2026
Type: article
Abstract: This study assesses transformer-based 3D semantic segmentation models for detecting structural components in terrestrial laser scans, given that no training data currently exists for shell construction sites. Manual annotation of 3D point clouds is expensive, yet high-quality labels remain essential for supervised computer vision and validation. Automated pre-labeling can cut down annotation effort by shifting human tasks from exhaustive labeling to targeted verification and correction, assuming models can robustly identify the most common structural elements. We designed a three-stage evaluation protocol covering (i) supervised learning, (ii) cross-domain generalization, and (iii) transfer learning with limited labeled data in the target domain to test model generalization in this context. Three transformer architectures (Point Transformer V2, Point Transformer V3, and Swin3D) are evaluated using four established indoor datasets (S3DIS, ScanNetV2, Structured3D, and VASAD) and a custom domain-specific dataset of annotated construction scenes. Training only on the limited construction dataset results in weak generalization. In contrast, pretraining on loosely related synthetic data and fine-tuning on a minimal number of labeled construction scenes enable reliable segmentation of core building components. A sensitivity analysis also showed that just 12 samples are sufficient to calibrate a pretrained model to a specific building type. The models perform well despite differences between synthetic training data and noisy real-world scans. Among the evaluated architectures, Swin3D delivers the best performance, with +18% mIoU improvement through general pretraining, while PTv3 converges faster with fewer target-domain samples. These findings suggest that transfer learning with limited labeled construction data offers a practical foundation for scalable pre-labeling workflows and human-in-the-loop applications in architecture, engineering, and construction.
Authors: Lukas Rauch, Thomas Braml
Institution: Universität der Bundeswehr München
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7154991601