AR -based MEP progress monitoring using BIM and synthetic data
Year: 2026
Type: article
Abstract: Manual inspection of indoor construction sites is labor-intensive and prone to human error. Currently, automating this process using deep learning is limited by the lack of large-scale, annotated datasets for complex mechanical, electrical, and plumbing (MEP) environments. This paper proposes an indoor construction progress monitoring framework that integrates a Building Information Modeling (BIM)-driven synthetic data pipeline with augmented reality (AR). To address the issue of limited data, we created an automated pipeline with NVIDIA Isaac Sim to generate the MEP-SEG dataset. This dataset contains 8751 photorealistic synthetic images with automated pixel-level annotations. A YOLOv8 instance segmentation model trained via a domain adaptation strategy using this synthetic data and a 20% of available real-world data, achieved a Mean Average Precision (mAP 50 ) of 79.2% ± 1.4% on real site images. The model demonstrated resilience to varied occlusion and lighting; however, performance remains sensitive to low-contrast conditions below 300 lx. We deployed the trained model on a HoloLens 2 AR headset to provide a visual analysis of discrepancies between the as-built state and the BIM design at approximately two frames per second. A user study with 21 construction professionals indicates the system’s potential for practical adoption, yielding positive feedback regarding usability and perceived efficiency. Our study shows that synthetic data can reduce the real-world annotation burden by 80% for certain construction tasks. This provides a framework to narrow the “reality gap” for automated on-site progress monitoring.
Source: Results in Engineering
Language: English
Subfield: Building and Construction
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,940
Funders: SPIE, École Centrale de Lyon
OpenAlex ID: W7149928980