Construction Safety Hazard Identification and Corrective Action Generation Based on a Fine-Tuned Vision–Language Model
Year: 2026
Type: article
Abstract: To address the limited ability of general-purpose large vision–language models (VLMs) to identify construction safety hazards and the poor integration of identification results with corrective actions, this study proposes a method for construction safety hazard identification and corrective action generation using a fine-tuned VLM. For model training, an image-text dataset covering 11 categories of unsafe behaviors and 10 categories of unsafe physical conditions is established. Qwen3-VL-8B-Instruct was used as the base model for domain-specific fine-tuning with low-rank adaptation (Lo-RA). Combined with structured prompts, the fine-tuned model automatically identifies multiple categories of safety hazards and generates corresponding corrective actions conditioned on the identification results, establishing an integrated workflow for intelligent hazard identification and automated corrective action recommendation. The fine-tuning updates merely around 0.2532% of all model parameters. Under this setting, the model reaches mean per-category precision, recall and F1 values of 0.890, 0.842 and 0.861, respectively. The model showed marked improvements in identifying hazards involving inadequate personal protection and missing safety facilities. Still, it struggles to separate visually similar hazard classes and interpret complicated site scenes. By linking hazard labels, corrective actions and supporting standards, the proposed method converts image-based identification results into structured content for preparing corrective action documentation, providing intelligent support for construction site inspections.
Source: Buildings
Authors: Yiyi Ye, Qianmai Luo, Tongtong Zhao
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7225823748