Work

Design of an integrated model for pothole detection and repair optimization using multimodal transformers and hybrid deep learning

Year: 2025

Type: article

Abstract: The detection and timely repair of potholes are crucial for maintaining road safety and minimizing vehicle damage. However, existing methods often suffer from limitations such as reliance on single-modal data, poor generalization across diverse environments, and suboptimal resource management. To address these challenges, we propose a comprehensive framework for enhanced pothole detection and repair optimization using advanced deep learning techniques. Our approach integrates four key methodologies: Multimodal Enhanced Pothole Detection with Person-Level Data (M-E-Pot holeNet), Hybrid Machine Learning-Deep Learning for Classification (Hybrid-Pot holeNet), Deep Reinforcement Learning for Pot hole Detection and Repair Optimization (DRL-Pot holeOpt), and Transfer Learning for Pothole Detection in Diverse Environments (TL-Pot holeAdaptNet). M-E-Pot holeNet employs a Self-Supervised Multimodal Transformer (SSMT) to fuse camera, accelerometer, and crowdsourced smartphone data, achieving robust detection with a 97 % accuracy and under 2 % false positive rate. Hybrid-Pot holeNet combines Graph Attention Networks (GAT) and XGBoost, modeling spatial road features to classify potholes with 95 % accuracy and an F1-Score of 0.92. DRL-Pot holeOpt uses Soft Actor-Critic (SAC) with Bayesian Optimization to efficiently schedule repair tasks, reducing repair costs by up to 20 % and crew travel time by 15–25 %. Finally, TL-Pot holeAdaptNet leverages Domain-Adversarial Neural Networks (DANN) to ensure cross-domain adaptability, with 90 % accuracy in new environments and a 40–50 % reduction in domain discrepancy. This multi-faceted approach addresses the limitations of previous work by providing scalable, real-time, and resource-optimized solutions for pothole detection and maintenance, offering significant improvements in accuracy, cost efficiency, and adaptability.

Source: Case Studies in Construction Materials

DOI: 10.1016/j.cscm.2025.e05431