Work

Knowledge distillation and fine-tuning for corrosion and salt damage classification

Year: 2025

Type: article

Abstract: This paper proposes a classification model based on a corrosion and salt damage dataset of civil structures: ViT is used as the teacher model, while MobileNetV2 and MobileNetV3 are used as student models. The second-best model is obtained by improving the classifier and setting different ratios of fine-tuning layers under frozen versus non-frozen classification layers. This model is added with an improved attention mechanism to get the optimal model. The final results show that using ViT with the second weight and MobileNetV2 gives higher Accuracy and Weighted-f1 value, where heat maps generated by applying Grad-CAM reflect that it can generally identify the damage location. The optimal model obtained by choosing a fine-tuning strategy that freezes the classification layers and retrains 15% of the top feature layers can achieve an Accuracy and Weighted-f1 score of above 0.94, better than many advanced deep learning architectures using pre-trained weights.

Source: Ain Shams Engineering Journal

DOI: 10.1016/j.asej.2025.103895

Authors: Ziheng Zhao, Elmi Bin Abu Bakar, Norizham Bin Abdul Razak, Mohammad Nishat Akhtar

Institution: Universiti Sains Malaysia

Language: English

Topic: Infrastructure Maintenance and Monitoring

Subfield: Civil and Structural Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,200

Funder: Universiti Sains Malaysia