Investigating the Performance of the Attention Mechanism and the Interpretability in the Concrete Strength Prediction Model
Year: 2025
Type: article
Abstract: To address the limitations of traditional models in capturing complex features for concrete strength prediction, this study proposes a hybrid deep learning approach that integrates multiple attention mechanisms with gated recurrent units (GRU). The methodology employs a multi-scale validation framework, conducting three-dimensional validation across three datasets: the Kaggle standard dataset, the lightweight foam concrete dataset, and the self-compacting concrete dataset. Six attention mechanisms (SE attention, dot-product attention, self-attention, etc.) are comprehensively compared to optimise the GRU network structure. A Newton–Raphson-based optimiser (NRBO) enables hyperparameter adaptive tuning. Experimental results show significant improvements over the baseline GRU model: mean R2 increased by 6.99%, while RMSE and MAE decreased by 38.5% and 37.5%, respectively. SHAP interpretability analysis confirms that attention mechanisms effectively capture key parameters like SP and VMA in the self-compacting concrete dataset. Based on the findings, this study recommends using self-attention for datasets smaller than 200 samples and selecting the higher-accuracy model between self-attention and stacked attention mechanisms for larger datasets.
Source: Buildings
Authors: Ziang Jia, Noor Azline Mohd Nasir, Nabilah Abu Bakar
Institution: Universiti Putra Malaysia
Language: English
Subfield: Civil and Structural Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W4414349547