Knowledge-Distilled Multi-Task Model With Enhanced Transformer and Bidirectional Mamba2 for Air Quality Forecasting
Year: 2025
Type: article
Abstract: Accurate air quality forecasting is crucial for public health and environmental policy, yet existing deep learning models often suffer from high computational costs and inadequate modeling of temporal dependencies and multi-pollutant interactions. This paper proposes a novel deep learning framework that integrates an Enhanced Transformer with Bidirectional Mamba2, optimized throughmulti-task learningandknowledge distillation. Using a teacher–student paradigm, the teacher model captures rich temporal semantics, while the lightweight student model retains predictive accuracy with reduced inference costs. Key innovations include a hybrid architecture combining multi-scale global-local attention and long-range dependency modeling, a regression-specific knowledge distillation approach withsoft target smoothingandintermediate representation transfer, and an end-to-end multi-task design for joint forecasting of multiple pollutants. Extensive experiments on real-world datasets fromGuangzhou, Chengdu, andBeijing(2018–2022) across four key pollutants (PM2.5, PM10, NO2, SO2) demonstrate that our model significantly outperforms baseline methods, with the student model maintaining accuracy within 5% of the teacher while requiring fewer parameters. These results highlight the framework’s potential for accurate, scalable, and efficient air quality forecasting in real-time and resource-constrained environments.
Source: IEEE Access
Authors: Zi-Ang Xie, Chee-Onn Chow, Joon Huang Chuah, Wong Jee Keen Raymond
Institution: University of Malaya
Language: English
Subfield: Environmental Engineering
Field: Environmental Science
Domain: Physical Sciences
SDG: Climate action
Open Access status: gold
Estimated APC paid: $2,075
OpenAlex ID: W4412984978