MeteoFormer: The Accuracy–Faithfulness Trade-Off of Built-In Attention in Meteorological Forecasting
Year: 2026
Type: article
Abstract: Built-in attention is often presented as a free explanation of what a forecasting transformer relies on, but the claim is rarely tested. We test it on MeteoFormer, a patch-based transformer with single-head variable and temporal attention that forecasts four weather variables 24 h ahead from 96 h of hourly input on the Jena and Beijing PRSA datasets. Against ten baselines trained under one protocol with five seeds, MeteoFormer has the lowest mean absolute error on Jena (3.025 against 3.107 for Crossformer, Diebold–Mariano p < 0.05 against every baseline after Holm correction) and is level with Crossformer on Beijing (4.151 against 4.138, p = 0.72), while Crossformer has the lower squared error, a difference traced to humidity. The model depends heavily on the latest observations: losing the last 16 h raises its Jena error by 73% and drops it from first to sixth place. The explanation claim fails. The fidelity of the variable attention does not differ from zero (0.012 on Jena, 0.046 on Beijing), far below SHAP on the same model; the temporal attention is almost uniform although the forecast depends mainly on the last 16 h; and the configuration selected for accuracy has the least faithful attention (0.012 against 0.181 with instance normalization, p = 0.001), but the more faithful configuration is less accurate (Jena MAE 3.195 against 3.025) and its attention remains below SHAP.
Source: Forecasting
Authors: Moldir Ashimova, Nurlan Tashatov, Azamat Serek
Language: English
Subfield: Atmospheric Science
Field: Earth and Planetary Sciences
Domain: Physical Sciences
SDG: Climate action
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7215929351