Multi-horizon sequential prediction via informer ensemble with ProbSparse attention: A cross-domain study on four large-scale temporal datasets
Year: 2026
Type: article
Abstract: Accurate multi-horizon forecasting of non-stationary time series with long-range dependencies remains a fundamental challenge in sequential data modeling. Existing deep learning approaches often struggle with computational complexity, structural breaks, and initialization sensitivity when processing long input sequences. This paper proposes InformerEnsemble, a novel ensemble framework built upon the Informer architecture that integrates ProbSparse self-attention, a self-distillation mechanism, and multi-scale temporal embeddings to efficiently capture complex temporal dynamics across multiple prediction horizons ( $$h = 1, 3, 6$$ h = 1 , 3 , 6 , and 12 timesteps). We conduct comprehensive experiments on four large-scale monthly panel datasets spanning nearly four decades (1986–2025), each comprising ten multivariate features. The proposed framework is systematically benchmarked against ten competitive baselines including ARIMA, LSTM, GRU, Vanilla Transformer, TCN, N-BEATS, PatchTST, Autoformer, FEDformer, and the original Informer. Empirical results demonstrate that InformerEnsemble achieves the best average ranking (2.4 out of 11 models) across 16 evaluation scenarios (4 datasets $$\times $$ × 4 horizons), winning 7 scenarios outright and reducing mean absolute error (MAE) by up to 16.9% relative to the ARIMA baseline at the 12-month forecast horizon on the US dataset (corresponding to a MAPE margin of 7.6% over the best competing baseline). The architecture employs a three-path ensemble design with distinct encoder depths to capture short-, medium-, and long-scale temporal patterns, fused via learnable constrained optimal weights. ProbSparse attention reduces computational complexity from $$\mathcal {O}(L^2)$$ O ( L 2 ) to $$\mathcal {O}(L \log L)$$ O ( L log L ) , enabling tractable processing of long sequences while preserving the most informative attention patterns. Comprehensive ablation studies validate the contribution of each architectural component, including the self-distillation layer, multi-scale temporal embeddings, and ensemble fusion mechanism. SHAP-based interpretability analysis quantifies the marginal contribution of each input feature across all datasets, revealing dataset-specific feature importance patterns. Cross-domain zero-shot transfer experiments further demonstrate the generalization capability of learned representations across heterogeneous temporal datasets. This work provides a rigorous, reproducible, and interpretable deep learning framework for efficient long-sequence multi-horizon forecasting with significant implications for Transformer-based time series modeling, ensemble learning strategies, and cross-domain transfer in sequential prediction tasks.
Authors: Xiang Ma, Piaopiao Huang, Lingli Qing
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,990
OpenAlex ID: W7203924498