Get started
Work

ChronoSHAP: Revealing temporal patterns from transformers and LTSF-linear models in time series forecasting

Year: 2025
Type: article
Abstract: The surge of Transformers' popularity has motivated their usage in a multitude of domains with sequential data, as can be seen in time series forecasting. It is assumed that Transformers can easily handle time dependencies, encouraged by their ability to handle contextual information from texts. However, this hypothesis has been challenged by the uncovering of performance issues that arise from comparing Transformers with simpler models, such as the autoregressive LTSF-Linear. Moreover, no attempts have been made to extract interpretable knowledge on the patterns learned by these models that can help understand why they underperform in time series forecasting. This study proposes ChronoSHAP, an explainable AI approach based on Shapley additive explanations (SHAP) combined with a novel aggregation procedure along the temporal dimension. ChronoSHAP aims to provide visual explanations of the global temporal dependencies in time series forecasting, using them to reveal why Transformers are underperforming. ChronoSHAP is used with two Transformers and two LTSF-Linear models across a variety of datasets in a multivariate, multi-step prediction setting. The explanations are analyzed to extract interpretable knowledge about temporal patterns compared to those learned by simpler models with better performance, and those observed in the data. The results show that Transformers are prone to focus on the recent past, while tending to disregard or misunderstand the long-term dependencies observed in the data, while the simpler LTSF-Linear models learn clearer long-term dependencies more consistently, which explains why Transformers struggle to keep up with simpler models' performance.
DOI: 10.1016/j.knosys.2025.114802
Language: English
Open Access status: hybrid
Estimated APC paid: $3,430
OpenAlex ID: W4415892257
FWCI: 2.04
Cites: 31
Cited by: 5
Related to: