Year: 2025
Type: article
Abstract: Structural breaks and volatility clustering are fundamental challenges in time series analysis. We propose CQEformer, an encoder-only Transformer variant for time-series modeling that addresses these challenges via two complementary innovations. First, the causal residual embedding (CRE) ensures temporal causality and improves local adaptivity to abrupt structural changes. Second, the query-enhanced multi-head self-attention (QEAttention) incorporates multi-order moment statistics and entropy to guide attention dynamically toward high-volatility regions while preserving global dependence structures. For parameter optimization, we derive analytical gradients for all components and update them using the Adam stochastic optimization algorithm. Empirical evaluations on financial time series datasets and the public Traffic dataset show that CQEformer consistently outperforms established baselines, including LSTM, GRU, TCN, and the standard Transformer. Time-window sensitivity analyses demonstrate the robustness of the framework, while ablation studies further confirm that the proposed modules are complementary and contribute to improved forecasting performance across different volatility regimes.
Source: Mathematics
DOI: 10.3390/math13233750
Authors: Yuze Tao, Lu Li
Institution: Shanghai University of Engineering Science
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
Awards: G3294696819, G6265973807
OpenAlex ID: W4416596007