Year: 2021
Type: article
Abstract: Multivariate time-series forecasting derives key seasonality from past patterns to predict future time-series. In the industrial sector, multi-step forecasting is crucial because a continuous perspective leads to make more effective decisions. However, because it depends on previous prediction values, multi-step forecasting is highly unstable. To mitigate this problem, we introduce a novel model, named stacked dual attention neural network (StackDA), based on an encoder-decoder. In dual attention, the initial attention is for the time dependency between the encoder and decoder, and the second attention is for the time dependency in the decoder time steps. We stack dual attention to stabilize the long-term dependency and multi-step forecasting problem. We add an autoregression component to resolve the lack of linear properties because our method is based on a nonlinear neural network model. Unlike the conventional autoregression model, we propose skip autoregression to deal with multiple seasonalities. In addition, we propose a denoising training method to take advantage of both teacher forcing and without teacher forcing methods. We adopt multi-head fully connected layers for variable-specific modeling because of our multivariate time-series data. We add positional encoding to provide the model with time information, so we can more accurately recognize seasonality with the model. We compare our model performance with that of machine learning and deep learning models to verify our approach. Finally, we conduct various experiments, such as an ablation study, find seasonality test, and stack attention test, to demonstrate the performance of StackDA.
Source: IEEE Access
Authors: Jungsoo Hong, Jinuk Park, Sanghyun Park
Institution: Yonsei University
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,750
Awards: G2059405956, G7338744670
OpenAlex ID: W3210128914