Temporal pattern attention for multivariate time series forecasting

Shih, Shun-Yao; Sun, Fan-Keng; Lee, Hung-yi

doi:10.1007/s10994-019-05815-0

Cited by 629 publications

(215 citation statements)

References 21 publications

Supporting

Mentioning

215

Contrasting

Order By: Relevance

“…To obtain accurate forecasting of non-linear time series, such as the prediction of infectious disease, it is crucial to model the long-term dependency in time series data. The periodic patterns spanning multiple time steps are difficult for a typical machine learning method to identify, but this can be achieved by the LSTM network [53,54]. The results in Table 2 showed that the setting of the time step affected the model performance and the model obtained got the best parameters when the time step was set to 12, which suggested that the underlying mechanism of dengue outbreak may be related to long-term climate change.…”

Section: Discussionmentioning

confidence: 99%

Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method

et al. 2020

IJERPH

View full text Add to dashboard Cite

Dengue fever (DF) is one of the most rapidly spreading diseases in the world, and accurate forecasts of dengue in a timely manner might help local government implement effective control measures. To obtain the accurate forecasting of DF cases, it is crucial to model the long-term dependency in time series data, which is difficult for a typical machine learning method. This study aimed to develop a timely accurate forecasting model of dengue based on long short-term memory (LSTM) recurrent neural networks while only considering monthly dengue cases and climate factors. The performance of LSTM models was compared with the other previously published models when predicting DF cases one month into the future. Our results showed that the LSTM model reduced the average the root mean squared error (RMSE) of the predictions by 12.99% to 24.91% and reduced the average RMSE of the predictions in the outbreak period by 15.09% to 26.82% as compared with other candidate models. The LSTM model achieved superior performance in predicting dengue cases as compared with other previously published forecasting models. Moreover, transfer learning (TL) can improve the generalization ability of the model in areas with fewer dengue incidences. The findings provide a more precise forecasting dengue model and could be used for other dengue-like infectious diseases.

show abstract

Section: Discussionmentioning

confidence: 99%

Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method

et al. 2020

IJERPH

View full text Add to dashboard Cite

show abstract

“…The application of attention modules in time series models is also an active field of research. The works in [41,42] proposed the application of attention in the LSTM architecture for time series prediction. Attention models have also been used with CNN and time series data.…”

Section: Of 29mentioning

confidence: 99%

Multi-Step Short-Term Wind Speed Prediction Using a Residual Dilated Causal Convolutional Network with Nonlinear Attention

Kumar

Tzou

2020

Energies

View full text Add to dashboard Cite

Wind energy is the most used renewable energy worldwide second only to hydropower. However, the stochastic nature of wind speed makes it harder for wind farms to manage the future power production and maintenance schedules efficiently. Many wind speed prediction models exist that focus on advance neural networks and/or preprocessing techniques to improve the accuracy. Since most of these models require a large amount of historic wind data and are validated using the data split method, the application to real-world scenarios cannot be determined. In this paper, we present a multi-step univariate prediction model for wind speed data inspired by the residual U-net architecture of the convolutional neural network (CNN). We propose a residual dilated causal convolutional neural network (Res-DCCNN) with nonlinear attention for multi-step-ahead wind speed forecasting. Our model can outperform long-term short-term memory networks (LSTM), gated recurrent units (GRU), and Res-DCCNN using sliding window validation techniques for 50-step-ahead wind speed prediction. We tested the performance of the proposed model on six real-world wind speed datasets with different probability distributions to confirm its effectiveness, and using several error metrics, we demonstrated that our proposed model was robust, precise, and applicable to real-world cases.

show abstract

“…To further solve time series prediction problems, some researchers have introduced attention mechanisms into deep neural networks [ 37 , 38 , 39 ]. Inspired by [ 40 ], a temporal pattern attention (TPA)-based LSTM is applied to the DL model to capture inherent correlations among random numbers in this paper. Compared with the typical attention mechanism, the provided TPA mechanism can learn the hidden correlations in the intricate time series data with advantage.…”

Section: Experimental Schemementioning

confidence: 99%

“…In the temporal pattern attention mechanism [ 40 ], given the previous LSTM hidden states H = ( h 1 , h 2 , …, h t-1 ) ∈ ℝ m ×( t- 1) , a convolutional neural network (CNN) is used to improve the predictive performance of the model by employing CNN filters on the row vectors of H . The CNN has k filters, each of which has length of T .…”

Section: Experimental Schemementioning

confidence: 99%

Deep Learning-Based Security Verification for a Random Number Generator Using White Chaos

Zhang

Sang

et al. 2020

Entropy

View full text Add to dashboard Cite

In this paper, a deep learning (DL)-based predictive analysis is proposed to analyze the security of a non-deterministic random number generator (NRNG) using white chaos. In particular, the temporal pattern attention (TPA)-based DL model is employed to learn and analyze the data from both stages of the NRNG: the output data of a chaotic external-cavity semiconductor laser (ECL) and the final output data of the NRNG. For the ECL stage, the results show that the model successfully detects inherent correlations caused by the time-delay signature. After optical heterodyning of two chaotic ECLs and minimal post-processing are introduced, the model detects no patterns among corresponding data. It demonstrates that the NRNG has the strong resistance against the predictive model. Prior to these works, the powerful predictive capability of the model is investigated and demonstrated by applying it to a random number generator (RNG) using linear congruential algorithm. Our research shows that the DL-based predictive model is expected to provide an efficient supplement for evaluating the security and quality of RNGs.

show abstract

Temporal pattern attention for multivariate time series forecasting

Cited by 629 publications

References 21 publications

Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method

Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method

Multi-Step Short-Term Wind Speed Prediction Using a Residual Dilated Causal Convolutional Network with Nonlinear Attention

Deep Learning-Based Security Verification for a Random Number Generator Using White Chaos

Contact Info

Product

Resources

About