Stacked Denoising Auto-Encoders for Short-Term Time Series Forecasting

Romeu, Pablo; Zamora-Martínez, Francisco; Botella-Rocamora, Paloma; Pardo, Juan

doi:10.1007/978-3-319-09903-3_23

Cited by 18 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As we mentioned in this section, most of the previous work in unsupervised pre-training NN (or deep NNs) has focused on data compression 20 , dimensionality reduction 20,27 , classification 20,28 , and UTS forecasting 20 problems. Importantly, time series forecasting with deep learning techniques is an interesting research area that needs to be studied as well 19,26 . Moreover, even the recent time series forecasting research in the literature has focused on UTS problems.…”

Section: Related Workmentioning

confidence: 99%

“…The random initialization of a large numbers of neurons in such situations will lead the learning algorithm to converge to different local minima, depending on the values of the parameter initialization. Furthermore, and as a general practice, previous studies have demonstrated that training deep networks with several layers using random weights initialization and supervised training provide worse results than training shallow architectures 8,18,19 .…”

Section: Introductionmentioning

confidence: 99%

“…When they are applied to the convolutional neural network (CNN), they were shown to be useful to reduce the impact of vanishing gradient in deep models. Stacking many convolutional layers 26 instead of fully connected layers is an effective treatment to overcome this problem in deep neural networks 19 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems

Sagheer

Kotb

2019

Sci Rep

261

139

View full text Add to dashboard Cite

Currently, most real-world time series datasets are multivariate and are rich in dynamical information of the underlying system. Such datasets are attracting much attention; therefore, the need for accurate modelling of such high-dimensional datasets is increasing. Recently, the deep architecture of the recurrent neural network (RNN) and its variant long short-term memory (LSTM) have been proven to be more accurate than traditional statistical methods in modelling time series data. Despite the reported advantages of the deep LSTM model, its performance in modelling multivariate time series (MTS) data has not been satisfactory, particularly when attempting to process highly non-linear and long-interval MTS datasets. The reason is that the supervised learning approach initializes the neurons randomly in such recurrent networks, disabling the neurons that ultimately must properly learn the latent features of the correlated variables included in the MTS dataset. In this paper, we propose a pre-trained LSTM-based stacked autoencoder (LSTM-SAE) approach in an unsupervised learning fashion to replace the random weight initialization strategy adopted in deep LSTM recurrent networks. For evaluation purposes, two different case studies that include real-world datasets are investigated, where the performance of the proposed approach compares favourably with the deep LSTM approach. In addition, the proposed approach outperforms several reference models investigating the same case studies. Overall, the experimental results clearly show that the unsupervised pre-training approach improves the performance of deep LSTM and leads to better and faster convergence than other models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems

Sagheer

Kotb

2019

Sci Rep

261

139

View full text Add to dashboard Cite

show abstract

“…Digər tərəfdən, maşın təlimi metodları [6], o cümlədən dərin təlim yanaşması (ing. Deep Learning, DL) maliyyə zaman sıralarının proqnozlaşdırılmasında getdikcə böyük diqqət çəkir [7,8], çünki dərin neyron şəbəkələri əlamətləri avtomatik çıxarmaq imkanına malikdir, buna görə aprior analiz aparmağı və zaman sıralarının strukturunu əvvəlcədən bilməyi tələb etmir və qeyri-stasionar zaman sıralarına münasibətdə kifayət qədər etibarlıdırlar [9].…”

Section: Introductionunclassified

Bi̇tkoi̇ni̇n Qi̇yməti̇ni̇n Proqnozlaşdirilmasi Üçün Dəri̇n LSTM Metodu

İmamverdiyev¹,

İnstitutu²

2020

JPIT

View full text Add to dashboard Cite

Məqalədə Bitkoinin mübadilə kursunu proqnozlaşdırmaq üçün dərin neyron şəbəkə arxitekturası təklif olunur. Arxitekturanın əsasını rekurrent neyron şəbəkələrinin bir növü olan LSTM (ing. Long-Short Term Memory) təşkil edir. Son dövrlər kriptovalyutalar əhəmiyyətli maliyyə alətinə çevriliblər və bu səbəbdən onların ənənəvi valyutalara mübadilə kurslarının proqnozlaşdırılması məsələsi olduqca aktuallıq qazanmışdır. Bitkoin ilk kriptovalyutadır, hazırda kriptovalyutaların ümumi bazar kapitallaşmasının yarıdan çoxu ona məxsusdur və kriptovalyuta dünyasının "qızılı" rolunu oynayır, yəni digər kriptovalyutaların qiyməti çox zaman Bitkoin ilə ifadə edilir. Məhz bu səbəbdən Bitkoin kursunun proqnozlaşdırılması məsələsinə baxılmışdır və bir neçə ili əhatə edən böyük həcmli real verilənlərlə eksperimentlərdə təklif edilmiş dərin LSTM yanaşması Bitkoin zaman sıralarının proqnozlaşdırılmasında geniş istifadə edilən statistik metodlarla müqayisədə daha üstün nəticələr göstərmişdir. Alınmış nəticələr həm kriptovalyutalarla ötəri maraqlanan adi istifadəçilər, həm də kriptovalyutalar sahəsində fəal işləyən investorlar üçün əhəmiyyətlidir. Əldə edilmiş nəticələr həmçinin dərin təlim yanaşmalarının kriptovalyutalara aid qeyri-stasionar zaman sıralarının intellektual analizinin digər məsələlərinə tətbiqində də kifayət qədər məhsuldar ola biləcəyini təsdiqləyir.

show abstract

“…Specifically, this idea provides a better approach to (pre)train each layer in turn, initially using a local unsupervised criterion [36] with the aim of learning to produce useful higher-level representations from lower-level-representation output of the previous layer, which leads to much better solutions in terms of generalization performance. Due to such characteristics, DBNs and SDAs were successfully implemented in many nonlinear systems like dimensionality reduction [37][38][39], time-series forecasting [40][41][42], acoustic modeling [43][44][45], and digit recognition [46][47][48]. Therefore, we think the above-mentioned algorithms also have the potential to be applied in urban-sprawl simulations.…”

Section: Introductionmentioning

confidence: 99%

Integrating Cellular Automata with Unsupervised Deep-Learning Algorithms: A Case Study of Urban-Sprawl Simulation in the Jingjintang Urban Agglomeration, China

Ou¹,

Yang²,

Du³

et al. 2019

Sustainability

View full text Add to dashboard Cite

An effective simulation of the urban sprawl in an urban agglomeration is conducive to making regional policies. Previous studies verified the effectiveness of the cellular-automata (CA) model in simulating urban sprawl, and emphasized that the definition of transition rules is the key to the construction of the CA model. However, existing simulation models based on CA are limited in defining complex transition rules. The aim of this study was to investigate the capability of two unsupervised deep-learning algorithms (deep-belief networks, DBN) and stacked denoising autoencoders (SDA) to define transition rules in order to obtain more accurate simulated results. Choosing the Beijing–Tianjin–Tangshan urban agglomeration as the study area, two proposed models (DBN–CA and SDA–CA) were implemented in this area for simulating its urban sprawl during 2000–2010. Additionally, two traditional machine-learning-based CA models were built for comparative experiments. The implementation results demonstrated that integrating CA with unsupervised deep-learning algorithms is more suitable and accurate than traditional machine-learning algorithms on both the cell level and pattern level. Meanwhile, compared with the DBN–CA, the SDA–CA model had better accuracy in both aspects. Therefore, the unsupervised deep-learning-based CA model, especially SDA–CA, is a novel approach for simulating urban sprawl and also potentially for other complex geographical phenomena.

show abstract

Stacked Denoising Auto-Encoders for Short-Term Time Series Forecasting

Cited by 18 publications

References 21 publications

Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems

Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems

Bi̇tkoi̇ni̇n Qi̇yməti̇ni̇n Proqnozlaşdirilmasi Üçün Dəri̇n LSTM Metodu

Integrating Cellular Automata with Unsupervised Deep-Learning Algorithms: A Case Study of Urban-Sprawl Simulation in the Jingjintang Urban Agglomeration, China

Contact Info

Product

Resources

About