Time Series Prediction Using Restricted Boltzmann Machines and Backpropagation

Abstract

Time series prediction appear in many real-world problems, e.g., financial market, signal processing, weather forecasting among others. The underlying models and time series data of those problems are generally complex in a way that reasonable accurate estimation cannot be easily achieved, thus requiring more advanced techniques. Statistical models are the classical approaches for tackling this problem. Many works extended different architectures of Artificial neural networks to work with time series prediction, such as Feedforward, Boltzmann Machines and Deep Belief Network. A Deep Belief Network based on hybridization between Gaussian-Bernoulli Restricted Boltzmann Machine and the Backpropagation algorithm is presented. The hybrid algorithm is tested on three time series databases showing promising results.

Financial time series forecasting with deep learning : A systematic literature review: 2005–2019

“…RBM is a productive stochastic ANN that can learn probability distribution on the input set [65]. RBMs are mostly used for unsupervised learning [66]. RBMs are used in applications such as dimension reduction, classification, feature learning, collaborative filtering [67].…”

“…The training of RBMs is implemented through minimizing the negative log-likelihood of the model and data. Contrastive Divergence (CD) algorithm is used for the stochastic approximation algorithm which replaces the model expectation for an estimation using Gibbs Sampling with a limited number of iterations [66]. In the CD algorithm, the Kullback Leibler Divergence (KL-Divergence) algorithm is used to measure the distance between its reconstructed probability distribution and the original probability distribution of the input [71].…”

Optimasi Prediksi Dengan Algoritma Backpropagation Dan Conjugate Gradient Beale-Powell Restarts

“…Dengan melakukan suatu prediksi, pengetahuan tentang posisi perekonomian Indonesia dapat diketahui sehingga dapat mengantisipasi atau meminimalisasi resiko yang mungkin akan ditimbulkan, sehingga baik pemerintah maupun pihak swasta dapat terbantu dalam menentukan kebijakan maupun pengambilan keputusan. Akan tetapi proses prediksi tidaklah mudah, dibutuhkan model dasar dan data rangkaian waktu dari masalah-masalah tersebut, yang umumnya rumit dengan cara estimasi keakuratan yang tidak mudah dicapai, sehingga membutuhkan teknik yang lebih maju [2].…”

Predict Time Series with Multiple Artificial Neural Networks

“…Yu et al [11] proposed a new hyper-parameters selection approach for support vector machines to predict time series. Hrasko et al [12] used Restricted Boltzmann Machine and the Back propagation algorithm for time series prediction.…”