Minimum Message Length Moving Average Time Series Data Mining

Sak, M.; Dowe, David L.; Ray, Sid

doi:10.1109/cima.2005.1662352

“…In MML, there is (Bayesian) prior knowledge (or a prior distribution), π, over the parameter space. Following Wallace and Freeman [9], MML has been shown to work well in time series models, such as autoregressive (AR) and moving average (MA) models [18,27,28]. We can thus estimate the parameters [7,9] by minimizing the message length:…”

Section: Minimum Message Lengthmentioning

confidence: 99%

“…In this paper, we select the best ARMA( p , q ) model and then train the LSTM model for the residuals through the ARMA model. The time-step order used in LSTM is the parameter q in ARMA( p , q ) determined by different information-theoretic criteria [ 18 , 19 ].…”

Section: Introductionmentioning

confidence: 99%

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Fang

¹

,

Dowe

²

,

Peiris

³

et al. 2021

Entropy

View full text Add to dashboard Cite

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered.We also develop a simple MML ARIMA model.

show abstract

“…In MML, there is prior knowledge of π over the parameter space so the MML is part of the Bayesian approach. Following Wallace and Freeman [15], MML87 is extended version of MML, it has been shown to work well in time series model such as autoregressive model (AR) and moving average model (MA) [12,16]. Assuming the ARMA model parameters are given by β = (φ 1 , ..., φ p , θ 1 , ..., θ q , σ 2 , following MML87, we seek the parameter values in order to minimize the message length is:…”

Section: Minimum Message Lengthmentioning

confidence: 99%

“…We use the information-theoretic Minimum Message Length (MML), AIC and BIC to select the model orders of ARMA(p, q), and we then train the LSTM model by the residuals left from the ARMA model [12,13]. Most other papers using this kind of hybrid model use information-theoretic model selection techniques such as AIC and BIC.…”

Section: Introductionmentioning

confidence: 99%

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Fang¹,

Dowe²,

Peiris³

et al. 2021

Preprint

3

0

View full text Add to dashboard Cite

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

show abstract

“…In MML, there is prior knowledge of π over the parameter space so the MML is part of the Bayesian approach. Following Wallace and Freeman [15], MML87 is extended version of MML, it has been shown to work well in time series model such as autoregressive model (AR) and moving average model (MA) [12,16]. Assuming the ARMA model parameters are given by β = (φ 1 , ..., φ p , θ 1 , ..., θ q , σ 2 ) following MML87, we seek the parameter values in order to minimize the message length given by:…”

mentioning

confidence: 99%

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Fang¹,

Dowe²,

Peiris³

et al. 2021

Preprint

View full text Add to dashboard Cite

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

show abstract

Minimum Message Length Moving Average Time Series Data Mining

Cited by 6 publications

References 27 publications

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Contact Info

Product

Resources

About