Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions

Hewamalage, Hansika; Bergmeir, Christoph; Bandara, Kasun

doi:10.48550/arxiv.1909.00590

Cited by 13 publications

(16 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Scale Normalization as Preprocessing: Several forms of preprocessing have been recommended for global methods [31,33,3,42]. They are considered an important component of the performance of the methods.…”

Section: Modern Practices and Model Classes For Global Methodsmentioning

confidence: 99%

Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality

Montero-Manso¹,

Hyndman²

2020

Preprint

View full text Add to dashboard Cite

Forecasting groups of time series is of increasing practical importance. Some examples are: forecasting the demand for multiple products offered by a retailer, server loads within a data center or the number of completed ride shares in zones within a city. The local approach to this problem considers each time series separately and fits a function or model to each series. The global approach considers all time series as the same regression task and fits a single function to all series. For groups of similar time series, global methods outperform the more established local methods. However, there is recent empirical evidence showing surprisingly good performance of global models on heterogeneous groups of time series. This suggests a more general applicability of global methods, with major implications in forecasting theory and practice, in the form of more accurate tools for automatic forecasting and new scenarios to study. However, the evidence has been of empirical nature and a more fundamental study is required. In this paper, we formalize the setting of forecasting a set of time series with local and global learning algorithms, leading to the following contributions:• We show that global methods are not more restrictive than local methods for time series forecasting, a result which does not apply to sets of regression problems in general. Global and local methods can produce the same forecasts without any assumptions about similarity of the series in the set. This result shows that global models can succeed in a wider range of problems than previously thought. • We derive basic generalization bounds for local and global algorithms. We find that the complexity of local methods grows with the size of the set while it remains constant for global methods. Therefore a global algorithm can afford to be quite complex and still benefit from better generalization error than local methods for large datasets. These bounds serve to clarify and support recent experimental results in the area of time series forecasting, and guide the design of new algorithms. For the specific class of limited-memory autoregressive models, this bound leads to the design of global models with much larger memory than what is effective for local methods. • The findings are supported by an extensive empirical study. We show that purposely naïve algorithms derived from these principles, such as global linear models fit by least squares, deep networks or even high order polynomials, result in superior accuracy in benchmark datasets. In particular, global linear models show an unreasonable effectiveness, providing competitive forecasting accuracy with far fewer parameters than the simplest of local methods. Empirical evidence points towards global models being able to automatically learn long memory patterns and related effects that are only available to local models if introduced manually. Keywords Time SeriesConsider the problem of having to forecast many time series as a group. We might need to forecast tourist arrivals at all our resorts for n...

show abstract

Section: Modern Practices and Model Classes For Global Methodsmentioning

confidence: 99%

Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality

Montero-Manso¹,

Hyndman²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In order to make the data to fit the LSTM model, a preprocessing step is necessary to normalize raw data between 0 and 1 [33]. This process allows to adjust data on a common magnitude scale, providing a more effective weights adjustments for the neural networks [32]. The normalization is performed by Equation ( 7)…”

Section: Datasetmentioning

confidence: 99%

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

et al. 2021

View full text Add to dashboard Cite

Water flow forecasts are an essential information for energy production, management and hydropower control. Advanced actions to optimize electricity production can be taken based on predicted information. This work proposes an ensemble strategy using recurrent neural networks to generate a forecast of water flow at Jirau Hydroelectric Power Plant (HPP), installed on the Madeira River in Brazil. The ensemble strategy consists of combining three long short-term memory (LSTM) networks that model the Madeira River and two of its tributaries: Mamoré and Abunã rivers. The historical data from streamflow of the Madeira river and its tributaries are used to validate the ensemble LSTM model, where each time series of river tributaries are modeled separated by LSTM models and the result used as input for another LSTM model in order to forecast the streamflow of the main river. The experimental results present low errors for training and test sets for individual LSTM networks and ensemble model. In addition, these results were compared with the operational forecasts performed by Jirau HPP. The proposed model showed better accuracy in four of the five scenarios tested, which indicates a promising approach to be explored in water flow forecasting based on river tributaries.

show abstract

“…Recent trends in ML such as deep learning, especially deep recurrent NNs (RNNs), are very attractive for time series forecasting [15]. RNNs with connections between nodes forming a directed graph along a temporal sequence are able to exhibit temporal dynamic behavior using their internal state (memory) to process sequences of inputs.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Residual Dilated LSTM end Exponential Smoothing Model for Mid-Term Electric Load Forecasting

Dudek¹,

Pełka²,

Smyl³

2020

Preprint

View full text Add to dashboard Cite

This work presents a hybrid and hierarchical deep learning model for mid-term load forecasting. The model combines exponential smoothing (ETS), advanced Long Short-Term Memory (LSTM) and ensembling. ETS extracts dynamically the main components of each individual time series and enables the model to learn their representation. Multi-layer LSTM is equipped with dilated recurrent skip connections and a spatial shortcut path from lower layers to allow the model to better capture long-term seasonal relationships and ensure more efficient training. A common learning procedure for LSTM and ETS, with a penalized pinball loss, leads to simultaneous optimization of data representation and forecasting performance. In addition, ensembling at three levels ensures a powerful regularization. A simulation study performed on the monthly electricity demand time series for 35 European countries confirmed the high performance of the proposed model and its competitiveness with classical models such as ARIMA and ETS as well as state-ofthe-art models based on machine learning.

show abstract

Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions

Cited by 13 publications

References 46 publications

Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality

Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality

Water Flow Forecasting Based on River Tributaries Using Long Short-Term Memory Ensemble Model

A Hybrid Residual Dilated LSTM end Exponential Smoothing Model for Mid-Term Electric Load Forecasting

Contact Info

Product

Resources

About