Forecasting Nonlinear Systems with LSTM: Analysis and Comparison with EKF

Abstract

Certain difficulties in path forecasting and filtering problems are based in the initial hypothesis of estimation and filtering techniques. Common hypotheses include that the system can be modeled as linear, Markovian, Gaussian, or all at one time. Although, in many cases, there are strategies to tackle problems with approaches that show very good results, the associated engineering process can become highly complex, requiring a great deal of time or even becoming unapproachable. To have tools to tackle complex problems without starting from a previous hypothesis but to continue to solve classic challenges and sharpen the implementation of estimation and filtering systems is of high scientific interest. This paper addresses the forecast–filter problem from deep learning paradigms with a neural network architecture inspired by natural language processing techniques and data structure. Unlike Kalman, this proposal performs the process of prediction and filtering in the same phase, while Kalman requires two phases. We propose three different study cases of incremental conceptual difficulty. The experimentation is divided into five parts: the standardization effect in raw data, proposal validation, filtering, loss of measurements (forecasting), and, finally, robustness. The results are compared with a Kalman filter, showing that the proposal is comparable in terms of the error within the linear case, with improved performance when facing non-linear systems.

Cumulative error correction of inertial navigation systems using LIDAR sensors and extended Kalman filter

“…Typically, the EKF algorithm is divided into prediction and update phases. The prediction stage involves the utilization of control input to forecast the state of the system, as well as the covariance matrix [17], [18]. In order to adjust the state of the system, the Kalman gain is computed during the correction phase.…”

LSTM Network for the Oxygen Concentration Modeling of a Wastewater Treatment Plant

“…In this way, the gradient flows over time, even for long periods, and its derivative remains computable (they neither vanish nor diverge). LSTM networks have been demonstrated to be able to dynamically map the input and output of a dynamical system; their use for learning the inherent properties of a given dynamical system has been studied in [14,17,18,24,[26][27][28], obtaining promising results. They have proven to be powerful in representing long and short temporal dependencies in multiple examples with respect to other recurrent architectures, such as Gated Recurrent Unit (GRU), BIdirectional-LSTM (BI-LSTM), etc.…”

“…They have proven to be powerful in representing long and short temporal dependencies in multiple examples with respect to other recurrent architectures, such as Gated Recurrent Unit (GRU), BIdirectional-LSTM (BI-LSTM), etc. [26]. For these reasons, the LSTM architecture was chosen as a promising candidate to model the process under study.…”

An Improved Kalman Filter Based on Long Short-Memory Recurrent Neural Network for Nonlinear Radar Target Tracking

“…Moreover, they use an overall LSTM structure for estimation, which makes it more difficult to fit. Llerena et al [17] used the LSTM unit to estimate the motion state through the encoder decoder architecture. However, only the performance comparison with EKF algorithm, and only under several linear and specific motion paths, cannot fully explain the superiority of this algorithm.…”