A review of irregular time series data handling with gated recurrent neural networks

Weerakody, Philip B.; Wong, Kok Wai; Wang, Guanjin; Ela, Wendell P.

doi:10.1016/j.neucom.2021.02.046

Cited by 222 publications

(74 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to bursts of rapidly occurring events, human-created time series datasets are always separated by long inactive periods, showing irregular sparsity [ 17 , 18 ]. It presents a more significant challenge because sparse data and the resulting missing values severely limit the data’s ability to be analyzed and modeled for classification and forecasting tasks [ 19 ]. The sparsity modeling problem has been widely studied, and existing approaches mainly involve two categories.…”

Section: Introductionmentioning

confidence: 99%

A survival analysis based volatility and sparsity modeling network for student dropout prediction

et al. 2022

View full text Add to dashboard Cite

Student Dropout Prediction (SDP) is pivotal in mitigating withdrawals in Massive Open Online Courses. Previous studies generally modeled the SDP problem as a binary classification task, providing a single prediction outcome. Accordingly, some attempts introduce survival analysis methods to achieve continuous and consistent predictions over time. However, the volatility and sparsity of data always weaken the models’ performance. Prevailing solutions rely heavily on data pre-processing independent of predictive models, which are labor-intensive and may contaminate authentic data. This paper proposes a Survival Analysis based Volatility and Sparsity Modeling Network (SAVSNet) to address these issues in an end-to-end deep learning framework. Specifically, SAVSNet smooths the volatile time series by convolution network while preserving the original data information using Long-Short Term Memory Network (LSTM). Furthermore, we propose a Time-Missing-Aware LSTM unit to mitigate the impact of data sparsity by integrating informative missingness patterns into the model. A survival analysis loss function is adopted for parameter estimation, and the model outputs monotonically decreasing survival probabilities. In the experiments, we compare the proposed method with state-of-the-art methods in two real-world MOOC datasets, and the experiment results show the effectiveness of our proposed model.

show abstract

Section: Introductionmentioning

confidence: 99%

A survival analysis based volatility and sparsity modeling network for student dropout prediction

et al. 2022

View full text Add to dashboard Cite

show abstract

“…To compare the ESN performance, we train an LSTM network on the SPA data. However, the temporal irregularity of the data affects the ability of these types of networks to model the behavior of the system [32]. Two trials were conducted.…”

Section: Modeling Using Lstmmentioning

confidence: 99%

Modeling of Soft Pneumatic Actuators with Different Orientation Angles Using Echo State Networks for Irregular Time Series Data

et al. 2022

View full text Add to dashboard Cite

Modeling of soft robotics systems proves to be an extremely difficult task, due to the large deformation of the soft materials used to make such robots. Reliable and accurate models are necessary for the control task of these soft robots. In this paper, a data-driven approach using machine learning is presented to model the kinematics of Soft Pneumatic Actuators (SPAs). An Echo State Network (ESN) architecture is used to predict the SPA’s tip position in 3 axes. Initially, data from actual 3D printed SPAs is obtained to build a training dataset for the network. Irregular-intervals pressure inputs are used to drive the SPA in different actuation sequences. The network is then iteratively trained and optimized. The demonstrated method is shown to successfully model the complex non-linear behavior of the SPA, using only the control input without any feedback sensory data as additional input to the network. In addition, the ability of the network to estimate the kinematics of SPAs with different orientation angles θ is achieved. The ESN is compared to a Long Short-Term Memory (LSTM) network that is trained on the interpolated experimental data. Both networks are then tested on Finite Element Analysis (FEA) data for other θ angle SPAs not included in the training data. This methodology could offer a general approach to modeling SPAs with varying design parameters.

show abstract

“…Irregular data and resulting missing values severely limit the ability to analyze and model data for classification and forecasting tasks. Traditional methods used to process time series data often introduce bias and make strong assumptions about the underlying data generation process, which can lead to poor model predictions [7].…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

Application of the NARX neural network for predicting a one-dimensional time series

Serikov

Zhetpisbayeva

Mirzakulova

et al. 2021

EEJET

View full text Add to dashboard Cite

Time series data analysis and forecasting tool for studying the data on the use of network traffic is very important to provide acceptable and good quality network services, including network monitoring, resource management, and threat detection. More and more, the behavior of network traffic is described by the theory of deterministic chaos. The traffic of a modern network has a complex structure, an uneven rate of packet arrival for service by network devices. Predicting network traffic is still an important task, as forecast data provide the necessary information to solve the problem of managing network flows. Numerous studies of actually measured data confirm that they are nonstationary and their structure is multicomponent. This paper presents modeling using Nonlinear Autoregression Exogenous (NARX) algorithm for predicting network traffic datasets. NARX is one of the models that can be used to demonstrate non-linear systems, especially in modeling time series datasets. In other words, they called the categories of dynamic feedback networks covering several layers of the network. An artificial neural network (ANN) was developed, trained and tested using the LM learning algorithm (Levenberg-Macwardt). The initial data for the prediction is the actual measured network traffic of the packet rate. As a result of the study of the initial data, the best value of the smallest mean-square error MSE (Mean Squared Error) was obtained with the epoch value equal to 18. As for the regression R, its output ANN values in relation to the target for training, validation and testing were 0.97743. 0.9638 and 0.94907, respectively, with an overall regression value of 0.97134, which ensures that all datasets match exactly. Experimental results (MSE, R) have proven the method's ability to accurately estimate and predict network traffic

show abstract

A review of irregular time series data handling with gated recurrent neural networks

Cited by 222 publications

References 49 publications

A survival analysis based volatility and sparsity modeling network for student dropout prediction

A survival analysis based volatility and sparsity modeling network for student dropout prediction

Modeling of Soft Pneumatic Actuators with Different Orientation Angles Using Echo State Networks for Irregular Time Series Data

Application of the NARX neural network for predicting a one-dimensional time series

Contact Info

Product

Resources

About