Statistical Feature Construction for Forecasting Accuracy Increase and Its Applications in Neural Network Based Analysis

Abstract

This paper presents a feature construction approach called Statistical Feature Construction (SFC) for time series prediction. Creation of new features is based on statistical characteristics of analyzed data series. First, the initial data are transformed into an array of short pseudo-stationary windows. For each window, a statistical model is created and characteristics of these models are later used as additional features for a single window or as time-dependent features for the entire time series. To demonstrate the effect of SFC, five plasma physics and six oceanographic time series were analyzed. For each window, unknown distribution parameters were estimated with the method of moving separation of finite normal mixtures. First four statistical moments of these mixtures for initial data and increments were used as additional data features. Multi-layer recurrent neural networks were trained to create short- and medium-term forecasts with a single window as input data; additional features were used to initialize the hidden state of recurrent layers. A hyperparameter grid-search was performed to compare fully-optimized neural networks for original and enriched data. A significant decrease in RMSE metric was observed with a median of 11.4%. There was no increase in RMSE metric in any of the analyzed time series. The experimental results have shown that SFC can be a valuable method for forecasting accuracy improvement.

Comparison of Statistical Approaches for Reconstructing Random Coefficients in the Problem of Stochastic Modeling of Air–Sea Heat Flux Increments

“…These distributions, or, rather, the parameters and the connectivity components obtained from their estimation, could be used within the framework of physics-informed machine learning (ML). Some examples of effective improvement of neural network forecasts using this method were demonstrated earlier in [16] for a few heat fluxes in the Labrador Sea and the Gulf Stream. It is worth noting that obtaining the point estimates by the first approach, in this case, may require solving multiple SDEs.…”

Preface to the Special Issue on “Control, Optimization, and Mathematical Modeling of Complex Systems”

“…Gorshenin and Kuzmin [5] present a feature construction approach called statistical feature construction (SFC) for time-series prediction. The creation of new features is based on statistical characteristics of analyzed data series.…”

“…Numerical simulation in physical, social, and life sciences [1][2][3][4]; • Modeling and analysis of complex systems based on mathematical methods and AI/ML approaches [5,6]; • Control problems in robotics [3,[7][8][9][10][11][12]]; • Design optimization of complex systems [13]; • Modeling in economics and social sciences [4,14]; • Stochastic models in physics and engineering [1,[15][16][17][18]; • Mathematical models in material science [19]; • High-performance computing for mathematical modeling [20].…”

Short-Term Mobile Network Traffic Forecasting Using Seasonal ARIMA and Holt-Winters Models

“…In [16], this problem is considered based on network slicing, mobile edge computing, base station sleeping, and additional power during high-demand hours. Traffic forecasting is divided into short-term and long-term, but sometimes medium-term forecasting is also necessary [17].…”