Information Geometric Theory in the Prediction of Abrupt Changes in System Dynamics

Abstract

Detection and measurement of abrupt changes in a process can provide us with important tools for decision making in systems management. In particular, it can be utilised to predict the onset of a sudden event such as a rare, extreme event which causes the abrupt dynamical change in the system. Here, we investigate the prediction capability of information theory by focusing on how sensitive information-geometric theory (information length diagnostics) and entropy-based information theoretical method (information flow) are to abrupt changes. To this end, we utilise a non-autonomous Kramer equation by including a sudden perturbation to the system to mimic the onset of a sudden event and calculate time-dependent probability density functions (PDFs) and various statistical quantities with the help of numerical simulations. We show that information length diagnostics predict the onset of a sudden event better than the information flow. Furthermore, it is explicitly shown that the information flow like any other entropy-based measures has limitations in measuring perturbations which do not affect entropy.

Causal Information Rate

“…The difference between these two rates then quantifies the effect of the evolution of v on the entropy of x. Note that T v→x can be both negative and positive; a negative T v→x means that v acts to reduce the marginal entropy of x (S 1 ), as numerically observed in Reference [32].…”

“…This is explicitly shown in Section 5.4 (see Figure 5) in regard to causality. Although not widely recognized, it is important to point out the limitation of entropybased measures in measuring perturbations (in particular, caused by abrupt events) that do not affect entropy, as shown in Reference [32]. In addition, entropy has shortcomings, such as being non-invariant under coordinate transformations and insensitive to the local arrangement (shape) of p(x, t) for fixed t. Similar comments are applicable to other entropybased measures.…”

“…We have recently proposed information-geometric theory as a powerful tool to understand non-equilibrium stochastic processes that often involve high temporal variabilities and large fluctuations [20][21][22][23][24][25][26][27][28][29][30][31][32], as often the case of rare, extreme events. This is based on the surprisal rate, r(x, t) = ∂ t s(x, t) = −∂ t ln p(x, t), where p(x, t) is a probability density function (PDF) of a random variable x at time t, and s(x, t) = − ln p(x, t) is a local entropy.…”

“…Furthermore, L is a path-dependent dimensionless distance and is uniquely defined as a function of time for fixed parameters and initial condition. These properties are advantageous for quantifying correlation in time-varying data and understanding selforganization, long memory, and hysteresis involved in phase transitions [20,24,[27][28][29][30]32]. In particular, we recently investigated a non-autonomous Kramer equation by including a sudden perturbation to the system to mimic the onset of a sudden event [32], demonstrating that our information rate predicts the onset of a sudden event better than one of the entropybased measures (information flow) (see Section 5.4 for details).…”

“…The purpose of this paper is to develop an information-geometric measure of causality (the causal information rate) by generalizing E (Γ). Like Reference [32], our intention here is not on modeling the appearance of rare, extreme events (that are nonlinear, non-Gaussian) themselves, but on developing a new information-geometric causality method which is useful for predicting and understanding those events. The remainder of this paper is organized as follows.…”