Causal Information Rate

Abstract

Information processing is common in complex systems, and information geometric theory provides a useful tool to elucidate the characteristics of non-equilibrium processes, such as rare, extreme events, from the perspective of geometry. In particular, their time-evolutions can be viewed by the rate (information rate) at which new information is revealed (a new statistical state is accessed). In this paper, we extend this concept and develop a new information-geometric measure of causality by calculating the effect of one variable on the information rate of the other variable. We apply the proposed causal information rate to the Kramers equation and compare it with the entropy-based causality measure (information flow). Overall, the causal information rate is a sensitive method for identifying causal relations.

Information Length Quantification and Forecasting of Power Systems Kinetic Energy

“…Future work will investigate other possible probabilistic and dynamic metrics to measure power system related signals with highly intermittent big data. For instance, we plan to use information length to measure the information flow between the elements in the system by considering its causality properties [42]. We also see that the integration of ML and the IL needs to be explored in more detail.…”

“…IL has been successfully applied in different process including quantum, fluid and biological process [38]. In addition, inspired by the IL metric, [42] introduces a new information-geometric measure of causality by calculating the effect of one variable on the information rate of the other variable. Furthermore, [43] elucidates the meaning of IL in light of stochastic thermodynamics, in particular, detailing its connection to entropy production or free energy in the case of the non-autonomous Ornstein-Uhlenbeck process.…”

Information Geometry Theoretic Measures for Characterizing Neural Information Processing from Simulated EEG Signals

“…In this work, by using simulated EEG signals of both CTL groups and AD patients under both EC and EO conditions and based on our previous works on information geometry [ 24 , 25 , 26 ], we develop novel and powerful quantitative measures in terms of information rate and causal information rate to quantify the important features of neurological dynamics of brains. We are able to find significant and interesting distinctions between CTL subjects and AD patients when they switch between the eyes-open and eyes-closed status.…”

Monte Carlo Simulation of Stochastic Differential Equation to Study Information Geometry

“…The gradient of , , then represents a velocity on this manifold Note that we use the notation instead of to make it clear that it is a quantity defined for a time-dependent PDF. represents the rate of change of statistically distinguishable states in a time-evolving PDF, and is sometimes referred to in the literature [ 29 , 30 , 31 ] as the Information Rate. Note that in information theory [ 32 ], the term “Information Rate” is used for the rate at which information is passed over the channel [ 33 ].…”

“…As for the physical significance of , in a non-equilibrium thermodynamic system, is related to the entropy production [ 30 ]. has also been used to study causality [ 29 ] and abrupt changes in the dynamics of a system [ 8 ]. is equivalent to the (symmetric) KL divergence of infinitesimally close PDFs, as shown in Appendix E .…”