The Liang-Kleeman Information Flow: Theory and Applications

Abstract

Abstract:Information flow, or information transfer as it may be referred to, is a fundamental notion in general physics which has wide applications in scientific disciplines. Recently, a rigorous formalism has been established with respect to both deterministic and stochastic systems, with flow measures explicitly obtained. These measures possess some important properties, among which is flow or transfer asymmetry. The formalism has been validated and put to application with a variety of benchmark systems, such as the baker transformation, Hénon map, truncated Burgers-Hopf system, Langevin equation, etc. In the chaotic Burgers-Hopf system, all the transfers, save for one, are essentially zero, indicating that the processes underlying a dynamical phenomenon, albeit complex, could be simple. (Truth is simple.) In the Langevin equation case, it is found that there could be no information flowing from one certain time series to another series, though the two are highly correlated.Information flow/transfer provides a potential measure of the cause-effect relation between dynamical events, a relation usually hidden behind the correlation in a traditional sense.

A Recipe for the Estimation of Information Flow in a Dynamical System

“…In [13], Liang explored the information flow in dynamical systems that can be modeled by equations obtained by the underlying physical concepts. In such cases, the information flow has been analyzed by the evolution of the joint probability distributions using the Liouville equations and by the Fokker-Planck equations, in the cases of the deterministic and stochastic systems, respectively [13].…”

Thermodynamic aspects of information transfer in complex dynamical systems

“…For a recent comprehensive review of the Liang-Kleeman approach, we refer to [28]. For the latest developments on this approach, we refer instead to [29].…”

<strong>New insights on land surface-atmosphere feedbacks over tropical South America at interannual timescales</strong>

“…We overcame such limitation by using causality analyses to quantify the connectivity between variables. Such causality or information transfer has been defined within the framework of information theory [69,[84][85][86]. Three basic tenets of the information transfer are the following: (1) Causality implies correlation but correlation does not imply causality, (2) Causality implies directionality, which means that the transfer of information detects the direction of information transfer between two systems, and (3) Asymmetry is a basic property of information transfer.…”

“…(A4) uses the concept of Shannon or Absolute Entropy [59]. For two-variable systems, [85] proved that the flow of information, x y T  , is the same for both absolute and relative entropy. The latter is more suitable to study predictability, owing to its invariance properties under non-linear interactions [54,56,84].…”