Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics

Abstract

Abstract:In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an alternative decomposition evidences the so-called cross entropy (CE) and conditional SE (cSE), quantifying the cross information and internal information of the target system, respectively. This study presents a thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical structure of coupled dynamic processes. First, we investigate the theoretical properties of these measures, providing the conditions for their existence and assessing the meaning of the information theoretic quantity that each of them reflects. Then, we present an approach for the exact computation of information dynamics based on the linear Gaussian approximation, and exploit this approach to characterize the behavior of SE, TE, CE and cSE in benchmark systems with known dynamics. Finally, we exploit these measures to study cardiorespiratory dynamics measured from healthy subjects during head-up tilt and paced breathing protocols. Our main result is that the combined evaluation of the measures of information dynamics allows to infer the causal effects associated with the observed OPEN ACCESSEntropy 2015, 17 278 dynamics and to interpret the alteration of these effects with changing experimental conditions.

Information Decomposition in Multivariate Systems: Definitions, Implementation and Application to Cardiovascular Networks

“…In this section we provide a derivation of the exact values of any of the information measures entering in the decompositions defined above under the assumption that the observed dynamical network S = {X,Y} = {V,W,Y} is composed by Gaussian processes [12]. Specifically, we assume that the overall vector process S has a joint Gaussian distribution, which means that any vector variable extracted sampling the constituent processes at present and past times takes values from a multivariate Gaussian distribution.…”

“…An important but not fully explored aspect is that the measures of information dynamics are often used in isolation, thus limiting their interpretational capability. Indeed, recent studies have pointed out the intertwined nature of the measures of information dynamics, and the need to combine their evaluation to avoid misinterpretations about the underlying network properties [4,12,30]. Moreover, the specificity of measures of information storage and transfer is often limited by the fact that their definition incorporates multiple aspects of the dynamical structure of network processes; the high flexibility of information-theoretic measures allows to overcome this limitation by expanding these measures into meaningful quantities [13,29].…”

“…All these measures have been successfully applied in diverse fields of science ranging from cybernetics to econometrics, climatology, neuroscience and others [6,7,[18][19][20][21][22][23][24][25][26][27][28]. In particular, recent studies have implemented these measures in cardiovascular physiology to study the short-term dynamics of the cardiac, vascular and respiratory systems in terms of information storage, transfer and modification [12,13,29].…”

“…These tools formalize different information-theoretic concepts applied to a "target" system in the observed dynamical network: the predictive information about the system describes the amount of information shared between its present state and the past history of the whole observed network [4,5]; the information storage indicates the information shared between the present and past states of the target [6,7]; the information transfer defines the information that a group of systems designed as "sources" provide about the present state of the target [8,9]; and the information modification reflects the redundant or synergetic interaction between multiple sources sending information to the target [3,10]. Operational definitions of these concepts have been proposed in recent years, which allow to quantify predictive information through measures of prediction entropy or full-predictability [11,12], information storage through the self-entropy or self-predictability [11,13], information transfer through transfer entropy or Granger causality [14], and information modification through entropy and prediction measures of net redundancy/synergy [11,15] or separate measures derived from partial information decomposition [16,17]. All these measures have been successfully applied in diverse fields of science ranging from cybernetics to econometrics, climatology, neuroscience and others [6,7,[18][19][20][21][22][23][24][25][26][27][28].…”

Time-varying coupling functions: Dynamical inference and cause of synchronization transitions

“…The presentation in this paper points to the importance of the phenomenon of time-varying sub-coupling components, and hopefully it will stimulate further developments of some aspects of these methods. Here, it is worth noting out that there have recently been efforts to use information based methods to perform coupling decomposition [62,63] by exploiting the conditional functional dependencies. These methods could advance the detection of certain sub-coupling relations, without exploiting a dynamical model.…”