The Relation between Granger Causality and Directed Information Theory: A Review

Abstract

This report reviews the conceptual and theoretical links between Granger causality and directed information theory. We begin with a short historical tour of Granger causality, concentrating on its closeness to information theory. The definitions of Granger causality based on prediction are recalled, and the importance of the observation set is discussed. We present the definitions based on conditional independence. The notion of instantaneous coupling is included in the definitions. The concept of Granger causality graphs is discussed. We present directed information theory from the perspective of studies of causal influences between stochastic processes. Causal conditioning appears to be the cornerstone for the relation between information theory and Granger causality. In the bivariate case, the fundamental measure is the directed information, which decomposes as the sum of the transfer entropies and a term quantifying instantaneous coupling. We show the decomposition of the mutual information into the sums of the transfer entropies and the instantaneous coupling measure, a relation known for the linear Gaussian case. We study the multivariate case, showing that the useful decomposition is blurred by instantaneous coupling. The links are further developed by studying how measures based on directed information theory naturally emerge from Granger causality inference frameworks as hypothesis testing.

Information diffusion, cluster formation and entropy-based network dynamics in equity and commodity markets

“…Transfer entropy has been widely used in a variety of scientific fields such as the cellular automata in computer science (e.g., Lizier and Mahoney, 2013), the brain neural cortex in medicine (e.g., Vicente et al, 2011, Faes et al, 2013, social networks (e.g,, Ver Steeg and Galstyan, 2012), causal influences and applied statistics (e.g., Barnett, 2009, Amblard and Michel, 2013, Liu et al, 2014, and in dynamical systems (e.g., Nichols et al, 2013, Prokopenko et al, 2013. Applications of transfer entropy to the analysis of financial market include, among others, Marschinski and Kantz (2002), Kwon and Yang (2008), Altiparmak and Dengiz (2009), Dimpfl andPeter (2012), Sandoval (2014), and Sandoval et al (2015).…”

Learning Causal Graph: A Genetic Programming Approach

“…For example the link between information theory and Granger causality has been reviewed in [23]: they discuss the conceptual and theoretical relations between Granger causality and directed information theory; and report that "measures based on directed information theory naturally emerge from Granger causality inference frameworks,"…”

Transfer Entropy for Nonparametric Granger Causality Detection: An Evaluation of Different Resampling Methods

“…Table 3. Observed size and power of the TE-based test for the bivariate ARCH process in Equation (11). Table 6.…”

“…Instead, with the information theoretical interpretation, the TE offers a natural way to measure directional information transfer and Granger causality. We refer to [10,11] for detailed reviews of the relation between Granger causality and directed information theory. However, the direct application of entropy and its variants, though attractive, turns out to be difficult, if not impossible altogether, due to the lack of asymptotic distribution theory for the test statistics.…”