Information Theoretic Causal Effect Quantification

Abstract

Modelling causal relationships has become popular across various disciplines. Most common frameworks for causality are the Pearlian causal directed acyclic graphs (DAGs) and the Neyman-Rubin potential outcome framework. In this paper, we propose an information theoretic framework for causal effect quantification. To this end, we formulate a two step causal deduction procedure in the Pearl and Rubin frameworks and introduce its equivalent which uses information theoretic terms only. The first step of the procedure consists of ensuring no confounding or finding an adjustment set with directed information. In the second step, the causal effect is quantified. We subsequently unify previous definitions of directed information present in the literature and clarify the confusion surrounding them. We also motivate using chain graphs for directed information in time series and extend our approach to chain graphs. The proposed approach serves as a translation between causality modelling and information theory.

Probability trees and the value of a single intervention

“…The effect of a cause has been quantified using information theory (Wieczorek and Roth 2019), however, without considering learning in a Bayesian setting. Entropic causal inference (see (Compton et al 2021)) specifies circumstances where the causal direction between categorical variables can be determined from observational data under assumptions of limited entropy.…”

Information Structures for Causally Explainable Decisions

“…Parents’ heights are potential causes of offspring heights, but not vice versa. This intuitively appealing idea leads to tests for predictive causality , such as classical Granger causality testing in time-series analysis and its nonparametric generalization to transfer entropy [ 19 ]. In these tests, X is defined as a predictive cause of Y if future values of Y are not conditionally independent of past and present values of X , given past and present values of Y itself.…”

“…Specifically, to predict how changing X (via an exogenous intervention or manipulation) would change the distribution of one of its descendants, Y , it is necessary to identify an adjustment set of other variables to condition upon [ 30 ]. Adjustment sets generalize the principle that one must condition on any common parents of X and Y to eliminate confounding biases, but must not condition on any common children to avoid introducing selection biases [ 19 ]. Appropriate adjustment sets can be computed from the DAG structure of a causal BN for both direct causal effects and total causal effects [ 30 ].…”

“…The distribution of Y would then change in response to the change in the distribution of its input, Z . If all relevant confounders are measured and controlled in this way, then the changes in Y ’s distribution are presumably caused by the differences in X values [ 16 , 19 ].…”

Active learning of causal probability trees

“…Our approach combines two ideas: First, we make use of the ability of probability trees to represent context-dependent relationships by representing the different causal hypothesis as sub-trees in a single large probability tree, thereby reducing the problem of causal induction to a simple inference problem in this larger probability tree, which can be solved using Bayes theorem; second, by combining the causal hypotheses in a single model, we can predict the information gain associated with each intervention in advance, and thus select the intervention which has the highest gain, thereby leading to a natural activelearning method for causal induction on probability trees 1 . a) Related work: Information theory has previously been used to quantify the causal effect between variables [17], or to specify circumstances where the causal orientation of categorical variables can be determined from observational data [18]. Information geometry was used to infer causal orientation from observational data, using assumptions on generative mechanisms [19], however both settings are different from the Bayesian learning problem considered here.…”