We Are Not Your Real Parents: Telling Causal from Confounded using MDL

Abstract

Given data over variables (X 1 , ..., X m , Y ) we consider the problem of nding out whether X jointly causes Y or whether they are all confounded by an unobserved latent variable Z . To do so, we take an information-theoretic approach based on Kolmogorov complexity. In a nutshell, we follow the postulate that rst encoding the true cause, and then the e ects given that cause, results in a shorter description than any other encoding of the observed variables. e ideal score is not computable, and hence we have to approximate it. We propose to do so using the Minimum Description Length (MDL) principle. We compare the MDL scores under the models where X causes Y and where there exists a latent variables Z confounding both X and Y and show our scores are consistent. To nd potential confounders we propose using latent factor modeling, in particular, probabilistic PCA (PPCA).Empirical evaluation on both synthetic and real-world data shows that our method, C C , performs very well-even when the true generating process of the data is far from the assumptions made by the models we use. Moreover, it is robust as its accuracy goes hand in hand with its con dence.

Information Bottleneck for Estimating Treatment Effects with Systematically Missing Covariates

“…The latter is essentially a VAE where X is replaced by Y in the decoder. Finally, Kaltenpoth and Vreeken [ 42 ] propose using Probabilistic PCA to deduce a set of confounders and reason about treatment effects. Unlike both of the aforementioned methods however, our approach considers a different causal graph and specifically uses the IB method to learn a compressed representation of the covariate information for reasoning about treatment effects where covariates are systematically missing at test time.…”

Detect and Correct Bias in Multi-Site Neuroimaging Datasets

“…Even if they would be known, they would not be available in all publicly available neuroimaging datasets. Consequently, we consider a recently proposed approach for causal inference by Kaltenpoth and Vreeken (2019) that assumes an unobserved confounding variable. The method explicitly models the hidden confounder with probabilistic PCA, which allows comparing the causal X → Y and confounded model X ← Z → Y to conclude whether the relationship is causal or confounded.…”

“…Together with causal sufficiency, this allows to identify the true causal network as the least complex one. Although the AMC generally relies on causal sufficiency, Kaltenpoth and Vreeken (2019) proposed to integrate the confounder as a latent variable Z. This approach allows for computing whether confounding is present, without explicitly knowing the confounder or having data on it, which is very relevant in practice.…”

“…We estimate the complexity of both models by computing (2) and (3) using a no-U-turn sampler (Hoffman and Gelman, 2014) with four chains and 3,000 samples each (half of which are used for burn-in) as implemented in PyStan 2.19.0 (Carpenter et al, 2017). We experienced instability estimating the complexity using variational inference as proposed by Kaltenpoth and Vreeken (2019).…”

The Blessings of Multiple Causes

“…Mooij et al (2010) use Gaussian processes to depict causal mechanisms; Zhang and Hyvärinen (2009) study post-nonlinear causal models and their identifiability; Mckeigue et al (2010) builds on sparse methods to infer causal structures; Moghaddass et al (2016) generalize the self-controlled case series method to multiple causes and multiple outcomes using factor models. More recently, Louizos et al (2017) use variational autoencoders to infer unobserved confounders, Shah and Meinshausen (2018) develop projection-based techniques for high-dimensional covariance estimation under latent confounding, and Kaltenpoth and Vreeken (2019) leverages information theory principles to differentiate causal and confounded connections.…”