Causal Inference by Compression

Budhathoki, Kailash; Vreeken, Jilles

doi:10.1109/icdm.2016.0015

Cited by 27 publications

(31 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is one particular area which we would like to mention specifically, since that area had hardly seen any MDL applications before 2007 whereas nowadays such applications are flourishing: this is the field of data mining. Some representative publications are Vreeken et al [2011], Koutra et al [2015], Budhathoki et al [2018]. Most of this work centers on the use of two-part codes, but sometimes NML and other sophisticated universal distributions/codes are used as well [Tatti and Vreeken, 2008].…”

Section: Discussionmentioning

confidence: 99%

Minimum description length revisited

Grünwald

Roos

2019

Int. J. Math. Ind.

View full text Add to dashboard Cite

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of MDL estimators. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.

show abstract

Section: Discussionmentioning

confidence: 99%

Minimum description length revisited

Grünwald

Roos

2019

Int. J. Math. Ind.

View full text Add to dashboard Cite

show abstract

“…Mooij instantiates the first practical compression-based approach [18] using the Minimum Message Length. Budhathoki and Vreeken approximate K(X) and K(Y | X) through MDL, and propose Origo, a decision tree based approach for causal inference on multivariate binary data [1]. Marx and Vreeken [17] proposed Slope, an MDL based method employing local and global regression for univariate numeric data.…”

Section: Related Workmentioning

confidence: 99%

“…Our approach is based on algorithmic information theory. That is, we follow the postulate that if X → Y , it will be easier-in terms of Kolmogorov complexityto first describe X, and then describe Y given X, than vice-versa [11,33,1]. Kolmogorov complexity is not computable, but can be approximated through the Minimum Description Length (MDL) principle [26,5], which we use to instantiate this framework.…”

Section: Introductionmentioning

confidence: 99%

Causal Inference on Multivariate and Mixed-Type Data

Vreeken

2019

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

Given data over the joint distribution of two random variables X and Y , we consider the problem of inferring the most likely causal direction between X and Y . In particular, we consider the general case where both X and Y may be univariate or multivariate, and of the same or mixed data types. We take an information theoretic approach, based on Kolmogorov complexity, from which it follows that first describing the data over cause and then that of effect given cause is shorter than the reverse direction.The ideal score is not computable, but can be approximated through the Minimum Description Length (MDL) principle. Based on MDL, we propose two scores, one for when both X and Y are of the same single data type, and one for when they are mixed-type. We model dependencies between X and Y using classification and regression trees. As inferring the optimal model is NP-hard, we propose Crack, a fast greedy algorithm to determine the most likely causal direction directly from the data.Empirical evaluation on a wide range of data shows that Crack reliably, and with high accuracy, infers the correct causal direction on both univariate and multivariate causeeffect pairs over both single and mixed-type data.

show abstract

“…Vreeken [29] proposed to approximate the Kolmogorov complexity for numeric data using the cumulative residual entropy, and gave an instantiation for multivariate continuous-valued data. Perhaps most related to SLOPE is ORIGO [4], which uses MDL to infer causal direction on binary data, whereas we focus on univariate numeric data.…”

Section: Related Workmentioning

confidence: 99%

Telling Cause from Effect Using MDL-Based Local and Global Regression

Vreeken

2017

2017 IEEE International Conference on Data Mining (ICDM)

Self Cite

View full text Add to dashboard Cite

We consider the fundamental problem of inferring the causal direction between two univariate numeric random variables X and Y from observational data. The two-variable case is especially difficult to solve since it is not possible to use standard conditional independence tests between the variables.To tackle this problem, we follow an information theoretic approach based on Kolmogorov complexity and use the Minimum Description Length (MDL) principle to provide a practical solution. In particular, we propose a compression scheme to encode local and global functional relations using MDL-based regression. We infer X causes Y in case it is shorter to describe Y as a function of X than the inverse direction. In addition, we introduce SLOPE, an efficient linear-time algorithm that through thorough empirical evaluation on both synthetic and real world data we show outperforms the state of the art by a wide margin.

show abstract

Causal Inference by Compression

Cited by 27 publications

References 17 publications

Minimum description length revisited

Minimum description length revisited

Causal Inference on Multivariate and Mixed-Type Data

Telling Cause from Effect Using MDL-Based Local and Global Regression

Contact Info

Product

Resources

About