Telling Cause from Effect Using MDL-Based Local and Global Regression

Abstract

We consider the fundamental problem of inferring the causal direction between two univariate numeric random variables X and Y from observational data. The two-variable case is especially difficult to solve since it is not possible to use standard conditional independence tests between the variables.To tackle this problem, we follow an information theoretic approach based on Kolmogorov complexity and use the Minimum Description Length (MDL) principle to provide a practical solution. In particular, we propose a compression scheme to encode local and global functional relations using MDL-based regression. We infer X causes Y in case it is shorter to describe Y as a function of X than the inverse direction. In addition, we introduce SLOPE, an efficient linear-time algorithm that through thorough empirical evaluation on both synthetic and real world data we show outperforms the state of the art by a wide margin.

Telling cause from effect by local and global regression

“…Our key contributions can be summarized as follows, we (a) show how to model unobserved mechanisms via compound deterministic and nondeterministic functions, (b) propose an MDL score for causal inference on pairs of univariate numeric random variables, (c) formulate two analytic significance tests based on compression, (d) introduce the linear-time algorithms Slope and Sloper, (e) give extensive empirical evaluation, including a case study (f) and make all code, data generators and data available. This paper builds upon and extends the work appearing in ICDM'17 [18]. Notably, we provide a link between the confidence and significance score of our method.…”

Predictive Data Mining Techniques for Fault Diagnosis of Electric Equipment: A Review

“…Regarding regression analysis, Le et al [124] present the geometric-based online Gaussian process that could scale with massive datasets, guaranteeing that the proposed algorithm produces a good enough solution (close to the optimal one) and a fast-online regression. Marx and Vreeken [125] present an information theory-based approach using the Kolmogorov complexity and the principle of minimum description length to provide a practical solution to the problem of inferring the direction of causal dependence of observational data. Rudaś and Jaroszewicz [126] analyze two uplift modeling approaches for linear regression and identify the situations in which each model works best; in fact, they propose a third model that combines the benefits of both approaches.…”

Ordinal Causal Discovery

“…Under such representation, BNs are generally identifiable if the noises are non-Gaussian (Shimizu et al, 2006), if the functional form of the additive noise model is nonlinear (Hoyer et al, 2009;Zhang and Hyvärinen, 2009), or if the noise variances are equal (Peters and Bühlmann, 2014). Also see much of the recent literature that focuses on bivariate causal discovery (Mooij et al, 2010;Janzing et al, 2012;Chen et al, 2014;Sgouritsa et al, 2015;Hernandez-Lobato et al, 2016;Marx and Vreeken, 2017;Blöbaum et al, 2018;Marx and Vreeken, 2019;Tagasovska et al, 2020). For count data, Categorical Data.…”