Efficient Neural Causal Discovery without Acyclicity Constraints

Abstract

Learning the structure of a causal graphical model using both observational and interventional data is a fundamental problem in many scientific fields. A promising direction is continuous optimization for score-based methods, which efficiently learn the causal graph in a data-driven manner. However, to date, those methods require constrained optimization to enforce acyclicity or lack convergence guarantees. In this paper, we present ENCO, an efficient structure learning method for directed, acyclic causal graphs leveraging observational and interventional data. ENCO formulates the graph search as an optimization of independent edge likelihoods with the edge orientation being modeled as a separate parameter. Consequently, we can provide convergence guarantees of ENCO under mild conditions without constraining the score function with respect to acyclicity. In experiments, we show that ENCO can efficiently recover graphs with hundreds of nodes, an order of magnitude larger than what was previously possible, while handling deterministic variables and latent confounders.

Learning to Induce Causal Structure

“…We also compared to an all-absent model corresponding to a zero adjacency matrix, which acts as a sanity check baseline. We also considered other methods (Chickering, 2002;Hauser & Bühlmann, 2012;Zhang et al, 2012;Gamella & Heinze-Deml, 2020), but only presented a comparison with non-linear ICP and DAG-GNN as these have shown to be strong performing models in other works (Ke et al, 2020a;Lippe et al, 2021;Scherrer et al, 2021). For Section 6.3, we also compared to additional baselines from Chickering (2002); Hauser & Bühlmann (2012); Zheng et al (2018); Gamella & Heinze-Deml (2020).…”

“…Although this could in principle yield cycles in the graph, in practice we observed strong performance regardless. Nevertheless, one could likely improve the results using post-processing (Lippe et al, 2021) or by extending the method with an accept-reject algorithm (Castelletti & Mascaro, 2022;Li et al, 2022).…”

“…Recently, Zheng et al (2018); Yu et al (2019); Lachapelle et al (2019) framed the structure search as a continuous optimization problem, which can be seen as a way to optimize for the scoring function. There also exist score-based methods that use a mix of continuous and discrete optimization (Bengio et al, 2019;Ke et al, 2020a;Lippe et al, 2021;Scherrer et al, 2021). Constraint-based methods (Spirtes et al, 2000;Sun et al, 2007;Zhang et al, 2012;Monti et al, 2019;Zhu & Chen, 2019) infer the DAG by analyzing conditional independencies in the data.…”

Learning Neural Causal Models with Active Interventions

“…Due to the limitations of purely observational data, and extend the continuous optimization framework to make use of interventional data. Lippe et al (2021) scales in a concurrent work with ours the work of to higher dimensions by splitting structural edge parameters in separate orientation and likelihood parameters and leveraging it in an adapted gradient formulation with lower variance. In contrast to and our work, they require interventional data on every variable.…”

“…As a result, there has been a surge in interest in differentiable structure learning and the combination of deep learning and causal inference . Such methods define a structural causal model with smoothly differentiable parameters that are adjusted to fit observational data (Zheng et al, 2018;Yu et al, 2019;Zheng et al, 2020;Bengio et al, 2019;Lorch et al, 2021;Annadani et al, 2021), although some methods can accept interventional data, thereby significantly improving the identification of the underlying data-generating process Lippe et al, 2021). However, the improvement critically depends on the experiments and interventions available.…”