Estimating Categorical Counterfactuals via Deep Twin Networks

Abstract

Counterfactual inference is a powerful tool, capable of solving challenging problems in high-profile sectors. To perform counterfactual inference, one requires knowledge of the underlying causal mechanisms. However, causal mechanisms cannot be uniquely determined from observations and interventions alone. This raises the question of how to choose the causal mechanisms so that resulting counterfactual inference is trustworthy in a given domain. This question has been addressed in causal models with binary variables, but the case of categorical variables remains unanswered. We address this challenge by introducing for causal models with categorical variables the notion of counterfactual ordering, a principle that posits desirable properties causal mechanisms should posses, and prove that it is equivalent to specific functional constraints on the causal mechanisms. To learn causal mechanisms satisfying these constraints, and perform counterfactual inference with them, we introduce deep twin networks. These are deep neural networks that, when trained, are capable of twin network counterfactual inference---an alternative to the abduction, action, & prediction method. We empirically test our approach on diverse real-world and semi-synthetic data from medicine, epidemiology, and finance, reporting accurate estimation of counterfactual probabilities while demonstrating the issues that arise with counterfactual reasoning when counterfactual ordering is not enforced.

Towards Learning Causal Representations of Technical Word Embeddings for Smart Troubleshooting

“…In terms of evaluation, given that CI is not a well-defined task in language processing, the results may be questioned due to their strict dependence on subjective human criteria. This is a clear point of general improvement (beyond the specific purposes of this work) toward the fair assessment of other related CI approaches such as the Twin Networks method to estimate the probabilities of causation (Vlontzos, A., Kainz, B., and Gilligan-Lee, C. M., 2021), the causal regularization of neural networks to improve their interpretability (Bahadori, M. T., Chalupka, K., Choi, E., Chen, R., Stewart, W. F., and Sun, J., 2017; Shen, Z., Cui, P., Kuang, K., Li, B., and Chen, P., 2018), or the learning of causally disentangled representations using Variational Autoencoders (Suter, R., Miladinović, D., Schölkopf, B., and Bauer, S.,, 2019;Yang, M., Liu, F., Chen, Z., Shen, X., Hao, J., and Wang, J., 2020).…”

A Review of Causality for Learning Algorithms in Medical Image Analysis

“…(Pawlowski et al, 2020) developed a normalizing flow model to perform the abduction step in an abduction-action-prediction counterfactual inference task and are able to generate plausible brain MRI volumes. assume a different approach and develop a generative model based on Deep Twin Networks (Vlontzos et al, 2021a). Performing counterfactual inference in the latent space embeddings, the authors are able to generate realistic Ultrasound Videos with different Left Ventricle Ejection Fractions.…”

“…Indeed, P (Y x = y ′ | E = e) is given by (Pearl, 2009) u P (Y x (u) = y ′ )P (u|e) . There are two main ways to resolve this type of questions; the Abduction-Action-Prediction paradigm and the Twin Network paradigm shown respectively in ML literature among others in (Castro et al, 2020b;Vlontzos et al, 2021a). In short given SCM M with latent distribution P (U ) and evidence e, the conditional probability P (Y x | e) is evaluated as follows: 1) Abduction: Infer the posterior of the latent variables with evidence e to obtain P (U | e), 2) Action: Apply do(x) to obtain submodel M x , 3) Prediction: Compute the probability of Y in the submodel M x with P (U | e).…”

Causal machine learning for healthcare and precision medicine

“…Counterfactuals can be estimated with (i) a three-step procedure [53] ( abduction–action–prediction ) which has been recently enhanced with deep learning [15,92] using generative models such as normalizing flows [93], variational autoencoders [94] and diffusion probabilistic models [95] or (ii) twin networks [96] which augment the original SCM resulting in both factual and counterfactual variables represented simultaneously. Deep twin networks [97] leverage neural networks to further improve flexibility of the causal mechanisms. We note that quantifying the effect of interventions usually assumes that causal models are given either explicitly [15,98] or learned via causal discovery [99].…”