External Validity: From Do-Calculus to Transportability Across Populations

Abstract

The generalizability of empirical findings to new environments, settings or populations, often called "external validity," is essential in most scientific explorations. This paper treats a particular problem of generalizability, called "transportability," defined as a license to transfer causal effects learned in experimental studies to a new population, in which only observational studies can be conducted. We introduce a formal representation called "selection diagrams" for expressing knowledge about differences and commonalities between populations of interest and, using this representation, we reduce questions of transportability to symbolic derivations in the do-calculus. This reduction yields graph-based procedures for deciding, prior to observing any data, whether causal effects in the target population can be inferred from experimental findings in the study population. When the answer is affirmative, the procedures identify what experimental and observational findings need be obtained from the two populations, and how they can be combined to ensure bias-free transport.

Conditioning on Post-treatment Variables

“…One such research question is the unconditional causal effect of X on Y, denoted Pðy j doðxÞÞ, which is fully analyzed using the do-calculus [4], namely, using Q do . Another research question benefitting from Q do occurs in transportability problems [5,6], where the target query is P Ã ðy j doðxÞÞ (the causal effect in a new population), and has been fully analyzed in do-calculus, again, using Q do . I have not seen Q do presented as a target query on its own right.…”

In Defense of Unification (Comments on West and Koch's review of Causality)

“…The first is mathematized in (Pearl and Bareinboim, 2014) and the second in (Shpitser and Pearl, 2008). Threats may warn us of problems -they do not solve problems.…”

Generalizing Experimental Findings

“…A similar change would occur as more individuals become aware of the efficacy of the treatment. The result is an inherent disparity between the target population and the population under study.The problem of generalizing across disparate populations has received a formal treatment in Pearl and Bareinboim [6] where it was labeled "transportability," and where necessary and sufficient conditions for valid generalization were established (see [7]). The problem of selection bias, though it has some unique features, can also be viewed as a nuance of the transportability problem, thus inheriting all the theoretical results established in Pearl and Bareinboim [6] that guarantee valid generalizations.…”

“…I SB on the other hand is characterized by the fact that do-expressions are always conditioned on S, reflecting the fact that we have experimental information only on the selected sample, S ¼ 1. The analysis reported in Pearl and Bareinboim [6] has resulted in an algorithmic criterion for deciding whether transportability is feasible and, when confirmed, the algorithm produces an estimand for the desired effects [7]. The algorithm is complete, in the sense that, when it fails, a consistent estimate of the target effect does not exist (unless one strengthens the assumptions encoded in M).…”

“…A given disparity between Pðx; y; zÞ and P Ã ðx; y; zÞ may demand 260 J. Pearl: Generalizing Experimental Findings Unauthenticated Download Date | 5/11/18 2:38 PM different adjustments, depending on the location of S in the causal structure. A simple example of this phenomenon is demonstrated in Figure 3(b) of [6] where a disparity in the average reading ability of two cities requires two different treatments, depending on what causes the disparity. If the disparity emanates from age differences, adjustment is necessary, because age is likely to affect the potential outcomes.…”