The Necessity of Construct and External Validity for Deductive Causal Inference

Abstract

The credibility revolution has facilitated tremendous progress in the social sciences by advancing design-based strategies that rely on internal validity to deductively identify causal effects. We demonstrate that prioritizing internal validity while neglecting construct and external validity prevents causal generalization and misleadingly converts a deductive claim of causality into a claim based on speculation and exploration -- undermining the very goals of the credibility revolution. We develop a formal framework of causal specification to demonstrate that internal, external and construct validity are jointly necessary for generalized claims regarding a causal effect. If one lacks construct validity, one cannot assign meaningful labels to the cause or to the outcome. If one lacks external validity, one cannot make statements about the conditions required for the cause to occur. Re-balancing considerations of internal, construct and external via causal specification preserves and advances the intent of the credibility revolution to understand causal effects.

Treatment for natural experiments: How to improve causal estimates using conceptual definitions and substantive interpretations

“…27 Gibson (2019) also shows that researchers often do not know enough about the data they use-how they were collected, what the variables are measuring, measurement errors etc.-and, therefore, fail to analyze them correctly. Esterling, Brady, and Schwitzgebel (2021) show that internal validity is not sufficient for estimating useful and reliable causal effects. They argue convincingly as to why the empirical natural experiments literature must begin considering both construct and external validity to claim to estimate credible causal effects.…”

“…Holland's statement is, therefore, useful for reminding us that any empirical causal effect is defined and estimated based on the empirical data available to us. The effect estimated by a (natural) experiment is only informative about the effect of interest to the extent that the empirically measured variables correspond to the outcome and treatment of interest (see also Esterling et al, 2021). Whereas this issue can be less problematic in controlled experiments, applications using observational data (e.g., natural experiments) need to carefully describe how the empirically measured variables relate to the outcome and treatment of interest.…”

“…This assumption has largely been overlooked in the social sciences, including economics, but is necessary for estimating reliable and useful causal effects (Esterling et al, 2021). The assumption has been discussed and elaborated upon in the epidemiology literature, where it is referred to as the consistency assumption (e.g., Cole and Frangakis, 2009;VanderWeele, 2009;Rehkopf et al, 2016).…”

“…As you will have realized by now, my arguments are related to external validity and construct validity (e.g., Shadish et al, 2002, chaps. 1-3;Esterling et al, 2021). The natural experiments literature has focused almost exclusively on the internal validity of its estimates, but we need all types of validity to provide useful and reliable results.…”

External validity in translational biomedicine: understanding the conditions enabling the cause to have an effect

“…While not the focus of this article, additional aspects of experimental validity important for generalization, variously termed construct, predictive and translational validity, among others, are also critical components of successful translation. These terms usually relate to how well an animal model mimics the human disease of interest, or how well measured variables map onto underlying constructs (preprint: Esterling et al, 2021). For example, while establishing an immune phenotype of similar maturity to humans will increase external validity, known and unknown species and strain differences in immune responses might still affect generalizability in certain contexts.…”