Generalizing Study Results

Abstract

Great care is taken in epidemiologic studies to ensure the internal validity of causal effect estimates; however, external validity has received considerably less attention. When the study sample is not a random sample of the target population, the sample average treatment effect, even if internally valid, cannot usually be expected to equal the average treatment effect in the target population. The utility of an effect estimate for planning purposes and decision making will depend on the degree of departure from the true causal effect in the target population due to problems with both internal and external validity. Herein, we review concepts from recent literature on generalizability, one facet of external validity, using the potential outcomes framework. Identification conditions sufficient for external validity closely parallel identification conditions for internal validity, namely: conditional exchangeability; positivity; the same distributions of the versions of treatment; no interference; and no measurement error. We also require correct model specification. Under these conditions, we discuss how a version of direct standardization (the g-formula, adjustment formula, or transport formula) or inverse probability weighting can be used to generalize a causal effect from a study sample to a well-defined target population, and demonstrate their application in an illustrative example.

Estimated Prevalence of Nonverbal Learning Disability Among North American Children and Adolescents

“…Applying inflation factor weights to our sample aligned the NVLD estimate with rates of known However, epidemiologic methods are rapidly developing to implement high-quality weighting algorithms, allowing for greater transparency and utility of clinical and community-based data. 32,33 Identifying Downloaded From: https://jamanetwork.com/ on 09/09/2020…”

A Comparison of Common Plastic Surgery Operations Using the NSQIP and TOPS Databases

“…We chose to use inverse probability of selection weights (IPSWs), which are proposed in the literature, where the intent is to generalize results from randomized controlled trials to a target population. 23 We developed a model for the IPSW within each surgical category because we felt each surgical category represented potentially unique patient populations. In each case, the selection model involved a multivariable logistic regression on all preoperative patient variables shared across the datasets (each variable in Table 1 ), where interaction and higher order terms were included if the resultant model improved the balance in covariates across datasets.…”

Baseline Characteristics Explain Differences in Effectiveness of Randomization to Daily Oral TDF/FTC PrEP Between Transgender Women and Cisgender Men Who Have Sex With Men in the iPrEx Trial

“…14 Using this reduced set of variables W* , we applied a generalization of the g-formula 15 to estimate IRR msm . 15,16 This approach is analogous to model-based direct standardization in which the MSM population is standardized to resemble the distribution of covariates observed in TGW. 17 Assuming correct model specification, IRR msm estimates what the ITT incidence rate ratio would have been in MSM had they shared the same distribution of baseline covariates as TGW in iPrEx.…”