Propensity scores in the design of observational studies for causal effects

Abstract

Summary The design of any study, whether experimental or observational, that is intended to estimate the causal effects of a treatment condition relative to a control condition, refers to those activities that precede any examination of outcome variables. As defined in our 1983 article (Rosenbaum & Rubin, 1983), the propensity score is the unit-level conditional probability of assignment to treatment versus control given the observed covariates; so, the propensity score explicitly does not involve any outcome variables, in contrast to other summaries of variables sometimes used in observational studies. Balancing the distributions of covariates in the treatment and control groups by matching or balancing on the propensity score is therefore an aspect of the design of the observational study. In this invited comment on our 1983 article, we review the situation in the early 1980’s, and we recall some apparent paradoxes that propensity scores helped to resolve. We demonstrate that it is possible to balance an enormous number of low-dimensional summaries of a high-dimensional covariate, even though it is generally impossible to match individuals closely for all of the components of a high-dimensional covariate. In a sense, there is only one crucial observed covariate, the propensity score, and there is one crucial unobserved covariate, the ‘principal unobserved covariate’. The propensity score and the principal unobserved covariate are equal when treatment assignment is strongly ignorable, that is, unconfounded. Controlling for observed covariates is a prelude to the crucial step from association to causation, the step that addresses potential biases from unmeasured covariates. The design of an observational study also prepares for the step to causation: by selecting comparisons to increase the design sensitivity, by seeking opportunities to detect bias, by seeking mutually supportive evidence affected by different biases, by incorporating quasi-experimental devices such as multiple control groups, and by including the economist’s instruments. All of these considerations reflect the formal development of sensitivity analyses that were largely informal prior to the 1980s.

Quantifying Influence: Propensity Score Matching Unravels the True Effect Sizes of Learning Management Models on Students’ Analytical Thinking

“…Secondly, upon considering the influence of research characteristics following propensity score matching, it was found that propensity score matching completely eradicated the influence of research characteristics. (Rosenbaum & Rubin, 2023). Thirdly, before conducting propensity score matching, a comparison of learning management models developed for students' analytical thinking revealed that the mean effect size of the studies did not differ significantly from zero (Qa = 4.577), and the residuals from the estimation were not zero (Qb = 1184.007***).…”

“…Propensity score matching is a statistical method that simulates a randomized controlled trial to reduce bias in nonrandomized samples (Kane et al, 2020;Morgan, 2018). It aims to decrease the variability of confounding variables (Rosenbaum & Rubin, 2023) by ensuring that the comparison groups are as similar as possible (Benedetto et al, 2018). The technique adjusts experimental outcomes based on research characteristics serving as confounding variables, allowing for a controlled comparison of treatment effects (Haukoos & Lewis, 2015;Thoemmes, 2012).…”

The U.S. Government's Program of Welfare for the Wealthy

“…The role of PSM is to determine the efficacy of applied treatment and helps in removing selection and endogeneity biases. PSM is an extensively used method to estimate causal effects by matching treatment with control cases based on a set of covariates (Rosenbaum and Rubin 2022). Using PSM, researchers want to compare the outcomes between a treatment and control group by ensuring that both groups have similar characteristics on the observables.…”

A Second Evidence Factor for a Second Control Group

“…In parallel with results about propensity scores for observed covariates, we may always represent the combined influence of many unobserved covariates on treatment $$Z_{ij}$$ by a single scalar unobserved covariate $$u_{ij}$$ with $$0\le u_{ij}\le 1$$ called the “principal unobserved covariate,” the only unobserved covariate that introduces relevant bias into treatment assignment having controlled for $$\mathbf {x}_{ij}$$; see Rosenbaum and Rubin (2023, Section 4.2). This principal unobserved covariate is $$u_{ij}=\Pr (Z_{ij}=1\,|\,r_{Tij},\,r_{Cij},\,\mathbf {x}_{ij})$$, and it is unobserved because $$(r_{Tij},\,r_{Cij})$$ are never jointly observed.…”