Latent variables and propensity score matching: a simulation study with application to data from the Programme for International Student Assessment in Poland

Abstract

This paper examines how including latent variables can benefit propensity score matching. Latent variables can be estimated from the observed manifest variables and used in matching. This paper demonstrates the benefits of such an approach by comparing it with a method where the manifest variables are directly used in matching. Estimating the propensity score on the manifest variables introduces a measurement error that can be limited with estimating the propensity score on the estimated latent variable. We use Monte Carlo simulations to test how the proposed approach behaves under distinct circumstances found in practice, and then apply it to real data. Using the estimated latent variable in the propensity score matching limits the measurement error bias of the treatment effects' estimates and increases their precision. The benefits are larger for small samples and with better information about the latent variable available.

Causal Inference on Education Policies: A Survey of Empirical Studies Using Pisa, Timss and Pirls

“…This would help researchers to apply an appropriate evaluation procedure for the process of evaluating interventions or practices. For example, several such enhancements have already been implemented as national options for the PISA studies in Germany or Poland (Klieme, ; Jakubowski, ). In view of the importance of assessing the impacts of educational policies in particular, we would like to draw attention to the need to build longitudinal datasets at student or school level.…”

“…Jakubowski () evaluates differences in the magnitude of student progress across two types (vocational and general vocational) of upper secondary education in Poland using data from the PISA 2006 national study that extended the sample to cover 16‐ and 17‐year‐olds (enrolled in tenth and eleventh grade in the Polish school system). This dataset provides supplementary information on students’ previous scores in national exams.…”

Propensity Score Analysis With Latent Covariates: Measurement Error Bias Correction Using the Covariate’s Posterior Mean, aka the Inclusive Factor Score

“…Raykov argued, intuitively, that since the cFS better represents the true latent covariate, adjusting for it and the PS based on it should produce less bias than adjusting for the measurement items and the PS based on them. Subsequently, Jakubowski (2015) used simulation to evaluate the use of the cFS compared to using the measurement items directly in PS matching analysis. While the author's conclusions seem to favor the cFS, our reading of the simulation results is that these two methods have similar levels of bias.…”

“…The PS -the probability of exposure assignment given these covariates -is a balancing score, meaning conditional on it (e.g., through matching or weighting), there is balance between exposure conditions on these covariates, thus removing confounding by them (Rosenbaum & Rubin, 1983). Implicit in this no unobserved confounding assumption is the assumption that covariates are measured without error; measurement error often leads to bias in the estimated causal effect (Jakubowski, 2015;Pearl, 2010;Raykov, 2012;Steiner, Cook, & Shadish, 2011;Yi, Ma, & Carroll, 2012). In PS analysis, covariate measurement error biases the estimation of the PS, resulting in residual imbalance of the true covariate (after matching or weighting), which may lead to bias in the estimated causal effect.…”

Propensity Score Weighting with Mismeasured Covariates: An Application to Two Financial Literacy Interventions

“…As a cost to this extra flexibility, the authors only establish consistency; further characterization of the asymptotic properties are left as a gap to be filled. Jakubowski (2010) assesses the performance of propensity score matching when an unobserved covariate is proxied by several variables. The author considers two estimation methods.…”