The importance of covariate selection in controlling for selection bias in observational studies.

Steiner, Péter; Cook, Thomas D.; Shadish, William R.; Clark, M. H.

doi:10.1037/a0018719

Cited by 403 publications

(326 citation statements)

References 64 publications

(112 reference statements)

Supporting

Mentioning

317

Contrasting

Unclassified

Order By: Relevance

“…une expérimentation randomisée étant contrôlée, les individus y agissent donc dans un environnement qui n'est pas totalement naturel, ce qui peut les amener à modifier leurs comportements d'une maniè-re différente de ce qu'ils feraient sans un tel contrôle. avec un devis quasi expérimental, les individus présentent vraisemblablement des différences au stade initial de l'intervention (Grimes & schulz, 2002;steiner, cook, shadish, & clark, 2010). il est pour ainsi dire naturel de ne pas obtenir une équirépartition des chances de traitement, les individus n'ayant pas été affectés aléatoirement au groupe traité ou témoin.…”

Section: Aurélie Lecocq Mehdi Ammi Et éLodie Bellarbreunclassified

Le score de propension : un guide méthodologique pour les recherches expérimentales et quasi expérimentales en éducation

Lecocq

Ammi

Bellarbre

2016

mee

View full text Add to dashboard Cite

La méthode du score de propension devient de plus en plus populaire pour estimer les effets causaux d’un programme d’intervention. Si les applications empiriques de cette méthode sont encore rares dans les recherches en éducation, des exemples de son utilisation se trouvent aisément dans d’autres disciplines. Cependant, sa mise en place soulève plusieurs questions. L’objectif de cet article est de fournir des éléments de réponses guidant le chercheur et l’évaluateur du domaine de l’éducation pour l’estimation et l’utilisation du score de propension. Les différentes étapes de son application sont présentées pas à pas : évaluation du biais de sélection, construction du score de propension et mesure de sa qualité, et choix des stratégies d’utilisation du score dans l’estimation des effets d’un traitement. Les questions méthodologiques soulevées sont discutées à chaque étape. Pour faciliter la compréhension, un exemple d’une expérimentation en maternelle illustre la méthode.Propensity score methods become increasingly popular for estimating the causal effects of an intervention. If empirical applications of this method are still unusual in education researches, examples can be found easily in other fields. However, its implementation raises several questions. The aim of this paper is to provide guidance to educational researchers regarding the estimation and the utilization of the propensity score. The various stages of its application are presented step by step: evaluation of the selection bias, construction of the propensity score and measurement of its quality, and decision on the strategies to use in the estimation of treatment effects. Methodological issues are discussed at each stage. To facilitate the understanding, an example of an experiment in kindergarten illustrates the method.O método da pontuação de propensão está tornar-se cada vez mais popular para estimar os efeitos causais de um programa de intervenção. Embora as aplicações empíricas deste método ainda sejam raras na investigação em educação, exemplos do seu uso podem ser facilmente encontrados noutras disciplinas. No entanto, a sua implementação levanta diversas questões. O objetivo deste artigo é fornecer algumas respostas para orientar o investigador e o avaliador do domínio da educação para a estimativa e uso da pontuação de propensão. As diferentes etapas da sua aplicação são apresentadas passo a passo: avaliação do viés de seleção, construção da pontuação de propensão e medição da sua qualidade e escolha das estratégias para o uso da pontuação na estimativa dos efeitos do tratamento. As questões metodológicas são discutidas em cada etapa. Para facilitar o entendimento, um exemplo de uma experiência no jardim-de-infância ilustra o método

show abstract

Section: Aurélie Lecocq Mehdi Ammi Et éLodie Bellarbreunclassified

Le score de propension : un guide méthodologique pour les recherches expérimentales et quasi expérimentales en éducation

Lecocq

Ammi

Bellarbre

2016

mee

View full text Add to dashboard Cite

show abstract

“…We join them in that caution because we doubt that many applications of propensity scores meet some of the most basic conditions for their valid use (Shadish, 2012;Steiner, Cook & Shadish, 2011;Shadish, Cook, Steiner & Clark, 2010). In that sense, we applaud the PMO article.…”

mentioning

confidence: 95%

A Case Study About Why It Can Be Difficult To Test Whether Propensity Score Analysis Works in Field Experiments

Shadish

Steiner

Cook³

2013

JMMSS

Self Cite

View full text Add to dashboard Cite

Peikes, Moreno and Orzol (2008) sensibly caution researchers that propensity score analysis may not lead to valid causal inference in field applications. But at the same time, they made the far stronger claim to have performed an ideal test of whether propensity score matching in quasi-experimental data is capable of approximating the results of a randomized experiment in their dataset, and that this ideal test showed that such matching could not do so. In this article we show that their study does not support that conclusion because it failed to meet a number of basic criteria for an ideal test. By implication, many other purported tests of the effectiveness of propensity score analysis probably also fail to meet these criteria, and are therefore questionable contributions to the literature on the effects of propensity score analysis. Keywords: Propensity Scores, Strong Ignorability, Quasi-Experiments, Within-Study ComparisonIn 2008, Peikes, Moreno and Orzol (henceforth PMO) published a case study in The American Statistician that cautioned social program evaluators that propensity score analysis may yield quite different results than those from a randomized experiment. We join them in that caution because we doubt that many applications of propensity scores meet some of the most basic conditions for their valid use (Shadish, 2012;Steiner, Cook & Shadish, 2011;Shadish, Cook, Steiner & Clark, 2010). In that sense, we applaud the PMO article. Field researchers need to appreciate how difficult it can be to use propensity score analysis in a way that yields confidence in the results.At the same time, however, PMO made a second claim, that they performed an "ideal" (pp. 222, 223, 230) test of whether propensity score matching in quasi-experimental data could approximate the results of a randomized experiment. In fact, a careful analysis of PMO suggests just the opposite conclusion, that they neither implemented an ideal propensity score analysis nor an ideal comparison of results from a propensity score analysis to results from a randomized experiment. In this article, we show why the PMO study was not ideal in both respects. Just as practitioners need to appreciate how difficult it can be to use propensity

show abstract

“…If the propensity score is estimated or the covariates measured imperfectly, this bias may affect the estimated intervention effect (33). One way to cope with this problem is to construct different sets of propensity scores to test its robustness (39)(40)(41).…”

Section: Schelvis Et Almentioning

confidence: 99%

Evaluation of occupational health interventions using a randomized controlled trial: challenges and alternative research designs

Schelvis¹,

Hengel²,

Burdorf³

et al. 2015

Scand J Work Environ Health

104

View full text Add to dashboard Cite

This overview aims to guide researchers in occupational health in conducting evaluative research. Several appropriate alternatives for the randomized controlled trial design are available and feasible (ie, stepped wedge, propensity scores, instrumental variables, multiple baseline design, interrupted time series, difference-in-difference, and regression discontinuity), which may provide sufficiently strong evidence to guide decisions on implementation of interventions in workplaces.Affiliation:

show abstract

The importance of covariate selection in controlling for selection bias in observational studies.

Cited by 403 publications

References 64 publications

Le score de propension : un guide méthodologique pour les recherches expérimentales et quasi expérimentales en éducation

Le score de propension : un guide méthodologique pour les recherches expérimentales et quasi expérimentales en éducation

A Case Study About Why It Can Be Difficult To Test Whether Propensity Score Analysis Works in Field Experiments

Evaluation of occupational health interventions using a randomized controlled trial: challenges and alternative research designs

Contact Info

Product

Resources

About