The Econometrics of Randomized Experiments

Athey, Susan; Imbens, Guido W.

doi:10.1016/bs.hefe.2016.10.003

Cited by 502 publications

(329 citation statements)

References 133 publications

Supporting

Mentioning

324

Contrasting

Unclassified

Order By: Relevance

“…Duflo et al . (), Gerber and Green () and Athey and Imbens ()) and technology companies (e.g. Kohavi and Longbotham (2017)).…”

Section: Introductionmentioning

confidence: 99%

Rerandomization and Regression Adjustment

Ding

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary Randomization is a basis for the statistical inference of treatment effects without strong assumptions on the outcome‐generating process. Appropriately using covariates further yields more precise estimators in randomized experiments. R. A. Fisher suggested blocking on discrete covariates in the design stage or conducting analysis of covariance in the analysis stage. We can embed blocking in a wider class of experimental design called rerandomization, and extend the classical analysis of covariance to more general regression adjustment. Rerandomization trumps complete randomization in the design stage, and regression adjustment trumps the simple difference‐in‐means estimator in the analysis stage. It is then intuitive to use both rerandomization and regression adjustment. Under the randomization inference framework, we establish a unified theory allowing the designer and analyser to have access to different sets of covariates. We find that asymptotically, for any given estimator with or without regression adjustment, rerandomization never hurts either the sampling precision or the estimated precision, and, for any given design with or without rerandomization, our regression‐adjusted estimator never hurts the estimated precision. Therefore, combining rerandomization and regression adjustment yields better coverage properties and thus improves statistical inference. To quantify these statements theoretically, we discuss optimal regression‐adjusted estimators in terms of the sampling precision and the estimated precision, and then measure the additional gains of the designer and the analyser. We finally suggest the use of rerandomization in the design and regression adjustment in the analysis followed by the Huber–White robust standard error.

show abstract

“…Duflo et al . (), Gerber and Green () and Athey and Imbens ()) and technology companies (e.g. Kohavi and Longbotham (2017)).…”

Section: Introductionmentioning

confidence: 99%

Rerandomization and Regression Adjustment

Ding

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

show abstract

“…While most individual characteristics appear balanced between “Categorical Labels” and “No Labels” groups, the gender of participants indicates some random imbalance (see Tables and ). We tested the sensitivity of main results obtained from t‐test by also estimating multivariate regression models that include all observable characteristics, including gender, following recommended guidelines in the event of an observed imbalance (Athey & Imbens, , p. 93).…”

mentioning

confidence: 99%

Categorical cognition and outcome efficiency in impact investing decisions

Lee¹,

Adbi

Singh

2019

Strategic Management Journal

View full text Add to dashboard Cite

Research Summary: The emerging practice of "impact investing" optimizes both financial and social outcomes, and thus promises to support hybrid organizations that simultaneously pursue financial and social goals. We argue, however, that impact investing decisions may be prone to behavioral factors that limit their outcome efficiency. In a portfolio allocation task designed to reflect the essential features of an impact investing decision, we find across a range of scenarios that individuals systematically fail to choose investment portfolios that achieve financial and social outcomes efficiently and thereby waste opportunities for value creation. We further show in online and inperson experiments that outcome inefficiency is related to "categorical cognition": suppression of categorical labels on investment options increases efficiency. Managerial Summary: The "impact investing" approach promises to encourage greater financial investments in hybrid organizations that pursue a combination of financial and social goals. We experimentally demonstrate a challenge of this approach: People struggle to identify portfolios of investments that simultaneously optimize across financial and social outcomes. This is partly due to "categorical cognition": a natural tendency to view investments in terms of known categories rather than the actual outcomes they produce. Our experiments show that removing the labels "for-profit company," "charity," and "social enterprise" from investment options-thus making it more

show abstract

“…Their use has accelerated dramatically in the past 10 to 15 years in academia, reflecting what Angrist and Pischke (2010) call "the credibility revolution." In terms of establishing causal claims, it is generally accepted within the discipline that randomized controlled trials are particularly credible from the point of view of internal validity (Athey and Imbens 2017). However, as critics have pointed out, this credibility applies to the interventions studied-at that time, on that population, implemented by the organization that was studied-but does not necessarily extend beyond.…”

mentioning

confidence: 99%

From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application

Banerjee

Banerji

Berry³

et al. 2017

Journal of Economic Perspectives

238

View full text Add to dashboard Cite

The promise of randomized controlled trials is that evidence gathered through the evaluation of a specific program helps us—possibly after several rounds of fine-tuning and multiple replications in different contexts—to inform policy. However, critics have pointed out that a potential constraint in this agenda is that results from small “proof-of-concept” studies run by nongovernment organizations may not apply to policies that can be implemented by governments on a large scale. After discussing the potential issues, this paper describes the journey from the original concept to the design and evaluation of scalable policy. We do so by evaluating a series of strategies that aim to integrate the nongovernment organization Pratham’s “Teaching at the Right Level” methodology into elementary schools in India. The methodology consists of reorganizing instruction based on children’s actual learning levels, rather than on a prescribed syllabus, and has previously been shown to be very effective when properly implemented. We present evidence from randomized controlled trials involving some designs that failed to produce impacts within the regular schooling system but still helped shape subsequent versions of the program. As a result of this process, two versions of the programs were developed that successfully raised children’s learning levels using scalable models in government schools. We use this example to draw general lessons about using randomized control trials to design scalable policies.

show abstract

The Econometrics of Randomized Experiments

Cited by 502 publications

References 133 publications

Rerandomization and Regression Adjustment

Rerandomization and Regression Adjustment

Categorical cognition and outcome efficiency in impact investing decisions

From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application

Contact Info

Product

Resources

About