Improving Inference from Simple Instruments through Compliance Estimation

Abstract

Instrumental variables (IV) regression is widely used to estimate causal treatment effects in settings where receipt of treatment is not fully random, but there exists an instrument that generates exogenous variation in treatment exposure. While IV can recover consistent treatment effect estimates, they are often noisy. Building upon earlier work in biostatistics (Joffe and Brensinger, 2003) and relating to an evolving literature in econometrics (including Abadie et al., 2019;Huntington-Klein, 2020;Borusyak and Hull, 2020), we study how to improve the efficiency of IV estimates by exploiting the predictable variation in the strength of the instrument. In the case where both the treatment and instrument are binary and the instrument is independent of baseline covariates, we study weighting each observation according to its estimated compliance (that is, its conditional probability of being affected by the instrument), which we motivate from a (constrained) solution of the first-stage prediction problem implicit to IV. The resulting estimator can leverage machine learning to estimate compliance as a function of baseline covariates. We derive the large-sample properties of a specific implementation of a weighted IV estimator in the potential outcomes and local average treatment effect (LATE) frameworks, and provide tools for inference that remain valid even when the weights are estimated nonparametrically. With both theoretical results and a simulation study, we demonstrate that compliance weighting meaningfully reduces the variance of IV estimates when first-stage heterogeneity is present, and that this improvement often outweighs any difference between the compliance-weighted and unweighted IV estimands. These results suggest that in a variety of applied settings, the precision of IV estimates can be substantially improved by incorporating compliance estimation.

The thin blue line in schools: New evidence on school‐based policing across the U.S.

“…A value of less than 1-0.5 for instance-could mean that the school has a 50% likelihood of receiving a full-time SRO or that the school has a 100% likelihood of sharing a single SRO with another school. We can use this measure to estimate an "interacted RD" model that interacts the above-discontinuity indicator with our compliance measure to strengthen the first stage prediction of school SRO presence (Caetano et al, 2021;Coussens & Spiess, 2021;Huntington-Klein, 2020). Coussens & Spiess (2021) showed that an interacted model estimates a "super local average treatment effect" (SLATE), in our case giving more weight to schools for which the agency explicitly requests more SROs.…”

“…We can use this measure to estimate an "interacted RD" model that interacts the above-discontinuity indicator with our compliance measure to strengthen the first stage prediction of school SRO presence (Caetano et al, 2021;Coussens & Spiess, 2021;Huntington-Klein, 2020). Coussens & Spiess (2021) showed that an interacted model estimates a "super local average treatment effect" (SLATE), in our case giving more weight to schools for which the agency explicitly requests more SROs. 18 Intuitively, there is more information in observations for schools where the agency says with certainty that one or more SROs will be placed than in a school that we expect to share a new SRO with several other schools.…”

“…AlthoughCoussens and Spiess (2021) showed that including an interaction is equivalent to weighting by the variable interacted with the instrument, we do not explicitly weight by the requested number of SROs. We continue to weight by the number of students in the school, thereby providing an estimate of an impact for the average student in a "compliant" school.…”

Zero to Hero: Exploiting Null Effects to Achieve Variance Reduction in Experiments with One-sided Triggering

“…There are, however, extensions of the standard IV estimator that attempt to improve estimation efficiency. Weighted IV methods [3,15,22] use predicted compliance to weight both treatment and control groups when computing the LATE estimator. Our method is similar in that we also use predicted compliance or triggering probabilities, but whereas the weighted IV estimator is unbiased for LATE only when there is no correlation between treatment effect heterogeneity and compliance, our estimator is unbiased for ITT (and asymptotically unbiased for LATE after dividing by the triggering rate) as long as the augmentation term that we construct equals zero in expectation.…”

“…In practice, we recommend using bootstrap [8] to estimate Cov(Δ(𝑌 ), τ0 ) and Var( τ0 ), and to refit the triggering probability model for each bootstrap sample. We can then estimate 3 the variance of 𝜏 𝑡𝑟𝑔1 with…”