Constructed Second Control Groups and Attenuation of Unmeasured Biases

Abstract

The informal folklore of observational studies claims that if an irrelevant observed covariate is left uncontrolled, say unmatched, then it will influence treatment assignment in haphazard ways, thereby diminishing the biases from unmeasured covariates. We prove a result along these lines: it is true, in a certain sense, to a limited degree, under certain conditions. Alas, the conditions are neither inconsequential nor easy to check in empirical work; indeed, they are often dubious, more often implausible. We suggest the result is most useful in the computerized construction of a second control group, where the investigator can see more in available data without necessarily believing the required conditions. One of the two control groups controls for the possibly irrelevant observed covariate, the other control group either leaves it uncontrolled or forces separation; therefore, the investigator views one situation from two angles under different assumptions. A pair of sensitivity analyses for the two control groups is coordinated by a weighted Holm or recycling procedure built around the possibility of slight attenuation of bias in one control group. Issues are illustrated using an observational study of the possible effects of cigarette smoking as a cause of increased homocysteine levels, a risk factor for cardiovascular disease.

Instrumental variables as bias amplifiers with general outcome and confounding

“…Therefore, a more practical approach, as suggested by Rosenbaum (2010 , Ch. 18.2), Brookhart et al (2010) and Pimentel et al (2016) , may be to conduct analysis both with and without adjusting for the covariate. If two analyses give similar results, as in the example in Table 2 , then we need not worry about Z-bias; otherwise, we need additional information and analysis before making decisions.…”

Rebar: Reinforcing a Matching Estimator With Predictions From High-Dimensional Covariates

“…Most recently, Steiner, Cook, Li, and Clark (2015) argued via case study for including all available covariates, unless "strong substantive theory" (p. 573) suggests the presence of bias-amplifying covariates covariates (they write that bias amplification "seems less likely as the size of the covariate set increases"); ideally, researchers should include covariates from multiple domains, with each domain including as many covariates as possible. Pimentel, Small, and Rosenbaum (2016) suggested conducting two analyses, each matching on a different set of covariates. Methods attempting to limit the MSE penalty by limiting propensity modeling variables to those that correlate with observed outcomes have been met with criticism of a different nature: In Rubin's view, in order to maximize objectivity, during matching researchers should keep outcome measurements in a virtual locked box, only to emerge once the matching structure and other study design elements have been determined (Rubin, 2008).…”

Large, Sparse Optimal Matching with R package rebalance

“…However in some cases it may be desirable to select controls with a covariate distribution that differs in some way from the treated distribution. For example, (Pimentel et al, 2015b) describes how unmeasured biases in observational studies may be attenuated if treated and control distributions differ on certain "innocuous" covariates. Another situation where differences in treated and control distributions might be desirable is when a number of smaller matches are being performed and the researchers desire to make each control group similar to a single reference distribution.…”