Misclassification in Binary Choice Models

Abstract

While measurement error in the dependent variable does not lead to bias in some well-known cases, with a binary dependent variable the bias can be pronounced. In binary choice, Hausman, Abrevaya and Scott-Morton (1998) show that the marginal effects in the observed data differ from the true ones in proportion to the sum of the misclassification probabilities when the errors are unrelated to covariates. We provide two sets of results that extend this analysis. First, we derive the asymptotic bias in parametric models allowing for correlation of the errors with both observables and unobservables. Second, we examine the bias in a prototypical application in two different datasets, using a variety of methods that differ in the amount of knowledge that is assumed about the error process. Our application is receipt of food stamps, the largest and most widely received welfare program in the U.S. Monte Carlo results and our empirical application show that the bias formulas accurately describe the bias in finite samples. Our results indicate that the robustness of signs and relative magnitudes of coefficients implied by the earlier proportionality results does not necessarily extend to estimated Probit coefficients, and does not apply when errors are correlated with covariates. Using administrative records linked to survey data as validation data, we evaluate estimators that are consistent under misclassification. Estimators based on the assumption that misclassification is independent of the covariates are sensitive to their functional form assumptions and aggravate the bias if the conditional independence assumption is invalid in all cases we examine. On the other hand, estimators that allow misreporting to be correlated with the covariates perform well if an accurate model of misreporting or validation data are available. Estimators that incorporate more information about the errors, such as aggregate underreporting rates, tend to be more robust to misspecification of the misreporting model.

Implications of Misclassification Errors in Empirical Studies of Adolescent Smoking Behaviours

“…In the absence of within-sample validation data, true marginal effect estimates can be recovered either by plugging in misclassification probabilities obtained from external sources as suggested in other literature (Meyer and Mittag, 2014) and represented here using HAS approach 1. Alternatively, both covariate effects and misclassification probabilities can be estimated directly from the data using HAS approach 2 11 .…”

“…Arguably, wider application of HAS approach 1 is likely to be limited, given that the extent of misclassification error can be context-specific and vary with survey design features. Furthermore, the performance of both HAS approaches 1 and 2 largely depends on whether the conditional independence assumption of the misclassification probabilities holds (Meyer and Mittag, 2014). This assumption is tested here, and although the findings show that the misclassification probabilities are weakly correlated with model covariates, this may not be the case in other data.…”

“…However, if the conditional independence assumption fails, HAS approach can perform worse than the uncorrected model (Meyer and Mittag, 2014). Therefore, the conditional independence assumption is tested using a set of probit models on subsamples of cotinine-validated nonsmokers (Eqn 9) and cotinine-validated smokers (Eqn 10):…”

“…Conversely, white adolescents (Bauman and Ennett, 1994) and adolescents embedded within social networks of other smokers are more likely to over-report smoking behaviours (Dolcini et al, 1996;Mermelstein et al, 2002;Stein et al, 2002). Given that the performance of HAS approaches 1 and 2 can be influenced by the conditionally independence assumption (Meyer and Mittag, 2014), this study examines the association between the misclassification errors and adolescent characteristics. Where the misclassification errors are observed to be conditionally dependent on model covariates, HAS approach 2 is extended to allow for this correlation (HAS approach 3).…”

Household Determinants of the Adoption of Improved Cassava Varieties using DNA Fingerprinting to Identify Varieties in Farmer Fields: A Case Study in Colombia

“…It is also plausible that misidentification in previous studies has led to measurement errors. For the dependent variable (ID use) this can lead to biased and inconsistent estimators, particularly if using binary models where the bias can be more evident (Hausman, 2001;Meyer and Mittag, 2014).…”