Systematic Measurement Error with State-Level Crime Data: Evidence from the “More Guns, Less Crime” Debate

Abstract

Researchers have recently been cautioned regarding error in the Uniform Crime Reports’ (UCR) “Crime by County” cross-sectional time-series data. These data were the basis for analyses of the effects of laws regarding shall-issue concealed carry weapons (CCW) permits on UCR crime rates in the controversial book More Guns, Less Crime (MGLC). The authors conduct a critical analysis of the state-level data used in that study, compare it to readily available state-level UCR data, and discuss issues that may unduly influence the MGLC parameter estimates. Using alternative data, they reestimate the MGLC models, finding that the majority of the MGLC state-level findings are mere artifacts of reporting error and data anomalies resulting from the use of aggregated UCR “Crime by County” data. The authors conclude that any inferences regarding the effects of concealed carry weapons laws on crime rates drawn from analyses of the MGLC state-level data are seriously flawed.

Healthcare indicators and firearm homicide: an ecologic study

“…There are additional limitations associated with the firearm homicide data (Martin and Legault, 2005; Maltz, 2006), which could have impacted our results. Law enforcement agencies report to the DCJS UCR on a voluntary basis, and therefore, some firearm homicides may have been missed.…”

The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of Gun Policies in the United States, Second Edition

“…However, the paper, and presumably the paper, also had an unfavorable ratio of model covariates to observations (less than one to eight), suggesting that the model may have been overfit, and thus its estimates and their CIs may be unreliable. Martin and Legault (2005) demonstrated that Lott (2000) used incorrect state crime rate estimates that differed substantially from official FBI state estimates. They replicated Lott (2000)'s model despite misgivings about its specification to demonstrate that the effects Lott reported were sensitive to this measurement error.…”

Evaluating Methods to Estimate the Effect of State Laws on Firearm Deaths: A Simulation Study

“…We also investigated log-linear (or log-normal) models, which are commonly used in the gun policy literature (e.g., Aneja, Donohue, and Zhang, 2014;Cheng and Hoekstra, 2013;Kendall and Tamura, 2010;Kovandzic, Marvell, and Vieraitis, 2005;La Valle, 2013;La Valle and Glover, 2012;Lott, 2010;Lott and Mustard, 1997;Martin and Legault, 2005). These are linear models conducted on the natural logarithm of the outcome rather than the outcome in its natural units.…”