Measuring Law for Evaluation Research

Abstract

Evaluations that combine social science and law have tremendous potential to illuminate the effects of governmental policies and yield insights into how effectively policy makers' efforts achieve their aims. This potential is infrequently achieved, however, because such interdisciplinary research contains often overlooked substantive and methodological challenges. This article offers detailed guidance for conducting successful multidisciplinary evaluations that use legal data. It addresses major issues that commonly arise and offers practical solutions based both on the authors' extensive experience and recommended best practices developed in concert with the Robert Wood Johnson Foundation's Public Health Law Research Program.

Empirical evidence on legal levers aimed at addressing child maltreatment

“…For one, law is difficult to measure. There is variation in the interpretation of laws, which leads to variation in how laws are implemented (Tremper, Thomas, & Wagenaar, 2010). No central database documents when and where laws are enacted, implemented, enforced, or even repealed.…”

Mapping “Trauma-Informed” Legislative Proposals in U.S. Congress

“…Policy mapping is a methodology in which policies and/or policy proposals related to an issue are systematically identified and then analyzed using content analysis (Burris, 2015; Burris et al, 2010; Tremper, Thomas, & Wagenaar, 2010). By cataloguing the policies ‘on the books,’ mapping studies can identify policy gaps and provide a foundation for future research on policy implementation, enforcement, and impact (Burris et al, 2010).…”

Editors' Introduction

“…Legal text can form the basis for a variable predicting an outcome or, in this case, an outcome predicted by exogenous circumstances. Translating legal text into one or more quantitative variables can be an arduous task for many reasons, including the inherent subjectivity involved in classifying text into categories (Tremper, Thomas, and Wagenaar 2010;Anderson et al 2012). VanSickle-Ward and Hollis-Brusky, for example, were confronted with the need to distinguish ''high,'' ''moderate,'' and ''low'' specificity in statutory language.…”

“…The RWJF Program on Public Health Law Research has been instrumental in building this consensus by sponsoring and disseminating methodological guides (Program on Public Health Law Research 2013a), most notably Charles Tremper and colleagues' work on how to gather and measure statutes and regulations (Tremper, Thomas, and Wagenaar 2010;Anderson et al 2012). These guides have been collected in a book (Wagenaar and Burris 2013) that presents a comprehensive catalog of available methods for PHLR and best practices for applying each method to the study of legal questions.…”

“…Data sets also require constant updating to reflect new developments in the law. The availability of standardized protocols for obtaining and coding laws (Tremper, Thomas, and Wagenaar 2010;Anderson et al 2012) and practical tools to facilitate this work (Program on Public Health Law Research 2013a) at least reduce the time costs associated with building and updating data sets, as well as leading to higher-quality results.…”