Mending the gap: Measurement needs to address policy implementation through a health equity lens

Abstract

Policies represent a key opportunity to improve the health outcomes of populations, and if implemented well, can reduce disparities affecting marginalized populations. Many policies are only evaluated on whether they elicit their intended health outcome. However, a lack of understanding regarding if and how they are implemented may hinder the intended impact overall and on addressing health disparities. Implementation science offers an array of frameworks and methodological approaches for assessing policy delivery, yet few examples exist that meaningfully include health equity as a core focus. This commentary describes the importance of equity-informed implementation measurement by providing case examples and implications for assessment. In addition, we highlight examples of emerging work in policy implementation grounded in health equity with suggested steps for moving the field forward. The ultimate goal is to move toward open-access measurement approaches that can be adapted to study implementation of a variety of policies at different stages of implementation, driven by input from marginalized populations and implementation practitioners, to move the needle on addressing health disparities.

Design considerations for developing measures of policy implementation in quantitative evaluations of public health policy

“…This represents a key area where implementation scientists with expertise in measure development and evaluation could greatly enhance policy-focused implementation science. Ideally, determinant and outcome measures would be transferrable across levels of policy (e.g., local, state, national), consistent within content areas, and include a focus on health equity ( 13 – 15 , 35 ). Transferrable measurements will greatly enhance our ability to derive broadly generalizable knowledge from policy studies like those discussed here.…”

“…Measures that are too study-specific will have limited generalizability (though potentially higher internal validity) and limit the ability of broader learning for the field of policy-focused implementation science. Including attention to health equity in measure development will help provide comprehensive understanding of how marginalized populations are impacted by policies ( 15 ). As measures are developed, improving the coordination and use of common measures through publicly available measure repositories is crucial to improving the efficiency, reproducibility, and learning potential of policy-focused implementation science research ( 15 ).…”

“…Including attention to health equity in measure development will help provide comprehensive understanding of how marginalized populations are impacted by policies ( 15 ). As measures are developed, improving the coordination and use of common measures through publicly available measure repositories is crucial to improving the efficiency, reproducibility, and learning potential of policy-focused implementation science research ( 15 ). Existing repository and field-building efforts can provide guidance for how to build and disseminate such repositories ( 36 – 38 ).…”

“…To appropriately evaluate policy impacts on health, we must adequately measure how the policy is implemented, rather than assuming a policy is implemented just because it is “on the books” ( 3 ). In turn, this can help researchers better understand the full picture of why policies do or do not affect health outcomes, and their impact on health disparities ( 3 , 10 , 15 ). Policy-focused implementation science research focuses on understanding just this, but the measurement of policy implementation is lacking.…”

“…Using such analytic approaches ultimately depends on being able to measure policy implementation in a rigorous way, and there is a major lack of valid and reliable measures of health policy implementation determinants, processes, and outcomes ( 10 – 15 ). As a starting point to address the policy implementation measurement gap, provide guidance for researchers designing studies, and ultimately improve how policy implementation is analytically incorporated into quantitative policy evaluation research, we discuss four considerations for developing or identifying existing measures of policy implementation.…”