The realities of scaling within evidence-based policy

Abstract

<jats:title>Abstract</jats:title> <jats:p>The economic model for scaling described by Al-Ubaydli and colleagues offers recommendations to policymakers who make decisions about whether or not to implement evidence-based programs. The core economic model does not currently acknowledge the broader context of policy decision-making and therefore may fail to achieve its objectives. The model focuses primarily on the generation and use of available research in the decision on whether to scale a program. Research studying the use of evidence in policymaking points to a complex set of factors beyond just the strength of the evidence such as leadership, relationships, timing and financial resources that contribute to decisions to scale a program. Second, there is already a robust evidence-based policy movement at the federal, state and local levels. The economic model should leverage this movement rather than providing recommendations that might stall or redirect the movement. The economic model can push the field to strengthen the available evidence while providing recommendations on selecting models to scale within the currently available evidence. This commentary finishes with suggestions for moving forward.</jats:p>

A rejoinder: ‘How can experiments play a greater role in public policy? Twelve proposals from an economic model of scaling’

“…We also agree on the importance of the politics in the project selection process, and many of the issues raised by Mani (2020) reflect those raised by Ho et al (2020), John (2020) and Supplee and Kane (2020), so we do not add further comments here.…”

“…Ho et al 's (2020) Suggestion #5 aligns perfectly with our checklist (e.g., see the section for researchers within the checklist) that pushes for greater transparency, particularly regarding the information that concerns the characteristics of the population and the situation of the experimental study. Finally, Suggestion #6 echoes in part comments made by John (2020), Mani (2020) and Supplee and Kane (2020), so we will not expand upon this here other than to point the reader to our thoughts laid out above in the introduction to this rejoinder.…”

“…We acknowledge that capturing this political dynamic was beyond the scope of our work. Many of the excellent issues raised were pointed out by Ho et al (2020), Mani (2020) and Supplee and Kane (2020).…”

“…Leveraging insights from the political science literature, John (2020) delves deeper into our footnote 9, noting, “Bureaucrats too are political animals whose interests are tied to those of the politicians who usually command them and who have an interest in the preservation of their bureaus and in promoting their own careers.” We acknowledge that capturing this political dynamic was beyond the scope of our work. Many of the excellent issues raised were pointed out by Ho et al (2020), Mani (2020) and Supplee and Kane (2020).…”

“…At this point, the astute reader might stop and pause to wonder why so many of the pundits focused on the third link in our statement (see Ho et al , 2020; John, 2020; Mani, 2020; Supplee & Kane, 2020). In this context, the focus for some reviewers seemed to be on what we did not model rather than what we did.…”