The twenty-first century experimenting society: the four waves of the evidence revolution

Abstract

This paper presents a personal perspective-drawing especially on the author's experience in international development-of the evidence revolution, which has unfolded in fours waves over the last 30 years: (1) the results agenda as part of New Public Management in the 1990s, (2) the rise of impact evaluations, notably randomized controlled trials (RCTs) since the early 2000s, (3) increased production of systematic reviews over the last ten years, and (4) moves to institutionalize the use of evidence through the emergence of knowledge brokering agencies, most notably the What Works movement in the United States and the United Kingdom. A fifth wave may come from the potential from AI, machine learning and Big Data. Each successive wave has built on the last, and together they comprise the supply side of the evidence architecture. To support the use of evidence demand side activities such as Evidence Needs Assessments and Use of Evidence Awards are proposed.

Research Worth Using: (Re)Framing Research Evidence Quality for Educational Policymaking and Practice

“…Evidence syntheses range from systematic reviews, evidence portals, and policy briefs, to other review-derived products, where the degree and scope of synthesis that is most useful depends on both the context and the purpose of its use (Caird et al, 2015; White, 2019). Different purposes may be to highlight information relevant to decisions, to identify gaps in the policy questions that the research literature has addressed, or to illuminate problems and policy options (Lavis, 2009).…”

“…We also note that syntheses will be most valuable if they undertake a comprehensive search of the available evidence for potential inclusion, such as by including unpublished literature or soliciting nominations of studies submitted by external contributors. To meet the demand for evidence synthesis, we encourage the field to continue to pursue multiple forms of synthesis, including rapid evidence assessments, systematic reviews, meta-analyses, and reviews of reviews (Lavis, 2009; White, 2019).…”

Working Out What Works: The Case of the Education Endowment Foundation in England

“…The Government’s WWC network represents the political hegemony of the What Works movement, which is largely built on the rise of impact evaluations (particularly randomized controlled trials [RCTs]) since the early 2000s and the increased production of systematic reviews over the last 10 years (White, 2019). These developments have been paralleled in the UK with the emphasis placed on evidence-based policy and practice by the incoming New Labour Government in 1997 and taken forward through successive Labour administrations, the following Conservative/Liberal Democrat Coalition Government 2011–2015 and the Conservative Government thereafter (see also Connolly et al, 2017).…”

“…The articles in this issue focus on a general introduction to evidence-based educational reform in different countries and the effectiveness of evidence-based education reforms. There have been various identifiable waves in the process of evidence revolution in education with the 2010s characterized by attempts to institutionalize the use of evidence through the emergence of knowledge brokering agencies, most notably the What Works movement in the U.S. and the UK (White, 2019). This article focuses on the case of the Education Endowment Foundation (EEF) in England and presents an internal perspective on the work of the EEF in generating, documenting, and promoting the use of high-quality evidence and evaluation to inform teaching and other school practices, for transfer and possible adaptation in other contexts.…”

Research Synthesis Infrastructures: Shaping Knowledge in Education

“…Considerations like these take on particular urgency in the present digital age, when the ubiquity of networked, mobile, and cloud computing promise to disrupt the “control zone” (Lagoze, 2010) within which scholarly results are evaluated, disseminated, and curated (see Borgman, 2015). Big data and automation are augmenting the ways research syntheses can be—and are being—produced and used (see Gough, Thomas, & Oliver, 2019; Snilstveit et al, 2018; White, 2019). As Bowker (2017) notes, “The great potential of the current moment is to re-imagine the engines of knowledge production in ways which produce new forms of alignment” (p. 397).…”