Working Out What Works: The Case of the Education Endowment Foundation in England

Abstract

Purpose: This article gives an overview of the successes and lessons learnt to date the Education Endowment Foundation (EEF), one of the leading organizations of the What Works movement. Design/Approach/Methods: Starting with its history, this article covers salient components of the EEF’s unique journey including lessons learned and challenges in evidence generation. Findings: The EEF has demonstrated that it is feasible to rapidly expand the use of school-based randomized controlled trials (RCTs) in a country context, set high standards for research independence, transparency, and design, and generate new evidence on what works. Challenges include the need to consider alternative designs to RCTs to answer a range of practice-relevant questions, how to best test interventions at scale, and how study findings are reported and interpreted. Originality/Value: This article addresses some of the key components required for the success of What Works organizations globally.

Education of Students with Disabilities, Science, and Randomized Controlled Trials

“…In a quote attributed to different individuals but that I first heard from Fixsen et al (2015), "all organziations and systems are perfectly designed to achieve exactly the results they obtain" (p. 702). As a field, we are following a gold standard methodology that primarily generates effect sizes for educational interventions that are hardly better than standard educational practice (Edovald & Nevill, 2021). Certainly, exceptions exist (e.g., Bradshaw et al, 2010), but they are the exception rather than the rule.…”

“…A common experience is that school-based RCTs may affect more proximal measures but fail to produce effects on the standardized, norm-referenced assessments. In their reflection on the RCTs funded by the EEF, Edovald and Nevill (2021) who were staff with the organization noted the absence of significant effects on standardized measures that were distal to the intervention and more frequent significant effects on proximal measures. Recently, Sam et al (2021) conducted an efficacy study of an intervention program to promote teachers' use of evidence-based practices with children with autism in 60 elementary schools.…”

“…By ratcheting up the rigor of the methodological standards, organizations may have created a situation where 75% of the studies find no effects and average effect sizes are very small. Similarly, Edovald and Nevill (2021) have questioned the sanctity of the arbitrary but inflexible .05 significance level in their discussion of the outcomes of EEF studies. As Cartwright (2019) notes, one could be relatively sure that an effect will occur 9 out of 10 times if a study were done again and still have nonsignificant findings.…”

“…When teachers, students and their parents, are randomly assigned to control groups after they have agreed to participate in an experimental intervention that could have some value to them, reactive effects may occur (e.g., attrition in the control group, greater effort by teachers in the control group). In the RCT studies funded through EEF, Edovald and Nevill (2021) reported sometimes reactive effects of participants assigned to services-as-usual (SAU) groups (e.g., high attrition). Anecdotally, in a recent RCT study that stratified random assignment (i.e., citation withheld by author for confidentiality), two schools within a district were randomly assigned to a school-wide high school intervention and a SAU condition.…”

Evidence-Based Reform in Education: Global Perspectives

“…Pellegrini and Vivanet (2020) also discuss the most relevant challenges facing the development of evidence-informed education policies in Europe. Like Pellegrini and Vivanet (2020), Edovald and Nevill (2020) summarize the successes and lessons of the Education Endowment Foundation (EEF), one of the leading organizations of the What Works movement. The success of the EEF demonstrates that it is feasible to rapidly expand the use of school-based randomized controlled trials in a country context, set high standards for research independence, transparency, and design, and generate new evidence on what works.…”