Assessing the range and evidence‐base of interventions in a cluster of schools

Abstract

The last decade has seen an increased focus through policy and research for schools to move towards an evidence‐informed practice. Although some practitioners now access the external research evidence when deciding which interventions to adopt in their school, research suggests many still do not. Instead, approaches to teaching and learning are often informed by trends and the opinions and experiences of practitioners. Little is known about what intervention programmes/approaches are used in schools and whether they are evidence‐based. We conducted this study to assess the range and evidence of interventions used in a school cluster in Wales, which comprised two secondary schools, seven primary schools and one special school. Using questionnaires, we evidenced 242 interventions. Following screening, we included 138 of these in the analysis and categorised them according to the ‘SEN Areas of Need’. We then conducted a rapid systematic review of the literature for these interventions and found that 30% had some evidence of positive impact on pupil outcomes, 67% had no published evidence, and 3% had causal evidence to suggest they were ineffective. One year later, we conducted a follow‐up study to assess if schools’ knowledge of the evidence for the interventions, presented through a summary report, had an impact on their provision. Our data suggest that the reports had very little impact on existing provision, and some schools continued to use the same interventions. The limitations of this study and directions for the cluster, policy and research are discussed.

From evidence‐informed to evidence‐based: An evidence building framework for education

“…We should be wary of the widespread adoption of programmes that lack robust evaluations and/or cannot be shown to work in everyday school settings (Pegram et al, 2022). Over the years, research has found several widely adopted educational theories such as learning styles (Evidence for Learning, n.d.; Pashler et al, 2009) and brain training apps (Spaulding et al, 2010; Stephenson, 2009) to have little positive impact on learner outcomes and/or be theoretically flawed.…”

“…In a recent study, Pegram et al (2022) identified 138 different interventions in use across a cluster of 10 schools—67% of these interventions had no published research evidence to support them and only 11% had promising evidence of positive causal impact on pupil outcomes. This poses important questions around why some schools adopt and use interventions with a limited evidence‐base, and whether it is even practical to implement multiple interventions to a high level of fidelity.…”

“…A definition for education research Sometimes stakeholders do not adopt an approach/intervention when research efforts end, even if it has a promising evidence-base. Although it is not an exhaustive list, there are several reasons schools may opt to stop using an approach/intervention after a research study ends, including: In a recent study, Pegram et al (2022) identified 138 different interventions in use across a cluster of 10 schools-67% of these interventions had no published research evidence to support them and only 11% had promising evidence of positive causal impact on pupil outcomes. This poses important questions around why some schools adopt and use interventions with a limited evidence-base, and whether it is even practical to implement multiple interventions to a high level of fidelity.…”

Special issue: Evidence into policy and practice

“…Somewhat depressingly, Pegram et al (2022) show us that most programmes and interventions used in schools are not evidence-led, and some are harmful. Of 242 interventions used in a cluster of schools, only 30% had any evidence of benefit.…”

An Exploratory Study of Pre-Service Teachers’ Evidence-Based Practice Related Knowledge, Attitudes, and Practices

“…Evidence-based practice (EBP) has become a standard of care in psychology (see APA Presidential Task Force on Evidence-Based Practice, 2006); however, EBP has not yet become a standard of practice in the field of education. There have been increasing calls to close the gap between research and practice in education, as studies have identified a disparity between educational research and practice both at the policy level and in schools (see Anwer & Reiss, 2022; Broekkamp & van Hout-Wolters, 2007; Pegram et al, 2022). This discrepancy was most recently exemplified by the Ontario Human Rights Commission’s (2022) Right to Read Report, which described the poor literacy outcomes resulting from a lack of evidence-based reading instruction in the provincial curriculum.…”