A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic

Abstract

March 2021protocol via Twitter in order to crowd-source additional relevant studies not identified through the search. All titles and abstracts identified in our search were double-screened using the Rayyan online application. Fourth, to ensure that our analysis is comprehensive in terms of recent and relevant research, on February 14, 2022, and on August 8, 2022, we conducted two comprehensive forward and backward citation searches of all eligible studies identified in the above steps. As noted above, in order to be included in our research sample, estimates of learning progress had to be based on test score measures that could be standardized across studies using Cohen's d.From the studies that meet our inclusion criteria we extracted all estimates of learning progress of school-aged children during the COVID-19 pandemic, separately for math and reading and for different school grades. We also extract the corresponding sample size, standard error, date(s) of measurement, author name(s), and country. Last, we recorded whether studies differentiate between children's socio-economic background, which measure is used to this end, and whether studies find an increase, decrease or no change in learning inequality. We contacted study authors, if any of the information was missing in the study. Data extraction was performed by the first author and validated independently by the second author, with discrepancies resolved through discussion and by conferring with the third author. All extracted information was manually compiled in a spreadsheet using Microsoft Excel (v.16.66.1) and then imported into the statistical program Stata (v.17.0) for our analyses. TimingOur data collection started on April 27, 2021 and ended on August 8, 2022. Data exclusionsStudies were included in the systematic review and meta-analysis based on the inclusion criteria described in the pre-registered study protocol (https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021249944). The PRISMA flow diagram in the manuscript (Figure 1) shows the number of studies included and excluded in the systematic review and meta-analysis, as well as the reasons for exclusion. Non-participationNo participants were involved in this study. RandomizationRandomization is not applicable to this study, since our systematic review and meta-analysis examines all available estimates of learning progress of school-aged children during the COVID-19 pandemic. Reporting for specific materials, systems and methodsWe require information from authors about some types of materials, experimental systems and methods used in many studies. Here, indicate whether each material, system or method listed is relevant to your study. If you are not sure if a list item applies to your research, read the appropriate section before selecting a response.

The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis

“…Not only are several new countries considered in the analysis (e.g., Slovenia, Egypt), but US and UK studies do not dominate the collected empirical evidence. For instance, while in Betthäuser et al (2023) about 71.1% of the effect sizes are derived from these studies, in our paper the corresponding figure is approximately 32.2%. 4 This makes our results of more general relevance.…”

“…One should observe that our findings from the RVE analysis are broadly consistent with those from previous meta-analyses. Storey and Zhang (2021a) concluded that due to Covid-19 students lost, on average, 0.15 standard deviations of learning, König and Frey (2022) found average losses of 0.175 standard deviations, and Betthäuser et al (2023) estimated average losses at 0.14 standard deviations. 16 Two considerations help put these results into perspective.…”

“…In line with the approach adopted by Betthäuser et al (2023) and Hammerstein et al (2021) , the risk of bias in nonrandomized studies was assessed in 38 11 studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool ( Sterne et al, 2016 ). Each study was independently evaluated by the author and another researcher, and any disagreements were resolved through discussion to reach a consensus.…”

“…Our findings are consistent with those in previous relevant meta-analyses. König and Frey (2022) as well as Betthäuser et al (2023) conclude that the presence of publication bias is unlikely.…”

Children and youth in the age of COVID‐19

“…Early in the pandemic, young people were more likely to be infected with COVID‐19 than other age groups (Wark, 2021), and some Australian research suggests that one in five appears to have had long COVID (Biddle & Korda, 2022). In addition to the direct disease impacts, lockdowns and restrictions in Australia have significantly disrupted the education (early childhood education, schooling, university or TAFE) and social lives of children and youth with implications for their mental health and education (Betthauser et al, 2023; Engzell et al, 2021; Meherali et al, 2021). 2 In some cases, the impact of the schooling disruptions was found to have negligible impacts on children's educational outcomes (Gillitzer & Prasad, 2022).…”