Do Test Score Gaps Grow before, during, or between the School Years? Measurement Artifacts and What We Can Know in Spite of Them

Abstract

Do test score gaps between advantaged and disadvantaged children originate inside or outside schools? One approach to this classic question is to ask (1) How large are gaps when children enter school? (2) How much do gaps grow later on? (3) Do gaps grow faster during school or during summer? Confusingly, past research has given discrepant answers to these basic questions.We show that many results about gap growth have been distorted by measurement artifacts. One artifact relates to scaling: Gaps appear to grow faster if measurement scales spread with age. Another artifact relates to changes in test form: Summer gap growth is hard to estimate if children take different tests in spring than in fall.Net of artifacts, the most replicable finding is that gaps form mainly in early childhood, before schooling begins. After school begins, most gaps grow little, and some gaps shrink. Evidence is inconsistent regarding whether gaps grow faster during school or during summer. We substantiate these conclusions using new data from the Growth Research Database and two data sets used in previous studies of gap growth: the Beginning School Study and the Early Childhood Longitudinal Study, Kindergarten Cohort of 1998-1999

Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement

“…This variability in estimates across studies can be seen in Table 1, where summer drop estimates range from 0.001 to 0.010 SDs per day of school missed across grades/ subjects. Despite the inconsistencies in the magnitude of summer loss, research using both recent data sources is consistent in failing to replicate the earlier finding of summer being the primary period in which socioeconomic inequalities widen (e.g., Kuhfeld, 2019;von Hippel, 2019;von Hippel & Hamrock, 2019). Additionally, both datasets indicate that summer is a particularly variable time for learning, with far higher variability in growth rates during the summer than the school year (Atteberry & McEachin, 2020;von Hippel et al, 2018; In fact, some students actually show learning gains during the summer.)…”

“…We do not expect all students to be affected by COVID-19 school closures equally. Prior summer learning loss research indicated that there is a considerable variability in students' learning patterns over the summer (e.g., Atteberry & McEachin, 2020;Kuhfeld et al, 2019), most of which cannot be explained by observed student and family characteristics (Borman et al, 2005;Kuhfeld, 2019;von Hippel et al, 2019). In addition to producing average estimates of learning rates during time out of school, we estimated variation across students in (a) out-of-school (summer) learning rates and (b) projected student test scores as schools reopen this fall under both typical and COVID Slide conditions (see Appendix C for more details).…”

Understanding Socioeconomic Differences in Parents’ Speech to Children

“…The SES-based achievement gap in language and literacy skills in the United States grows the most during early childhood and is large at the start of kindergarten (45). Most parentfocused interventions to improve the development of children's language and literacy have targeted middle-SES families or families with children who have language delays, but a growing number of promising parenting interventions that focus on the word gap aim to prevent it from emerging (e.g., Providence Talks, Thirty Million Words).…”

Opportunities and Challenges Arising from Holiday Clubs Tackling Children’s Hunger in the UK: Pilot Club Leader Perspectives

“…Holiday hunger is likely to have a detrimental effect on school children’s educational performance, health and wellbeing. Although there are common issues with measurement of changes in educational attainment, gaps in attainment between US children in high and low poverty schools which increase across the summer have been reported [6], with summer learning loss also being evidenced among low socioeconomic status (SES) school children in the UK [7]. Moreover, food insecurity is associated with poorer health status and emotional wellbeing [8].…”