Using the early childhood environment rating scale-Revised in high stakes contexts: Does evidence warrant the practice?

Abstract

Increasingly, states establish different thresholds on the Early Childhood Environment Rating Scale-Revised (ECERS-R), and use these thresholds to inform high-stakes decisions. However, the validity of the ECERS-R for these purposes is not well established. The objective of this study is to identify thresholds on the ECERS-R that are associated with preschool-aged children's social and cognitive development. Applying non-parametric modeling to the nationally-representative Early Childhood Longitudinal Study Birth Cohort (ECLS-B) dataset, we found that once classrooms achieved a score of 3.4 on the overall ECERS-R composite score, there was a leveling-off effect, such that no additional improvements to children's social, cognitive, or language outcomes were observed. Additional analyses found that ECERS-R subscales that focused on teaching and caregiving processes, as opposed to the physical environment, did not show leveling-off effects. The findings suggest that the usefulness of the ECERS-R for discerning associations with children's outcome may be limited to certain score ranges or subscales.

The when and what of measuring ECE quality: Analysis of variation in the Classroom Assessment Scoring System (CLASS) across the ECE day

“…Modifications to standard measure protocols (Mashburn, 2017) or measure usage (Setodji et al, 2018) in policy and practice applications have exacerbated concern.…”

“…Recent critiques of quality assessment in ECE have highlighted the need to establish if existing measures can safely guide policy and practice decisions (Mashburn, 2017;Setodji et al, 2018). A well-functioning assessment measure of ECE quality must meet the two fundamental psychometric assumptions of validity and reliability.…”

“…Exacerbating concerns about reliability and validity is the practice of modifying ECE quality assessment tools for application in policy and practice decisions (Buell, Han, & Vukelich, 2017;Mashburn, 2017;Setodji et al, 2018). Mashburn (2017), for example, examined the assumptions underpinning the use of CLASS in determining funding renewal for Head Start grantees.…”

“…Content and nature of educator-child interactions in ECE programs may also be targeted to meet assessment criteria, and therefore impact children's experiences (Harms, Clifford, & Cryer, 2014;Pianta, Mashburn, Downer, Hamre, & Justice, 2008). Yet the significant faith placed in quality assessment tools is increasingly being questioned (Mashburn, 2017;Pianta, Downer, & Hamre, 2016;Setodji, Schaack, & Le, 2018).…”

Infants in Dutch daycare: Exploring fine‐grained dimensions of temperament

“…Global process quality includes the quality of interactions of the child with caregivers and physical environmental features, such as the quality of the furniture, the organization of the classroom and the quantity of various materials. The widely used measures ITERS‐R (Harms et al, 2003) and ECERS‐R (Harms et al, 1998), which are designed to evaluate global process quality, are comprehensive in scope, but fewer items capture process variables such as the quality of child‐to‐adult interactions (Bisceglia et al, 2009; Setodji et al, 2018). The global process quality of center‐based child care is at a medium level in various countries, according to Vermeer et al (2016), although the quality of center‐based care for infants is generally lower than for children in early childhood education (i.e., measured with ITERS‐R and ECERS‐R, respectively).…”