Factors Associated With Accuracy in Prekindergarten Teacher Ratings of Students’ Mathematics Skills

Abstract

The No Child Left Behind Act requires that 95% of students in all public elementary and secondary schools are assessed in mathematics. Unfortunately, direct assessments of young students can be timely, costly, and challenging to administer. Therefore, policy makers have looked to indirect forms of assessment, such as teachers’ ratings of student skills, as a substitute. However, prekindergarten teachers’ ratings of students’ mathematical knowledge and skills are only correlated with direct assessments at the .50 level. Little is known about factors that influence accuracy in teacher ratings. In this study, we examine the influence of student and teacher characteristics on prekindergarten teachers’ ratings of students’ mathematical skills, controlling for direct assessment of these skills. Results indicate that students’ race/ethnicity and social competency, as well as teachers’ self-efficacy, are significantly related to prekindergarten teachers’ ratings of students’ mathematical skills.

The accuracy of teachers' judgments for assessing young children's emerging literacy and math skills

“…Teacher education and years of experience were not related to teacher accuracy in the current study. Previous research has reported similar negative findings with respect to these variables (Furnari et al., ), or mixed minor effects with higher teacher education actually being associated with less judgment accuracy (overestimation) for some student subgroups (Ready & Wright, ). Thus, general educational attainment by teachers, as measured in these studies (e.g., degree achieved), does not appear to enhance judgment accuracy.…”

“…Agreement between teachers’ judgments and the performance measure was lower for math skills ( r = .47, mean percent = 66%). Previous research has reported similar levels of agreement for pre‐K math (i.e., r = .50; Furnari et al., ; and r = .53, Kilday et al., ), suggesting that pre‐K teachers’ judgments may be less accurate in this domain. Whereas this decrement in math accuracy, relative to language arts, does not appear to be true of teachers working with older students (Südkamp et al., ), research indicates that pre‐K teachers view math as less important to teach than literacy (Ginsburg, Lee, & Boyd, ; Kowalski et al., ) and spend less time teaching it (Cross, Woods, & Schweingruber, ).…”

“…These judgments also play a role in referring children to special programs (Cooc, ; Elhoweris, ; Gerber & Semmel, ; Gresham, MacMillan, & Bocian, ) and can influence student motivation and self‐perception (Black & Wiliam, ; Brookhart, ). Teachers’ judgments of students’ skills and abilities also influence feedback students receive and parental communication (Harlen, ; Hoge & Coladarci, ) and are increasingly used for accountability purposes, especially in early education programs (Furnari, Whittaker, Kinzie, & DeCoster, ; Kim, Lambert, & Burts, ) where the cost of administering individual assessments to young children is prohibitive and early childhood experts discourage the use of “abstract paper‐and‐pencil tests,” arguing instead for more meaningful authentic forms of assessment (Bagnato, Neisworth, & Pretti‐Frontczak, ; Division for Early Childhood (DEC), 2009, ; Meisels & Atkins‐Burnett, ; National Association for the Education of Young Children and National Association of Early Childhood Specialist in State Departments of Education, ; Shepard et al., , p. 6) .…”

“…Despite the importance of teachers’ judgments in pre‐K settings, little research has been done on their accuracy, and what has been done suggests that these judgments may be less accurate than those of teachers working with older students. Recent research examining teachers’ judgments of pre‐K students’ math skills reported correlations of .50 (Furnari et al., ) and .53 (Kilday, Kinzie, Mashburn, & Whittaker, ) between teachers’ ratings and student skills demonstrated by direct assessment. Research examining early literacy skills reports similar findings with a mean correlation of .52 between teacher ratings and direct assessment of three indices of early literacy (Cabell, Justice, Zucker, & Kilday, ).…”

Real World Compromises: Policy and Practice Impacts of Kindergarten Entry Assessment‐Related Validity and Reliability Challenges

“…This issue is especially salient when discussing KEAs, as many of these measures rely on observational rubrics as opposed to direct assessments and thus may be unfamiliar to the kindergarten teachers who are tasked with administering them (Butts, ; Schultz, ; Weisenfeld, ). Moreover, the process of making reliable judgments about students' skills based on a rubric or rating scale can be challenging for many teachers (Cabell, Justice, Zucker, & Kilday, ; Furnari, Whittaker, Kinzie, & DeCoster, ; Kilday, Kinzie, Mashburn, & Whittaker, ; Mashburn & Henry, ; Waterman, McDermott, Fantuzzo, & Gadsden, ). Rubric or rating scale reliability can be affected if the scores are perceived to contribute to some type of consequential decision for the student, teacher, or program, as well (Harvey, Fischer, Weieneth, Hurwitz, & Sayer, ; Waterman et al, ).…”