How accurately do teachers’ judge students? Re-analysis of Hoge and Coladarci (1989) meta-analysis

Kaufmann, Esther

doi:10.1016/j.cedpsych.2020.101902

Cited by 16 publications

(10 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deater-Deckard et al [ 42 ] provided evidence that perceived polarities of pupils’ temperament were related to the mismatch between the assigned grades and children’s abilities. On the other hand, there are reports [ 157 ] supporting the accuracy of teachers’ assessments.…”

Section: Discussionmentioning

confidence: 99%

Temperament and Academic Achievement in Children: A Meta-Analysis

Nasvytienė

Lazdauskas

2021

EJIHPE

View full text Add to dashboard Cite

This study aimed to systematize the diverse and rather controversial findings of empirical research on the relationship between the temperament and academic achievement of school children, as well as to determine the average effect size between these variables. We included 57 original studies of published and unpublished research conducted in 12 countries between 1985 and 2019, with cumulative sample size of 79,913 (varying from 6333 to 14,126 for links between particular temperament dimensions and specific domains of achievement). A random-effects and mixed-effects model was fitted to the data for the central tendency of the temperament–achievement relation and for analyzing moderators, respectively. The high heterogeneity of studies was tackled by selected specific moderators, namely, education level, transition status, family’s socio-economic level, and sources of report on achievement and temperament. The main findings of this meta-analysis affirmed the positive association of effortful control (EC) and inverse relationship of negative affectivity (NA) with a child’s academic performance, together with no apparent trend of surgency (SU) in this relationship; additionally, the sources of report significantly moderated the link between temperament and academic achievement.

show abstract

Section: Discussionmentioning

confidence: 99%

Temperament and Academic Achievement in Children: A Meta-Analysis

Nasvytienė

Lazdauskas

2021

EJIHPE

View full text Add to dashboard Cite

show abstract

“…Recent reviews and meta-analyses on teachers' judgment accuracy have stressed the urgent need for intervention studies on improving the quality of teachers' judgment (Kaufmann, 2020;Urhahne & Wijnia, 2021). Facilitating diagnostic competencies is a relatively new field with only nine empirical studies, none of which contain concrete interventions or suggesting methods to foster the quality of teachers' judgments on students' written performances (Chernikova et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Don't Just Judge the Spelling! The Influence of Spelling on Assessing Second-Language Student Essays

Jansen

Vögelin

Machts

et al. 2021

FLR

View full text Add to dashboard Cite

When judging subject-specific aspects of students’ texts, teachers should assess various characteristics, e.g., spelling and content, independently of one another since these characteristics are indicators of different skills. Independent judgments enable teachers to adapt their classroom instruction according to students’ skills. It is still unclear how well teachers meet this challenge and which intervention could be helpful to them. In Study 1, N = 51 pre-service teachers assessed four authentic English as a Second Language (ESL) essays with different overall text qualities and different qualities of spelling using holistic and analytic rating scales. Results showed a negative influence of the experimentally manipulated spelling errors on the judgment of almost all textual characteristics. In Study 2, an experimental prompt was used to reduce this judgment error. Participants who were made aware of the judgment error caused by spelling errors formed their judgments in a less biased way, indicating a reduction of bias. The determinants of the observed effects and their practical implications are discussed.

show abstract

“…To narrow down this expansive subject, we concentrate specifically on the relative judgment accuracy of teachers in the following sections. We posit that a systematic examination of these various forms of judgment accuracy is imperative, as they are interdependent (see [ 15 ]). Furthermore, within the realm of studies concerning teachers’ relative judgment accuracy, two distinct categories emerge: traditional teacher judgment studies and those rooted in social judgment theory [ 16 – 18 ].…”

Section: Introductionmentioning

confidence: 99%

“…Empirical evidence supports the notion that tests serve as benchmarks for evaluating the accuracy of teachers’ judgments (see [ 19 ]; also [ 20 ]). Given the importance of this topic, numerous studies have been conducted, resulting in various meta-analyses (e.g., [ 15 , 21 , 22 ]). These meta-analyses have offered insights into the overall accuracy of teachers’ judgments.…”

Section: Introductionmentioning

confidence: 99%

“…A recent re-meta-analysis of Hoge and Coladarci’s [ 21 ] original review suggests that the accuracy of teachers’ judgments (initial meta-analysis: r = .65) has been underestimated, with the replicated meta-analysis showing higher correlations ( r = .80 / r = .74 when accounting for a measurement error of .90). The study found no evidence that the accuracy of teachers’ judgments varied depending on whether tasks were classified as “classical” or aligned with social judgment theory (for detailed insights, refer to below), nor did they find evidence that teachers’ judgment accuracy depended on the subject (language, mathematics, or miscellaneous), whether a student had a learning disability, or grade level (see [ 15 ]).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Teachers’ judgment accuracy: A replication check by psychometric meta-analysis

Kaufmann

2024

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Teachers’ judgment accuracy is a core competency in their daily business. Due to its importance, several meta-analyses have estimated how accurately teachers judge students’ academic achievements by measuring teachers’ judgment accuracy (i.e., the correlation between teachers’ judgments of students’ academic abilities and students’ scores on achievement tests). In our study, we considered previous meta-analyses and updated these databases and the analytic combination of data using a psychometric meta-analysis to explain variations in results across studies. Our results demonstrate the importance of considering aggregation and publication bias as well as correcting for the most important artifacts (e.g., sampling and measurement error), but also that most studies fail to report the data needed for conducting a meta-analysis according to current best practices. We find that previous reviews have underestimated teachers’ judgment accuracy and overestimated the variance in estimates of teachers’ judgment accuracy across studies because at least 10% of this variance may be associated with common artifacts. We conclude that ignoring artifacts, as in classical meta-analysis, may lead one to erroneously conclude that moderator variables, instead of artifacts, explain any variation. We describe how online data repositories could improve the scientific process and the potential for using psychometric meta-analysis to synthesize results and assess replicability.

show abstract

How accurately do teachers’ judge students? Re-analysis of Hoge and Coladarci (1989) meta-analysis

Cited by 16 publications

References 58 publications

Temperament and Academic Achievement in Children: A Meta-Analysis

Temperament and Academic Achievement in Children: A Meta-Analysis

Don't Just Judge the Spelling! The Influence of Spelling on Assessing Second-Language Student Essays

Teachers’ judgment accuracy: A replication check by psychometric meta-analysis

Contact Info

Product

Resources

About