Teacher rating versus measured academic achievement: Implications for paediatric research

Abstract

AimTo determine whether teachers' reports of student academic performance can suffice for research purposes by comparing it with a curriculum‐based standardised test.MethodsIn this longitudinal cohort study of children born at risk of neonatal hypoglycaemia, teachers' global assessment of student performance was compared with assessment tools for teaching and learning (asTTle) at 9–10 years. Performance on asTTLe was rated as being below, at or above that expected on the national curriculum for year and term of schooling. Teachers similarly rated the child's performance against the national curriculum.ResultsOf 125 children assessed, 104 had paired data for analysis. On asTTLe, 28% were rated below, 55% at and 17% above the expected curriculum level in reading and 24, 54 and 22%, respectively, in mathematics. Equivalent teacher ratings were 23, 58 and 19% in reading and 36, 55 and 9% in mathematics, respectively. However, there was limited agreement between asTTle and teacher rating of achievement in reading (κ = 0.23 (95% confidence interval 0.07–0.40)) and no significant agreement in mathematics (κ = 0.07 (95% confidence interval −0.09–0.22)). Only 45% of children performing below the curriculum level in reading and 52% in mathematics were correctly identified by teachers.ConclusionIn cohort studies, teacher ratings cannot substitute standardised educational testing.

Association of Neonatal Hypoglycemia With Academic Performance in Mid-Childhood

“…The e-asTTle assesses students’ performance in Curriculum Levels 2 to 6 (nominally, school years [grades] 4 to 11; approximately ages 8 to 16 years), with normative data for each year and term of schooling (4 terms per year) for years 4 to 12 inclusive . The psychometric properties and scoring methods have been detailed elsewhere . In brief, a student’s achievement was rated as being “at” curriculum level if their achievement was within the range of the normative data for students in the 3 terms either side of their current school year and term, “above” if 4 or more terms above, “below” if 4 to 7 terms below, or “well below” if 8 or more terms below their school year and term.…”

“…18 The psychometric properties and scoring methods have been detailed elsewhere. 19 In brief, a student's achievement was rated as being "at" curriculum level if their achievement was within the range of the normative data for students in the 3 terms either side of their current school year and term, "above" if 4 or more terms above, "below" if 4 to 7 terms below, or "well below" if 8 or more terms below their school year and term. Additionally, children's teachers were asked to rate their academic achievement against the expected curriculum level for school year and term (well below, below, at, or above) and in relation to peers (much worse, worse, about the same, better, or much better) using a structured teacher questionnaire.…”

“…Additionally, children's teachers were asked to rate their academic achievement against the expected curriculum level for school year and term (well below, below, at, or above) and in relation to peers (much worse, worse, about the same, better, or much better) using a structured teacher questionnaire. 19 Cognitive elements of executive function were evaluated using items from the tablet-based Cambridge Neuropsychological Test Automated Battery (CANTAB) system focusing on attention and cognitive flexibility (Attention Switching Task); planning and manipulation of information (One-Touch Stockings of Cambridge, Spatial Working Memory); visual memory and learning (Paired Associate Learning); and inhibitory control (Stop Signal Task) (eTable 1 in Supplement 3). 20 At 9 to 10 years of age, children are expected to have largely mastered these skills, with performance approximating 70% to 80% of adult function.…”

Mid-Childhood Outcomes after Dextrose Gel Treatment of Neonatal Hypoglycaemia: Follow-Up of the Sugar Babies Randomized Trial

“…In brief, academic achievement was assessed using the Assessment Tools for Teaching and Learning (e-asTTle), a standardized curriculum-based online achievement test for Reading Comprehension/Pānui and Mathematics/Pāngarau [9, 10]. The child’s performance was rated as at, above, below, or well below the normative range at that curriculum level.…”

Responding to Assessment for Learning

“…If the judges had been free from bias, the degree of mismatch between judgement and score should have been small and constant across all groups. A number of explanations for this discrepancy exist, including variable teacher expectations by student demographic factors, students' variable familiarity and practice with standardised tests, mismatch of teaching to test content, lack of OTJ moderation among teachers, or even unconscious bias (Shah et al, 2020). 'Otunuku and Brown (2007) even suggested that teachers may be delivering a curriculum that disguises from Māori and Pasifika students that their performance is well below expectations.…”