Connecting Judgment Process and Accuracy of Student Teachers: Differences in Observation and Student Engagement Cues to Assess Student Characteristics

Abstract

Teachers' ability to assess student cognitive and motivational-affective characteristics is a requirement to support individual students with adaptive teaching. However, teachers have difficulty in assessing the diversity among their students in terms of the intra-individual combinations of these characteristics in student profiles. Reasons for this challenge are assumed to lie in the behavioral and cognitive activities behind judgment processes. Particularly, the observation and utilization of diagnostic student cues, such as student engagement, might be an important factor. Hence, we investigated how student teachers with high and low judgment accuracy differ with regard to their eye movements as a behavioral and utilization of student cues as a cognitive activity. Forty-three participating student teachers observed a video vignette showing parts of a mathematics lesson to assess student characteristics of five target students, and reported which cues they used to form their judgment. Meanwhile, eye movements were tracked. Student teachers showed substantial diversity in their judgment accuracy. Those with a high judgment accuracy showed slight tendencies toward a more “experienced” pattern of eye movements with a higher number of fixations and shorter average fixation duration. Although all participants favored diagnostic student cues for their assessments, an epistemic network analysis indicated that student teachers with a high judgment accuracy utilized combinations of diagnostic student cues that clearly pointed to specific student profiles. Those with a low judgment accuracy had difficulty using distinct combinations of diagnostic cues. Findings highlight the power of behavioral and cognitive activities in judgment processes for explaining teacher performance of judgment accuracy.

Identifying Expert and Novice Visual Scanpath Patterns and Their Relationship to Assessing Learning-Relevant Student Characteristics

“…However, it remains relatively unclear which specific cognitive processes and behavioral activities underlie teachers' assessments (Leuders et al, 2018;Loibl et al, 2020;Schnitzler et al, 2020). Established models from social and general psychological (Brunswik, 1955;Fiske and Neuberg, 1990;Chaiken and Trope, 1999; Lens model; dual process theories) have helped to understand teachers' judgment processes.…”

A Systematic Literature Review of Empirical Research on Epistemic Network Analysis in Education

“…Finally, 10 articles (13.2%) were coded based on diverse coding schemes, for example, eye movements [95], [96], diagnostic activities [97], [98], written assignments [25], [99], and coding of online postings [100], [101].…”

“…All 76 reviewed articles included comparisons that can be grouped into one of eight categories (Figure 5). In 23 articles (30.3 %), the comparison was across levels of participants' performance, for example, high vs. low performers [19], [23], [25], [50], [74], [76], [86], student competency level [69], [102], quality of ideas [46], [73], learning outcomes [80], [83] , positive vs. negative gain [82], proficiency level [68], accuracy level [96] and correct graph solving [95].…”

Exploring the relations between teachers’ cue-utilization, monitoring and regulation of students’ text learning

“…The accuracy of cue judgments may play an important role in the monitoring process. Surprisingly, the accuracy of cue judgments has hardly been investigated (see for an exception Schnitzler et al (2020) who mention that teachers had troubles assessing the quality of students' contributions in classroom conversations, which can also be seen as a cue) and is not addressed in models of monitoring and regulation typically used in educational research (e.g., Koriat's cue-utilization model;Koriat, 1997). However, this issue is addressed in Funder's Realistic Accuracy Model of personality judgment (Funder, 2012).…”