Grader Consistency in using Standards-based Rubrics

Abstract

and her Ph.D. in Food Process Engineering from the Department of Agricultural and Biological Engineering at Purdue University. She is a member of Purdue's Teaching Academy. Since 1999, she has been a faculty member within the First-Year Engineering Program, teaching and guiding the design of one of the required first-year engineering courses that engages students in open-ended problem solving and design. Her research focuses on the development, implementation, and assessment of modeling and design activities with authentic engineering contexts.

Gathering evidence on e-rubrics: Perspectives and many facet Rasch analysis of rating behavior

“…Inconsistencies in grading stem from several factors related to the problem being graded, the individual grader, the time of day, the grader's level of fatigue, and the grader's overall experience. Graders are also affected by their general values and beliefs about grading, such as values of non-achievement factors, like effort, and perceptions that grades function as rewards or punishments (Hicks & Diefes-Dux, 2017). Reddy and Andrade (2010) emphasize the striking difference between students' and instructors' perceptions of rubric use; while students referred to rubrics as a learning tool, instructors focused on the role of a rubric as a tool to assign grades quickly, objectively, and accurately.…”

Validation of an Instrument to Measure Student Engagement with a Standards-Based Grading System

“…At least one other implementation issue may have contributed to this result in addition to those shared above. In reaction to an evaluation of grading quality [31], a grader-training program was being implemented for the first time in Spring 2017 to improve the reliability of grading and the quality of written feedback. While there were significant improvements, particularly with regards to the quantity of written feedback on problem sets, grading quality still varied considerably across the over 70 undergraduate graders for the entire course.…”

Standards-Based Grading Derived Data to Monitor Grading and Student Learning

“…A number of things can undermine reliability including grading fatigue, rushing to complete grading, and inattention to or misunderstanding of the rubric items. Given well designed rubrics, high reliability requires training and practice opportunities for and subsequent monitoring of those doing the grading [14].…”

“…In this period, much attention was given to easing the logistical issues of collecting clean data from the SGB rubrics and building confidence with the data by working towards fairness, validity, fidelity, integrity, and reliability. Confidence was built by clarifying the learning objectives and making them transparent to students, improving the alignment between problem sets and learning objectives, implementing various rubric designs for problem sets [14] and exams [15], and understanding the nature and extent of grader reliability issues [14]. In addition, potential representations of the data were explored to prepare for bringing the data before the instructors on a more regular basis for decision-making.…”