ComPAIR: A New Online Tool Using Adaptive Comparative Judgement to Support Learning with Peer Feedback

Abstract

Peer feedback is a useful strategy in teaching and learning, but its effectiveness particularly in introductory courses can be limited by the relative newness of students to both the body of knowledge upon which they are being asked to provide feedback and the skill set involved in providing good feedback. This paper applies a novel approach to facilitating novice feedback: making use of students' inherent ability to compare. The ComPAIR application discussed in this article scaffolds peer feedback through comparisons, asking students to choose the "better" of two answers in a series of pairings offered in an engaging online context. In contrast to other peer-feedback approaches that seek to train novices to be able to provide expert feedback (such as calibrated peer review) or to crowdsource grading, ComPAIR focuses upon the benefits to be gained from the critical process of comparison and ranking. The tool design is based on the longstanding psychological principle of comparative judgement, by which novices who may not yet have the compass to assess others' work confidently can still rank content as "better" with accuracy. Data from 168 students in pilot studies in English, Physics and Math courses at the University of British Columbia are reviewed. Though the use of ComPAIR required little classroom time, students perceived this approach to increase their facility with course content, their ability assess their own work, and their capacity to provide feedback on the work of others in a collaborative learning environment.

A Two‐Stage Method for Classroom Assessments of Essay Writing

“…Perhaps the most focus to date has been on judging and scaling students' writing performances (Heldsinger & Humphry, 2010;Steedle & Ferrara, 2016;van Daal, Lesterhuis, Coertjens, Donche, & De Maeyer, 2016) and students' problem solving in mathematics (Bisson, Gilmore, Inglis, & Jones, 2016;Jones & Inglis, 2015;Jones, Swan, & Pollitt, 2015). Other areas of focus have included peer assessment and feedback (Potter et al, 2017;Seery, Canty, & Phelan, 2012), the assessment of creative performances (Tarricone & Newhouse, 2016), and the assessment of oral narrative performances (Humphry, Heldsinger, & Dawkins, 2017). There has also been some research into the application of pairwise comparisons in setting grade boundaries (Benton & Elliot, 2016).…”

Recommendations for Orchestration of Formative Assessment Sequences: A Data-Driven Approach

“…Student performance over a course of an academic programme can be significantly affected and positively influenced through a series of feedback processes handled by peers [26]. Hence, a richer formative assessment process implemented by ComPAIR [31] lets teachers ask open-ended question, while learners provide textual answers. Afterwards, learners engage in a peer review loop.…”

Data-informed Decision-making in TEFA Processes: An Empirical Study of a Process Derived from Peer-Instruction

“…Histograms are then immediately displayed as feedback in order to help teachers and learners engage in a debriefing phase. A richer formative assessment process implemented by ComPAIR [15] lets teachers ask openended question, while learners provide textual answers. Afterwards, learners engage in a peer review loop.…”