Feasibility of using comparative judgement and student judges to assess writing performance of English language learners

Abstract

This study aims to identify how feasible it is to use comparative judgement (CJ) and student judges to assess the writing performance of English language learners. For this purpose, 35 paragraphs written by the students who were enrolled in a freshman Academic Writing course at a semi-private university located in the Turkish Republic of Northern Cyprus were selected and uploaded to http://www.nomoremarking.com website. Ten instructors of the Academic Writing course and 112 students taking the course volunteered to participate in the study. Then, the students were taken into 5 groups according to their writing performance level. In total, around 350 comparisons were done by each group. The results suggested that it could be feasible to use CJ to assess short writing performance like paragraphs if the instructors were experienced and trained. Moreover, instructors liked CJ and described it as a more practical, easier, fairer, faster, more enjoyable way of marking student papers. The students also liked CJ and it was also found that students who were high achievers in paragraph writing might be used to mark student papers through comparative judgement as long as they were trained.

Comparative Judgement

“…To date, Comparative Judgement has proliferated across many writing assessment contexts, including large-scale primary and secondary-level assessment (e.g., McGrane et al, 2018;Pollitt, 2012;Wheadon, Barmby, Christodoulou, & Henderson, 2020), primary-level classroom-based assessment (e.g., Heldsinger & Humphry, 2010, 2013Humphry & Heldsinger, 2019b), higher education contexts (e.g., Lesterhuis, Bouwer, van Daal, Donche, & De Maeyer, 2022;van Daal, Lesterhuis, Donche, De Maeyer, & Coertjens, 2019), English as an additional language writing contexts (e.g., Şahin, 2021;Sims, Cox, Eckstein, Hartshorn, Wilcox, & Hart, 2020), as well as for the longitudinal equating of a standardised writing assessment (Humphry & McGrane, 2015).…”