Structuring the peer assessment process: a multilevel approach for the impact on product improvement and peer feedback quality

Abstract

In order to optimize students' peer feedback processes, this study investigates how an instructional intervention in the peer assessment process can have a beneficial effect on students' performance in a wiki environment in first‐year higher education. The main aim was to study the effect of integrating a peer feedback template with a varying structuring degree. The present study involved three conditions: a no structure, a basic structure and an elaborate structure condition. Due to a clear hierarchical structure, in which over time (level 1), 168 students (level 2) are nested within 37 groups (level 3), multilevel analysis was performed to examine the effect of time, student and group level influences on students' peer feedback quality and product scores. The results revealed that both peer feedback quality and product scores increase significantly for all conditions over time, after multiple practice occasions. In addition, after several practice occasions, significant differences were found between the conditions in both peer feedback (elaborate higher than no structure) and product scores (elaborate and basic higher than no structure). Building on this, limitations, directions for future research and practical implications are presented.

Structuring peer assessment: Comparing the impact of the degree of structure on peer feedback content

“…More specifically, this study builds further on an earlier study (Gielen & De Wever, 2015) in which the added value of different peer feedback forms, with a varying degree of structuring, was studied in a wiki environment in higher education, with respect to product scores. Also, a general peer feedback quality index (Prins, Sluijsmans, & Kirschner, 2006) was used to assess the content quality of peer feedback messages.…”

“…Previous research claims that role assignment is an essential structuring tool to increase knowledge construction in asynchronous discussion groups (De Wever, Van Keer, Schellens, & Valcke, 2010). It is within this frame that the main aim of the present study can be situated (see also Gielen & De Wever, 2015): "How can we increase the peer feedback quality by structuring the PA process?" With respect to this question, suggestions have been made in the literature.…”

“…In earlier studies, the developed Feedback Quality Index (Prins, Sluijsmans, & Kirschner, 2006) was incorporated to measure the quality of feedback, with the help of a scoring rubric (e.g. Gielen & De Wever, 2012;Gielen & De Wever (2015). In the present study, however, the aim was to take a closer look at the peer feedback content and more specifically at the peer feedback style, type, and focus of messages that peers provide to each other during writing assignments in a wiki-based CSCL environment.…”

“…With respect to this question, suggestions have been made in the literature. As previous research has illustrated how structuring can be an effective strategy to improve both students' PFB quality and performance in function of enhancing the actual PA process (Gielen & De Wever, 2015), the instructor could structure the PA process by providing more detailed instructions on expected performance (Kollar, Fischer, & Slotta, 2007), e.g. by providing guiding questions to support the assessor while providing peer feedback (Gielen & De Wever, 2012).…”

“…In this respect, research has shown that involving learners in online PA activities appeared for example to have an advantageous effect on students' writing performance (eg. Gielen & De Wever, 2015). Yet, research on PA in higher education is up to now "very variable in type and quality, scattered and fragmentary in nature" (Topping, 1998, p. 267; see also Evans, 2013, who still presents the same conclusion).…”