A Review of Peer Code Review in Higher Education

Abstract

Peer review is the standard process within academia for maintaining publication quality, but it is also widely employed in other settings, such as education and industry, for improving work quality and for generating actionable feedback to content authors. For example, in the software industry peer review of program source code—or peer code review—is a key technique for detecting bugs and maintaining coding standards. In a programming education context, although peer code review offers potential benefits to both code reviewers and code authors, individuals are typically less experienced, which presents a number of challenges. Some of these challenges are similar to those reported in the educational literature on peer review in other academic disciplines, but reviewing code presents unique difficulties. Better understanding these challenges and the conditions under which code review can be taught and implemented successfully in computer science courses is of value to the computing education community. In this work, we conduct a systematic review of the literature on peer code review in higher education to examine instructor motivations for conducting peer code review activities, how such activities have been implemented in practice, and the primary benefits and difficulties that have been reported. We initially identified 187 potential studies and analyzed 51 empirical studies pertinent to our goals. We report the most commonly cited benefits (e.g., the development of programming-related skills) and barriers (e.g., low student engagement), and we identify a wide variety of tools that have been used to facilitate the peer code review process. While we argue that more empirical work is needed to validate currently reported results related to learning outcomes, there is also a clear need to address the challenges around student motivation, which we believe could be an important avenue for future research.

The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming

“…The extensive literature on peer review, including the review of code [17,21] makes it clear that there are many benefits that arise from looking at a variety of solutions to a given problem. These benefits are present even when the code is flawed (i.e., there are benefits from looking at poor solutions as well as good solutions).…”

Matching Authors and Reviewers in Peer Assessment Based on Authors’ Profiles

“…The most common barriers are low engagement (Adachi et al, 2018) and low review quality (J. . Some students' justification for not engaging in peer assessment is that they do not consider it to be a helpful learning experience (Indriasari et al, 2020). Further, the reviews produced by some students were not consistently trustworthy; there were, for example, instances of inaccurate assessments or low-quality feedback (Alkhalifa & Devlin, 2021).…”

“…However, many scholars have highlighted barriers to the use of peer assessment (Indriasari et al, 2020;Zlabkova et al, 2021). One of these barriers is the low learning engagement of students.…”

Using Peer Review for Student Performance Enhancement: Experiences in a Multidisciplinary Higher Education Setting

“…Amendola and Miceli [12] propose a peer review methodology completely conducted through online technologies (in particular using the Moodle e-learning platform) showing the benefits such as lack of space and time in the standard lesson environment applying this methodology. In Indriasari et al [20], a survey of peer review of source code in higher education showing how such activities have been implemented in practice, examining instructor motivations and the primary benefits and difficulties of this practice have been reported. Authors also identify a wide variety of tools to facilitate the peer code review process.…”