Modern code reviews in open-source projects: which problems do they fix?

Abstract

Code review is the manual assessment of source code by humans, mainly intended to identify defects and quality problems. Modern Code Review (MCR), a lightweight variant of the code inspections investigated since the 1970s, prevails today both in industry and open-source software (OSS) systems. The objective of this paper is to increase our understanding of the practical benefits that the MCR process produces on reviewed source code. To that end, we empirically explore the problems fixed through MCR in OSS systems. We manually classified over 1,400 changes taking place in reviewed code from two OSS projects into a validated categorization scheme. Surprisingly, results show that the types of changes due to the MCR process in OSS are strikingly similar to those in the industry and academic systems from literature, featuring the similar 75:25 ratio of maintainability-related to functional problems. We also reveal that 7-35% of review comments are discarded and that 10-22% of the changes are not triggered by an explicit review comment. Patterns emerged in the review data; we investigated them revealing the technical factors that influence the number of changes due to the MCR process. We found that bug-fixing tasks lead to fewer changes and tasks with more altered files and a higher code churn have more changes. Contrary to intuition, the person of the reviewer had no impact on the number of changes.

The last line effect explained

“…Our interviews with developers indicated that creating short-lived micro-clones might be common in software development, but that the developers usually catch them early, or at least during their own or someone else's review of the code (Beller et al 2014). The cognitive error in the remaining micro-clones we observed in this study is thus not only a production error, but also a proofreading error (Healy 1980): During revision of the code, the engineer fails to notice the error in the last and other lines.…”

Understanding Review Expertise of Developers: A Reviewer Recommendation Approach Based on Latent Dirichlet Allocation

“…The role of the reviewer assigner is important in order to efficiently perform the code review process [7]. If a reviewer assigner assigns an unsuitable reviewer who does not deeply understand the given source code change, the author of the source code change may obtain poor review outcomes.…”

“…Inefficient logic and latent bugs in the source code significantly increase the maintenance costs of a software project. Although code reviews are time consuming, it is beneficial to detect the defects in the source code at an early software development stage [1,[3][4][5][6][7][8][9].…”

“…The reviewer assignment is a major challenge in performing the code review process rapidly and successfully [2,[5][6][7]10,12,13]. Thongtanunam et al investigated the impact of the reviewer assignment problem by conducting an exploratory study in open source projects.…”

ReDA: A Web-Based Visualization Tool for Analyzing Modern Code Review Dataset

“…Extracting knowledge from these datasets has produced promising research with the goal of improving the software quality and software development process. Recently, many studies have used code review datasets to understand and improve both review effort [2]- [5] and review quality [6], [7]. However, a raw code review dataset is generally imperfect since the data collection process in each support tool 1 https://code.google.com/p/gerrit/ 2 https://www.reviewboard.org/ 3 https://code.google.com/p/rietveld/ varies in methodology, accuracy, and degree of automation [8].…”