What Types of Defects Are Really Discovered in Code Reviews?

Mäntylä, Martti; Lassenius, Casper

doi:10.1109/tse.2008.71

Cited by 161 publications

(136 citation statements)

References 55 publications

Supporting

Mentioning

124

Contrasting

Order By: Relevance

“…Similar to Mäntylä's results [32], the dominant change categories are code comments (20%) and identifiers (10%). However, our study shows that 7-35% of review comments are discarded, and that a substantial 10-22% of the total changes are not triggered by review suggestions; this is not considered in prior studies.…”

Section: Introductionsupporting

confidence: 80%

“…The standard was the basis for two classifications by IBM and HP [13]. Wagner et al [52] and Duraes and Madeira [16] topology on this empirically validated classification scheme [32]. They analysed code review comments and extrapolated the types of defects found, reporting a 75:25 ratio between maintainability and functional defects.…”

Section: Research Methods -Rq1 and Rq2mentioning

confidence: 99%

“…Most research comparing inspection with testing and pair programming considers only functional defects [4,28,30,44]. Given a reported 75% of non-functional defects in MCR [32], it stands to question whether the results from prior studies apply to modern review.…”

Section: Related Workmentioning

confidence: 99%

“…While a variety of studies focused on inspections, research on MCR is in the early stages. Only two studies analysed the outcome of MCR: (1) Bacchelli and Bird investigated motivations and outcomes of MCR at Microsoft [5], and (2) Mäntylä and Lassenius classified the types of defects found in review on university and three industrial software systems [32].…”

Section: Introductionmentioning

confidence: 99%

“…While we tried to rate modifications as fine-granular and precise as possible, we preferred to rate bigger changes with a recognizable functional modification as one larger functional change. If we could rate a change as either evolvable or functional, we preferred the latter, in accordance with Mäntylä and Lassenius [32]: We consider functional implications more severe, and most functional changes inevitably have maintainability implications.…”

mentioning

confidence: 96%

See 4 more Smart Citations

Modern code reviews in open-source projects: which problems do they fix?

Beller

Bacchelli

Zaidman

et al. 2014

Proceedings of the 11th Working Conference on Mining Software Repositories

201

150

View full text Add to dashboard Cite

Code review is the manual assessment of source code by humans, mainly intended to identify defects and quality problems. Modern Code Review (MCR), a lightweight variant of the code inspections investigated since the 1970s, prevails today both in industry and open-source software (OSS) systems. The objective of this paper is to increase our understanding of the practical benefits that the MCR process produces on reviewed source code. To that end, we empirically explore the problems fixed through MCR in OSS systems. We manually classified over 1,400 changes taking place in reviewed code from two OSS projects into a validated categorization scheme. Surprisingly, results show that the types of changes due to the MCR process in OSS are strikingly similar to those in the industry and academic systems from literature, featuring the similar 75:25 ratio of maintainability-related to functional problems. We also reveal that 7-35% of review comments are discarded and that 10-22% of the changes are not triggered by an explicit review comment. Patterns emerged in the review data; we investigated them revealing the technical factors that influence the number of changes due to the MCR process. We found that bug-fixing tasks lead to fewer changes and tasks with more altered files and a higher code churn have more changes. Contrary to intuition, the person of the reviewer had no impact on the number of changes.

show abstract

Section: Introductionsupporting

confidence: 80%

Section: Research Methods -Rq1 and Rq2mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 96%

See 3 more Smart Citations

Modern code reviews in open-source projects: which problems do they fix?

Beller

Bacchelli

Zaidman

et al. 2014

Proceedings of the 11th Working Conference on Mining Software Repositories

201

150

View full text Add to dashboard Cite

show abstract

Using peer code review to improve computational thinking in a blended learning environment: A randomized control trial

Lin

et al. 2021

Comp Applic In Engineering

View full text Add to dashboard Cite

Although the importance of computational thinking (CT) has been widely recognized, we still poorly understand how to effectively teach this skill. The aim of this study is to investigate whether peer code review (PCR) could affect CT, learning engagement, and learning satisfaction in a blended learning environment. To achieve this, we carried out a randomized control trial study involving 84 first-year undergraduates (second semester) from different majors. There was no difference in CT before this experiment and after the experiment, general CT, specific CT, learning engagement, and satisfaction were measured. An independent sample t test showed that students learning with code peer review showed better performance on computational concepts, computational perspectives, learning engagement, and learning satisfaction (p < .001). Bayesian t test further showed substantial or strong evidence for the improvement of computational concepts, learning engagement, learning satisfaction, and computational perspectives. Moreover, the absolute values of Cohen's effect sizes of computational concept, computational perspectives, emotional engagement, behavior engagement, learning engagement, learning satisfaction were all above 0.4, which is viewed as educationally meaningful. Although there was no difference in the general CT, we still saw evidence of improvement over the course of the experiment. Overall, our findings suggest instructors to use PCR reasonably to further improve CT, learning engagement, and satisfaction in programming courses.

show abstract

Integrating inspection and test processes based on context‐specific assumptions

Elberzhager

Münch

Rombach

et al. 2012

J Software Evolu Process

View full text Add to dashboard Cite

Inspections and testing are two of the most commonly performed software quality assurance processes today. Typically, these processes are applied in isolation, which, however, fails to exploit the benefits of systematically combining and integrating them. In consequence, tests are not focused based on early defect detection data. Expected benefits of such process integration include higher defect detection rates or reduced quality assurance effort. Moreover, when conducting testing without any prior information regarding the system's quality, it is often unclear how to focus testing. A systematic integration of inspection and testing processes requires context-specific knowledge about the relationships between inspections and testing. This knowledge is typically not available and needs to be empirically identified and validated. Often, context-specific assumptions can be seen as a starting point for generating such knowledge. Based on the In 2 Test approach, which uses inspection data to focus testing, we present in this article how knowledge about the relationship between inspections and testing can be gained, documented, and evolved in an analytical or empirical manner. In addition, this article gives an overview of related work and highlights future research directions.

show abstract

What Types of Defects Are Really Discovered in Code Reviews?

Cited by 161 publications

References 55 publications

Modern code reviews in open-source projects: which problems do they fix?

Modern code reviews in open-source projects: which problems do they fix?

Using peer code review to improve computational thinking in a blended learning environment: A randomized control trial

Integrating inspection and test processes based on context‐specific assumptions

Contact Info

Product

Resources

About