Factors influencing code review processes in industry

Baum, Tobias; Liskin, Olga; Niklas, Kai; Schneider, Kurt

doi:10.1145/2950290.2950323

Cited by 60 publications

(54 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Software code reviews play nowadays a major role in improving the quality of software in terms of defects identified [5], [6]. However, code reviews are a time-consuming process, with significant amount of human effort involved [7].…”

Section: Introductionmentioning

confidence: 99%

“…However, code reviews are a time-consuming process, with significant amount of human effort involved [7]. Managers might have concerns that code reviews will slow the projects down [6]. Code reviews could also lead to negative moods in teams if team members fear public criticism caused by others reviewing their source code [8].…”

Section: Introductionmentioning

confidence: 99%

“…At a process level, source code review processes in companies nowadays are similar to the processes adopted in open source projects [6], [9], [10], with a lot of variation in the process steps [6], [11]. Furthermore, source code reviews can be expensive: knowledgeable understanding of large code changes by reviewers is a time-consuming process, and finding the most knowledgeable code reviewers for the source code parts to be reviewed can also be very labor-intensive for developers.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Large-Scale Study on Source Code Reviewer Recommendation

Lipcak

Rossi

2018

2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)

View full text Add to dashboard Cite

Context: Software code reviews are an important part of the development process, leading to better software quality and reduced overall costs. However, finding appropriate code reviewers is a complex and time-consuming task. Goals: In this paper, we propose a large-scale study to compare performance of two main source code reviewer recommendation algorithms (RevFinder and a Naïve Bayes-based approach) in identifying the best code reviewers for opened pull requests. Method: We mined data from Github and Gerrit repositories, building a large dataset of 51 projects, with more than 293K pull requests analyzed, 180K owners and 157K reviewers. Results: Based on the large analysis, we can state that i) no model can be generalized as best for all projects, ii) the usage of a different repository (Gerrit, GitHub) can have impact on the the recommendation results, iii) exploiting sub-projects information available in Gerrit can improve the recommendation results.• provision of a large dataset of 51 projects mined from Gerrit (14) and Github (37) with 293,337 total pull requests analyzed, considering 180,111 owners and 157,885 reviewers. The dataset is available on Figshare [24]. • comparison of the results in the two repositories (Gerrit, GitHub) using both RevFinder and Naïve Bayes-based approaches in the context of the 51-projects dataset.The article is structured as follows. In section II, we propose the background on several algorithms for code reviewer recommendation. In section III we have the experimental study design, with research questions, context, data analysis and replicability information. In section IV, we answer the research questions with discussions and threats to validity. Section V proposes the related works and section VI the conclusions.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Large-Scale Study on Source Code Reviewer Recommendation

Lipcak

Rossi

2018

2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)

View full text Add to dashboard Cite

show abstract

“…This sparks discussions in development teams on the preferable way to perform reviews. To better understand the variations in review processes in practice, we performed in‐depth interviews in 19 software companies, ranging from small start‐ups and standard software producers to large in‐house IT departments . In these interviews, the software development professionals named several reasons why they believe their method of choice is more efficient: Pre‐commit reviews find defects before they impede other developers. Pre‐commit reviews might extend the cycle time of user stories, increasing the “work in progress” and consequently increasing task switch overhead. Post‐commit reviews support the early and often integration of changes, better suiting the mindset behind continuous integration and possibly reducing the risk of conflicts. …”

Section: Introductionmentioning

confidence: 99%

“…To better understand the variations in review processes in practice, we performed in-depth interviews in 19 software companies, ranging from small start-ups and standard software producers to large in-house IT departments. 5,7 In these interviews, the software development professionals named several reasons why they believe their method of choice is more efficient:…”

Section: Introductionmentioning

confidence: 99%

Comparing pre‐commit reviews and post‐commit reviews using process simulation

Baum

Kortum

Schneider

et al. 2017

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

Code review in practice is often performed change‐based, ie, using the code changes belonging to a task to determine which code to review. Previous studies found that 2 variations of this process are used in industry: pre‐commit review and post‐commit review. The choice between these has implications not only for practitioners deciding on a code review process to use but also for the development of review tools and for experimentation with review processes. In some situations, a specific variant is clearly preferable due to the nature of the development process or team. In other situations, there are conflicting opinions. So we asked: Are there practically relevant performance differences between pre and post‐commit reviews? How are these differences influenced by contextual factors? To assess these questions, we designed a parametric discrete event simulation model of certain agile development processes. We validated this model with practitioner's feedback and in part also with empirical data from industry. Our analysis indicates that the best choice does depend on the context but also that there are many situations with no practically relevant difference between both choices. We identified the main influencing factors and underlying effects and condensed our findings into heuristic rules.

show abstract