Characteristics of Useful Code Reviews: An Empirical Study at Microsoft

Bosu, Amiangshu; Greiler, Michaela; Bird, Christian

doi:10.1109/msr.2015.21

Cited by 129 publications

(155 citation statements)

References 21 publications

Supporting

Mentioning

146

Contrasting

Unclassified

Order By: Relevance

“…Code review characteristics such as review size, component, priority, organization, reviewer characteristics, and author experience significantly influence both review completion time and outcome [5]. Moreover, a reviewer's prior experience in changing or reviewing the artifact and the reviewer's project experience increases the likelihood that s/he will provide useful feedback [14]. While these studies focused on technical human factors and characteristics of the code changes, no studies have focused on the non-technical human factors (i.e., author's reputation, and relationship between an author and a reviewer).…”

Section: Overview Of Contemporary Code Review Researchmentioning

confidence: 99%

“…For example, reviewers may have different opinions on the effects of coding style on quality [14]. The next research question seeks to better understand these factors: RQ4: Which code characteristics are indicative of low quality code?…”

Section: Code Review Processmentioning

confidence: 99%

“…(36) OpenStack (32) CyanogenMod (28) Bing (54) Windows (48) Office (43) TYPO3 (20) Android (19) MediaWiki (17) Azure (24) Visual Studio (22) XBox (13) oVirt (17) Linux kernel (16) Chromium OS (14) Ad Center (11) Exchange (10) Dynamic AX (12) Eclipse (9) Gromacs (9) ITK (9) Windows Phone (10) IE (6) SQL Server (6) LibreOffice (8) OpenAFS (6) Scilab (5) Dynamic CRM (4) Sharepoint (4) Skype (3) VTK (5) Gerrit (4) AOKP (2) Halo (3) TFS (3) HPC (3) Couchbase (2) Debian (2) Others (24) Autopilot (3) Lync (3) Others (131) …”

Section: Respondent Demographicsmentioning

confidence: 99%

“…Apache [47], Chromium 1 , Mozilla 2 , Qt 3 , and Android 4 , now require peer-review prior to merging new code into the main project codebase, there are a large number of developers regularly participating in code reviews. From the industrial perspective, Google [56] and Facebook [33] have adopted mandatory code reviews and approximately 50,000 Microsoft developers actively practice code reviews [14]. Our recent survey of OSS developers found that they spend approximately six hours per week in code review [11].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Process Aspects and Social Dynamics of Contemporary Code Review: Insights from Open Source Development and Industrial Practice at Microsoft

Bosu

Carver

Bird

et al. 2017

IIEEE Trans. Software Eng.

Self Cite

130

125

View full text Add to dashboard Cite

Many open source and commercial developers practice contemporary code review, a lightweight, informal, tool-based code review process. To better understand this process and its benefits, we gathered information about code review practices via surveys of open source software developers and developers from Microsoft. The results of our analysis suggest that developers spend approximately 10-15% of their time in code reviews, with the amount of effort increasing with experience. Developers consider code review important, stating that in addition to finding defects, code reviews offer other benefits, including knowledge sharing, community building, and maintaining code quality. The quality of the code submitted for review helps reviewers form impressions about their teammates, which can influence future collaborations. We found a large amount of similarity between the Microsoft and OSS respondents. One interesting difference is that while OSS respondents view code review as an important method of impression formation, Microsoft respondents found knowledge dissemination to be more important. Finally, we found little difference between distributed and co-located Microsoft teams. Our findings identify the following key areas that warrant focused research: 1) exploring the non-technical benefits of code reviews, 2) helping developers in articulating review comments, and 3) assisting reviewers' program comprehension during code reviews.

show abstract

Section: Overview Of Contemporary Code Review Researchmentioning

confidence: 99%

Section: Code Review Processmentioning

confidence: 99%

Section: Respondent Demographicsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Process Aspects and Social Dynamics of Contemporary Code Review: Insights from Open Source Development and Industrial Practice at Microsoft

Bosu

Carver

Bird

et al. 2017

IIEEE Trans. Software Eng.

Self Cite

130

125

View full text Add to dashboard Cite

show abstract

“…Bosu et al [21] studied the characteristics of useful reviews given by reviewers and developed a classification model that can distinguish whether certain reviewer feedback would be helpful to the review process.…”

Section: Helping Reviewersmentioning

confidence: 99%

Will it pass? Predicting the outcome of a source code review

2018

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

It has been observed that allowing source code changes to be made only after source code reviews has a positive impact on the quality and lifetime of the resulting software. In some cases, code review processes take quite a long time and this negatively affects software development costs and employee job satisfaction. Establishing mechanisms that predict what kind of feedback reviewers will provide and what revisions they will ask for can reduce the number of times this problem occurs. Thanks to such mechanisms, developers can improve the maturity of their code change requests before the review process starts. In this way, when they start the review, the process may advance quickly and more smoothly. In this study, as a first step towards this goal, we developed a mechanism to identify whether a change proposal would require any revisions or not for approval.

show abstract

Automated evaluation of comments to aid software maintenance

Majumdar

Bansal

Das

et al. 2022

J Software Evolu Process

View full text Add to dashboard Cite

Approaches to evaluate comments based on whether they increase code comprehensibility for software maintenance tasks are important, but largely missing. We propose Comment Probe for automated classification and quality evaluation of code comments of C codebases based on how they can help to understand existing code. We conduct surveys and document developers' perceptions on the type of comments that prove useful to maintaining software in the form of comment categories. A total of 20,206 comments have been collected from open‐source Github projects and annotated with assistance from industry experts. We develop features to semantically analyze comments to locate concepts related to categories of usefulness. Additionally, features based on code and comment correlation are designed to infer whether the comment is also consistent and not superfluous. Using neural networks, comments are classified as useful, partially useful, and not useful with precision and recall scores of 86.27% and 86.42%, respectively. The proposed framework for comment quality evaluation incorporates industry practices and adds significant value to companies wanting to formulate better code commenting strategies. Furthermore, large codebases can be de‐cluttered by removing comments not helpful in maintaining code.

show abstract

Characteristics of Useful Code Reviews: An Empirical Study at Microsoft

Cited by 129 publications

References 21 publications

Process Aspects and Social Dynamics of Contemporary Code Review: Insights from Open Source Development and Industrial Practice at Microsoft

Process Aspects and Social Dynamics of Contemporary Code Review: Insights from Open Source Development and Industrial Practice at Microsoft

Will it pass? Predicting the outcome of a source code review

Automated evaluation of comments to aid software maintenance

Contact Info

Product

Resources

About