Does code quality affect pull request acceptance? An empirical study

Lenarduzzi, Valentina; Nikkola, Vili; Saarimäki, Nyyti; Taibi, Davide

doi:10.1016/j.jss.2020.110806

Cited by 31 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since then, many studies have continued investigating the role of different technical and social factors [34,36,45,50,69,77], as well as various personal and demographic factors [9,26,41,42,48,49,56] on the review latency or decision of PRs. Recently, Zhang et al [72] and Zhang et al [71] conducted a large-scale empirical study of how a large set of factors identified through a systematic literature review can explain the review latency and decision of PRs, respectively.…”

Section: Related Workmentioning

confidence: 99%

On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests–A Mixed-Methods Study of 10 Large Open-Source Projects

Khatoonabadi

Costa

Shihab

2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Pull-based development has enabled numerous volunteers to contribute to open-source projects with fewer barriers. Nevertheless, a considerable amount of pull requests (PRs) with valid contributions are abandoned by their contributors , wasting the effort and time put in by both the contributors and maintainers. To better understand the underlying dynamics of contributor-abandoned PRs, we conduct a mixed-methods study using both quantitative and qualitative methods. We curate a dataset consisting of 265,325 PRs including 4,450 abandoned ones from ten popular and mature GitHub projects and measure 16 features characterizing PRs, contributors, review processes, and projects. Using statistical and machine learning techniques, we find that complex PRs, novice contributors, and lengthy reviews have a higher probability of abandonment and the rate of PR abandonment fluctuates alongside the projects’ maturity or workload. To identify why contributors abandon their PRs, we also manually examine a random sample of 354 abandoned PRs. We observe that the most frequent abandonment reasons are related to the obstacles faced by contributors, followed by the hurdles imposed by maintainers during the review process. Finally, we survey the top core maintainers of the studied projects to understand their perspectives on dealing with PR abandonment and on our findings.

show abstract

Section: Related Workmentioning

confidence: 99%

On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests–A Mixed-Methods Study of 10 Large Open-Source Projects

Khatoonabadi

Costa

Shihab

2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Results show that technical debt are not correlated with the 28 software metrics. Considering another static analysis tool, a recent study (Lenarduzzi et al 2021) investigated if pull requests are accepted in open-source based on quality flaws identified by PMD. The study considered 28 Java open-source projects, analyzing the presence of 4.7M PMD rules in 36K pull requests.…”

Section: Machine Learning For Static Analysis Tools Detectionmentioning

confidence: 99%

On the adequacy of static analysis warnings with respect to code smell prediction

et al. 2022

Self Cite

View full text Add to dashboard Cite

Code smells are poor implementation choices that developers apply while evolving source code and that affect program maintainability. Multiple automated code smell detectors have been proposed: while most of them relied on heuristics applied over software metrics, a recent trend concerns the definition of machine learning techniques. However, machine learning-based code smell detectors still suffer from low accuracy: one of the causes is the lack of adequate features to feed machine learners. In this paper, we face this issue by investigating the role of static analysis warnings generated by three state-of-the-art tools to be used as features of machine learning models for the detection of seven code smell types. We conduct a three-step study in which we (1) verify the relation between static analysis warnings and code smells and the potential predictive power of these warnings; (2) build code smell prediction models exploiting and combining the most relevant features coming from the first analysis; (3) compare and combine the performance of the best code smell prediction model with the one achieved by a state of the art approach. The results reveal the low performance of the models exploiting static analysis warnings alone, while we observe significant improvements when combining the warnings with additional code metrics. Nonetheless, we still find that the best model does not perform better than a random model, hence leaving open the challenges related to the definition of ad-hoc features for code smell prediction.

show abstract

“…Regarding pull requests, data generated by pull-based workflows is already used by researchers to support decision making (e.g., [38], [39]) and to study pull request characteristics (e.g., [40]- [42]). For example, they find that the acceptance of pull requests is not directly influenced by the quality of the proposed code [40]. Furthermore, Gousios [42] found that squashed commits are hidden in VCS.…”

Section: Related Workmentioning

confidence: 99%

PR-SZZ: How pull requests can support the tracing of defects in software repositories

Peter

Pretschner

2022

2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

View full text Add to dashboard Cite

The SZZ algorithm represents a standard way to identify bug fixing commits as well as inducing counterparts. It forms the basis for data sets used in numerous empirical studies. Since its creation, multiple extensions have been proposed to enhance its performance. For historical reasons, related work relies on commit messages to map bug tickets to possibly related code with no additional data used to trace inducing commits from these fixes. Therefore, we present an updated version of SZZ utilizing pull requests, which are widely adopted today. We evaluate our approach in comparison to existing SZZ variants by conducting experiments and analyzing the usage of pull requests, inner commits, and merge strategies. We base our results on 6 open-source projects with more than 50k commits and 35k pull requests. With respect to bug fixing commits, on average 18% of bug tickets can be additionally mapped to a fixing commit, resulting in an overall F-score of 0.75, an improvement of 40 percentage points. By selecting an inducing commit, we manage to reduce the false-positives and increase precision by on average 16 percentage points in comparison to existing approaches.

show abstract

Does code quality affect pull request acceptance? An empirical study

Cited by 31 publications

References 30 publications

On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests–A Mixed-Methods Study of 10 Large Open-Source Projects

On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests–A Mixed-Methods Study of 10 Large Open-Source Projects

On the adequacy of static analysis warnings with respect to code smell prediction

PR-SZZ: How pull requests can support the tracing of defects in software repositories

Contact Info

Product

Resources

About