Pull Request Decisions Explained: An Empirical Overview

Abstract

The pull-based development model is widely used in open source projects, leading to the emergence of trends in distributed software development. One aspect that has garnered significant attention concerning pull request decisions is the identification of explanatory factors. Objective: This study builds on a decade of research on pull request decisions and provides further insights. We empirically investigate how factors influence pull request decisions and the scenarios that change the influence of such factors. Method: We identify factors influencing pull request decisions on GitHub through a systematic literature review and infer them by mining archival data. We collect a total of 3,347,937 pull requests with 95 features from 11,230 diverse projects on GitHub. Using these data, we explore the relations among the factors and build mixed effects logistic regression models to empirically explain pull request decisions. Results: Our study shows that a small number of factors explain pull request decisions, with that concerning whether the integrator is the same as or different from the submitter being the most important factor. We also note that the influence of factors on pull request decisions change with a change in context; e.g., the area hotness of pull request is important only in the early stage of project development, however it becomes unimportant for pull request decisions as projects become mature.

Pull request latency explained: an empirical overview

“…We collect a dataset of 11,230 projects on GitHub with 63 factors and 3,347,937 pull requests relating to the latency of pull requests. We open-source the relevant dataset (Zhang et al 2021) and model-building scripts. 2 2.…”

“…The research of pull requests has been a hot topic, including pull request reviewer recommendation (Yu et al 2014), automatic description generation (Liu et al 2019), @mechanism empirical study , etc. The work involving the whole pull request lifecycle mainly includes two aspects, i.e., pull request decision and latency (Zhang et al 2021). Therefore, a large amount of related research focuses on the study of pull request latency.…”

“…Reopen state focuses on the evaluation and analysis of the impact of reopening (Jiang et al 2019). For the entire lifetime, there are studies of factors influencing pull request decisions (Tsay et al 2014;Zhang et al 2021) and latency (Yu et al 2015), etc.…”

“…The analysis of the association between factors and latency of the two snapshots can help us understand the impact of factors' changes in the review process, which can provide actionable suggestions to those involved in the pull request. We have previously studied the factors that influence the final merge decision of pull request throughout its lifetime (Zhang et al 2021). As a critical research area in the entire lifetime of pull requests, the study of pull request latency also covers all the states of pull requests.…”

“…Meanwhile, as part of the effort evaluation (Maddila et al 2019), pull request latency analysis and prediction can also help reviewers save review time, improve review efficiency, and optimize pull request review priority. Therefore we would like to build on our previous work (Zhang et al 2021) and explore the impact of various factors on pull request latency.…”