New Developer Metrics for Open Source Software Development Challenges: An Empirical Study of Project Recommendation Systems

Abstract

Software collaboration platforms where millions of developers from diverse locations can contribute to the common open source projects have recently become popular. On these platforms, various information is obtained from developer activities that can then be used as developer metrics to solve a variety of challenges. In this study, we proposed new developer metrics extracted from the issue, commit, and pull request activities of developers on GitHub. We created developer metrics from the individual activities and combined certain activities according to some common traits. To evaluate these metrics, we created an item-based project recommendation system. In order to validate this system, we calculated the similarity score using two methods and assessed top-n hit scores using two different approaches. The results for all scores with these methods indicated that the most successful metrics were binary_issue_related, issue_commented, binary_pr_related, and issue_opened. To verify our results, we compared our metrics with another metric generated from a very similar study and found that most of our metrics gave better scores that metric. In conclusion, the issue feature is more crucial for GitHub compared with other features. Moreover, commenting activity in projects can be equally as valuable as code contributions. The most of binary metrics that were generated, regardless of the number of activities, also showed remarkable results. In this context, we presented improvable and noteworthy developer metrics that can be used for a wide range of open-source software development challenges, such as user characterization, project recommendation, and code review assignment.

Recommending GitHub Projects by Leveraging Developers' Social Networks and Genetic Algorithm

“…The knowledge obtained from this information can be used to leverage the metrics of developers and projects. According to Seker et al [28], such metrics can then be used to solve various challenges in software engineering domain. Developers' activities in GitHub offer the insights of collaboration, learning and reputation management in the community [8].…”

“…When it comes to recommending projects in GitHub, different metrics will be performed based on different models or frameworks. Seker et al [28] have applied collaborative filtering using item similarity logic model to evaluate different metrics. They found that commenting on any feature was as important as code contributions.…”

“…This work leverages interest vector and expertise vector to support out framework in providing the best recommendation combination. Previous works [28,54,55] mainly utilize fork, watch or pull-request data to represent the developer's interests to recommend projects. On the other hand, as discussed early on, previous research [117,118] tend to use README information or primary language as the baseline of developer's expertise.…”