On the prediction of the evolution of libre software projects

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

Dara³

et al. 2015

227

198

Abstract-Many defect prediction techniques are proposed to improve software reliability. Change classification predicts defects at the change level, where a change is the modifications to one file in a commit. In this paper, we conduct the first study of applying change classification in practice.We identify two issues in the prediction process, both of which contribute to the low prediction performance. First, the data are imbalanced-there are much fewer buggy changes than clean changes. Second, the commonly used cross-validation approach is inappropriate for evaluating the performance of change classification. To address these challenges, we apply and adapt online change classification, resampling, and updatable classification techniques to improve the classification performance.We perform the improved change classification techniques on one proprietary and six open source projects. Our results show that these techniques improve the precision of change classification by 12.2-89.5% or 6.4-34.8 percentage points (pp.) on the seven projects. In addition, we integrate change classification in the development process of the proprietary project. We have learned the following lessons: 1) new solutions are needed to convince developers to use and believe prediction results, and prediction results need to be actionable, 2) new and improved classification algorithms are needed to explain the prediction results, and insensible and unactionable explanations need to be filtered or refined, and 3) new techniques are needed to improve the relatively low precision.

Section: Related Workmentioning

confidence: 99%

“…Software defect prediction techniques leverage information such as code complexity, code authors and software development history to predict code areas that potentially contain defects [1]- [9]. Code areas that contain defects are also referred to as buggy code areas.…”

Section: Introductionmentioning

confidence: 99%

Online Defect Prediction for Imbalanced Data

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

Dara³

et al. 2015

227

198

“…Academia and industry expend much effort to predict software defects [1], [2], [3], [4], [5], [6], [7], [8], [9], [10]. These prior studies have made significant advances in defect prediction using features such as code complexity, code locations, the amount of in-house testing, historical data, and socio-technical networks.…”

Section: Introductionmentioning

confidence: 99%

Personalized defect prediction

Jiang

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Kim

2013

211

164

Abstract-Many defect prediction techniques have been proposed. While they often take the author of the code into consideration, none of these techniques build a separate prediction model for each developer. Different developers have different coding styles, commit frequencies, and experience levels, causing different defect patterns. When the defects of different developers are combined, such differences are obscured, hurting prediction performance.This paper proposes personalized defect prediction-building a separate prediction model for each developer to predict software defects. As a proof of concept, we apply our personalized defect prediction to classify defects at the file change level. We evaluate our personalized change classification technique on six large software projects written in C and Java-the Linux kernel, PostgreSQL, Xorg, Eclipse, Lucene and Jackrabbit. Our personalized approach can discover up to 155 more bugs than the traditional change classification (210 versus 55) if developers inspect the top 20% lines of code that are predicted buggy. In addition, our approach improves the F1-score by 0.01-0.06 compared to the traditional change classification.

“…affects the quality of that commit -are commits submitted after midnight buggier than other commits? Such correlations may be useful for predicting what commits are more likely to be buggy so that we can budget more testing effort on these commits, following prior studies [3,4,6,8,12,13,15,17,23,24], which predict buggy locations based on code complexity, code locations, the amount of in-house testing, historical data, socio-technical networks, etc. A second interesting question is whether more experienced developers are more or less likely to write buggy commits.…”

Section: Introductionmentioning

confidence: 99%

Do time of day and developer experience affect commit bugginess?

Eyolfson

Proceedings of the 8th Working Conference on Mining Software Repositories

Lam

2011

120

Modern software is often developed over many years with hundreds of thousands of commits. Commit metadata is a rich source of social characteristics, including the commit's time of day and the experience and commit frequency of its author. The "bugginess" of a commit is also a critical property of that commit. In this paper, we investigate the correlation between a commit's social characteristics and its "bugginess"; such results can be very useful for software developers and software engineering researchers. For instance, developers or code reviewers might be well-advised to thoroughly verify commits that are more likely to be buggy.In this paper, we study the correlation between a commit's bugginess and the time of day of the commit, the day of week of the commit, and the experience and commit frequency of the commit authors. We survey two widely-used open source projects: the Linux kernel and PostgreSQL.Our main findings include: (1) commits submitted between midnight and 4 AM (referred to as late-night commits) are significantly buggier and commits between 7 AM and noon are less buggy, implying that developers may want to double-check their own latenight commits; (2) daily-committing developers produce less-buggy commits, indicating that we may want to promote the practice of daily-committing developers reviewing other developers' commits; and (3) the bugginess of commits versus day-of-week varies for different software projects.