Tracking concept drift of software projects using defect prediction quality

Software engineering process information extracted from version control systems and bug tracking databases are widely used in empirical software engineering. In prior work, we showed that these data are plagued by quality deficiencies, which vary in its characteristics across projects. In addition, we showed that those deficiencies in the form of bias do impact the results of studies in empirical software engineering. While these findings affect software engineering researchers the impact on practitioners has not yet been substantiated. In this paper we, therefore, explore (i) if the process data quality and characteristics have an influence on the bug fixing process and (ii) if the process quality as measured by the process data has an influence on the product (i.e., software) quality. Specifically, we analyze six Open Source as well as two Closed Source projects and show that process data quality and characteristics have an impact on the bug fixing process: the high rate of empty commit messages in Eclipse, for example, correlates with the bug report quality. We also show that the product quality --measured by number of bugs reported --is affected by process data quality measures. These findings have the potential to prompt practitioners to increase the quality of their software process and its associated data quality. Abstract-Software engineering process information extracted from version control systems and bug tracking databases are widely used in empirical software engineering. In prior work, we showed that these data are plagued by quality deficiencies, which vary in its characteristics across projects. In addition, we showed that those deficiencies in the form of bias do impact the results of studies in empirical software engineering. While these findings affect software engineering researchers the impact on practitioners has not yet been substantiated. In this paper we, therefore, explore (i) if the process data quality and characteristics have an influence on the bug fixing process and (ii) if the process quality as measured by the process data has an influence on the product (i.e., software) quality. Specifically, we analyze six Open Source as well as two Closed Source projects and show that process data quality and characteristics have an impact on the bug fixing process: the high rate of empty commit messages in Eclipse, for example, correlates with the bug report quality. We also show that the product quality -measured by number of bugs reported -is affected by process data quality measures. These findings have the potential to prompt practitioners to increase the quality of their software process and its associated data quality.

show abstract

“…Hence, we do not seem to have any timeshifting effects in these datasets. A finding we find surprising given our previous work [4], [30].…”

Section: B Influence Of Process Quality On Product Qualitycontrasting

confidence: 63%

When process data quality affects the number of bugs: Correlations in software engineering datasets

Bachmann

Bernstein

2010

2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Area under curve (AUC) is also a robust measure to assess and compare the performance of classifiers [26]. For example, using "accuracy" as a measure of prediction is problematic in heavily skewed distributions since it does not relate the prediction to the probability distribution of the classes, but AUC is insensitive to probability distributions [21]. In many situations, a proposed approach (e.g., a classifier) may be superior to the baselines for some input data but not for others.…”

Section: Comments From 5 Papers]mentioning

confidence: 99%

The MSR Cookbook: Mining a decade of research

Hemmati

Nadi

Baysal

et al. 2013

2013 10th Working Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

Abstract-The Mining Software Repositories (MSR) research community has grown significantly since the first MSR workshop was held in 2004. As the community continues to broaden its scope and deepens its expertise, it is worthwhile to reflect on the best practices that our community has developed over the past decade of research. We identify these best practices by surveying past MSR conferences and workshops. To that end, we review all 117 full papers published in the MSR proceedings between 2004 and 2012. We extract 268 comments from these papers, and categorize them using a grounded theory methodology. From this evaluation, four high-level themes were identified: data acquisition and preparation, synthesis, analysis, and sharing/replication. Within each theme we identify several common recommendations, and also examine how these recommendations have evolved over the past decade. In an effort to make this survey a living artifact, we also provide a public forum that contains the extracted recommendations in the hopes that the MSR community can engage in a continuing discussion on our evolving best practices.

show abstract

“…Ekanayake et. al revealed that useless phase exists in defect prediction using the notion of concept drift, which invalidates a learned prediction model [63]. As history data is a good predictor of future bugs in the stable phase, however, in unstable phase, it is not the case, resulting in reducing the effectiveness of future effort and resource allocation.…”

Section: Purpose Of Msr Analysismentioning

confidence: 99%