The impact of context metrics on just-in-time defect prediction

Kondo, Masanari; Germán, Daniel M.; Mizuno, Osamu; Choi, Eun-Hye

doi:10.1007/s10664-019-09736-3

Cited by 45 publications

(27 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This observation suggests that it might be relatively uninformative to learn from later life cycle data. This was an interesting finding since, as mentioned in the introduction, it is common practice in defect prediction to perform "recent validation" where predictors are tested on the latest release after training from the prior one or two releases [10], [13], [14]. In terms of Figure 2, that strategy would train on red dots taken near the right-hand-side, then test on the most right-hand-side dot.…”

Section: A Github Resultsmentioning

confidence: 90%

See 1 more Smart Citation

Early Life Cycle Software Defect Prediction. Why? How?

Shrikanth

Majumder

2020

Preprint

View full text Add to dashboard Cite

Many methods in defect prediction are "datahungry"; i.e. (1) given a choice of using more data, or some smaller sample, researchers assume that more is better; (2) when data is missing, researchers take elaborate steps to transfer data from another project; and (3) given a choice of older data or some more recent sample, researchers usually ignore older data.Based on the analysis of hundreds of popular Github projects (with 1.2 million commits), we suggest that for defect prediction, there is limited value in such data-hungry approaches. Data for our sample of projects last for 84 months and contains 3,728 commits (median values). Across these projects, most of the defects occur very early in their life cycle. Hence, defect predictors learned from the first 150 commits and four months perform just as well as anything else.This means that, contrary to the "data-hungry" approach, (1) small samples of data from these projects are all that is needed for defect prediction; (2) transfer learning has limited value since it is needed only for the first 4 of 84 months (i.e. just 4% of the life cycle); (3) after the first few months, we need not continually update our defect prediction models.We hope these results inspire other researchers to adopt a 'simplicity-first" approach to their work. Certainly, there are domains that require a complex and data-hungry analysis. But before assuming complexity, it is prudent to check the raw data looking for "short cuts" that simplify the whole analysis.

show abstract

Section: A Github Resultsmentioning

confidence: 90%

“…"just in time", "software" AND " defect prediction" AND "sampling policy"). "Just in time (JIT)" defect prediction is a widely-used approach where the code seen in each commit is assessed for its defect prone-ness [13], [14], [38], [63].…”

Section: Sampling Policesmentioning

confidence: 99%

Early Life Cycle Software Defect Prediction. Why? How?

Shrikanth

Majumder

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Summary of answers to RQ2: 178, 188, 189, 191, 193, 194, 196, 198, 201, 203, 204, 207ś213, 215, 218, 222, 223, 226, 227, 230, 232, 237, 239, 242, 248, 249, 253, 257, 259, 264, 271, 274, 276, 278ś280, 280, 282, 284, 285, 289, 290, 292, 294ś297, 299, 300, 303, 305, 306, 306, 307, 312, 319, 326, 330, 331, 336, 337] Area Under the Curve (AUC) The area under the receiver operating characteristics curve. Independent of the cutof value [15, 19, 29, 34, 43, 44, 46, 53, 56, [19,29,31,56,57,123,154,210,211,237,303,329,330] False positive rate(FPR) the ratio between the number of negative events wrongly categorized as positive and the total number of actual negative events + [32,152,163,163,164,198,221,…”

Section: Evaluation Metrics For Predictive Models In Classification Tasksmentioning

confidence: 99%

“…Just-in-time (JIT) defect prediction: Compared with traditional defect predictions at class or ile level, Justin-Time (JIT) defect prediction is of more practical value for participants, which aims to identify defect-inducing changes. Many studies focused on JIT defect prediction by employing SZZ approach [61,110,111,123,252]. In order to identify bug-introducing changes, SZZ irst detects the bug-ixing changes whose the change log contains bug identiier.…”

Section: Sotware Maintenancementioning

confidence: 99%

Predictive Models in Software Engineering: Challenges and Opportunities

Yang

Xia

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Predictive models are one of the most important techniques that are widely applied in many areas of software engineering. There have been a large number of primary studies that apply predictive models and that present well-performed studies in various research domains, including software requirements, software design and development, testing and debugging and software maintenance. This paper is a first attempt to systematically organize knowledge in this area by surveying a body of 421 papers on predictive models published between 2009 and 2020. We describe the key models and approaches used, classify the different models, summarize the range of key application areas, and analyze research results. Based on our findings, we also propose a set of current challenges that still need to be addressed in future work and provide a proposed research road map for these opportunities.

show abstract

“…Normally, classifiers can be trained based on the dataset collected previously. The existing ML techniques have been facilitated many SDP approaches [30][31][32][33]. The classifier with the highest performance index is selected by evaluating the performance in terms of balance which reflects class imbalance.…”

mentioning

confidence: 99%

HASPO: Harmony Search-Based Parameter Optimization for Just-in-Time Software Defect Prediction in Maritime Software

et al. 2021

View full text Add to dashboard Cite

Software is playing the most important role in recent vehicle innovations, and consequently the amount of software has rapidly grown in recent decades. The safety-critical nature of ships, one sort of vehicle, makes software quality assurance (SQA) a fundamental prerequisite. Just-in-time software defect prediction (JIT-SDP) aims to conduct software defect prediction (SDP) on commit-level code changes to achieve effective SQA resource allocation. The first case study of SDP in the maritime domain reported feasible prediction performance. However, we still consider that the prediction model has room for improvement since the parameters of the model are not optimized yet. Harmony search (HS) is a widely used music-inspired meta-heuristic optimization algorithm. In this article, we demonstrated that JIT-SDP can produce better performance of prediction by applying HS-based parameter optimization with balanced fitness value. Using two real-world datasets from the maritime software project, we obtained an optimized model that meets the performance criterion beyond the baseline of a previous case study throughout various defect to non-defect class imbalance ratio of datasets. Experiments with open source software also showed better recall for all datasets despite the fact that we considered balance as a performance index. HS-based parameter optimized JIT-SDP can be applied to the maritime domain software with a high class imbalance ratio. Finally, we expect that our research can be extended to improve the performance of JIT-SDP not only in maritime domain software but also in open source software.

show abstract

The impact of context metrics on just-in-time defect prediction

Cited by 45 publications

References 50 publications

Early Life Cycle Software Defect Prediction. Why? How?

Early Life Cycle Software Defect Prediction. Why? How?

Predictive Models in Software Engineering: Challenges and Opportunities

HASPO: Harmony Search-Based Parameter Optimization for Just-in-Time Software Defect Prediction in Maritime Software

Contact Info

Product

Resources

About