File-Level Defect Prediction: Unsupervised vs. Supervised Models

Yan, Meng; Fang, Yicheng; Lo, David; Xia, Xin; Zhang, Xiaohong

doi:10.1109/esem.2017.48

Cited by 57 publications

(28 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two main types of defect identification approaches in terms of the identification granularity, i.e., module-level and changelevel. Module-level is targeted at discovering defective modules (e.g., packages, files, or methods) [11,22,26,28,41,50,60]. Change-level is targeted at identifying defect-introducing changes [17].…”

Section: Introductionmentioning

confidence: 99%

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Yan

Xia

Fan

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

Self Cite

View full text Add to dashboard Cite

Effort-aware Just-in-Time (JIT) defect identification aims at identifying defect-introducing changes just-in-time with limited code inspection effort. Such identification has two benefits compared with traditional module-level defect identification, i.e., identifying defects in a more cost-effective and efficient manner. Recently, researchers have proposed various effort-aware JIT defect identification approaches, including supervised (e.g., CBS+, OneWay) and unsupervised approaches (e.g., LT and Code Churn). The comparison of the effectiveness between such supervised and unsupervised approaches has attracted a large amount of research interest. However, the effectiveness of the recently proposed approaches and the comparison among them have never been investigated in an industrial setting. In this paper, we investigate the effectiveness of state-of-the-art effort-aware JIT defect identification approaches in an industrial setting. To that end, we conduct a case study on 14 Alibaba projects with 196,790 changes. In our case study, we investigate three aspects: (1) The effectiveness of state-of-the-art supervised (i.e., CBS+, OneWay, EALR) and unsupervised (i.e., LT and Code Churn) effortaware JIT defect identification approaches on Alibaba projects, (2) the importance of the features used in the effort-aware JIT defect identification approach, and (3) the association between projectspecific factors and the likelihood of a defective change. Moreover, we develop a tool based on the best performing approach and investigate the tool's effectiveness in a real-life setting at Alibaba.

show abstract

Section: Introductionmentioning

confidence: 99%

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Yan

Xia

Fan

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

Self Cite

View full text Add to dashboard Cite

show abstract

“…5. Nearly all the prior work on unsupervised learning focus on defect prediction [21,47,49,75,76,77,80,81,87,88]. The performance of our framework suggests that many more domains in software analytics could benefit from unsupervised learning.…”

Section: Resultmentioning

confidence: 99%

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

Tu¹

2022

Preprint

View full text Add to dashboard Cite

Keeping track of and managing Self-Admitted Technical Debts (SATDs) is important for maintaining a healthy software project. Current active-learning SATD recognition tool involves manual inspection of 24% of the test comments on average to reach 90% of the recall. Among all the test comments, about 5% are SATDs. The human experts are then required to read almost a quintuple of the SATD comments which indicates the inefficiency of the tool. Plus, human experts are still prone to error: 95% of the false-positive labels from previous work were actually true positives.To solve the above problems, we propose DebtFree, a two-mode framework based on unsupervised learning for identifying SATDs. In mode1, when the existing training data is unlabeled, DebtFree starts with an unsupervised learner to automatically pseudo-label the programming comments in the training data. In contrasts, in mode2 where labels are available with the corresponding training data, DebtFree starts with a pre-processor that identifies the highly prone SATDs from the test dataset. Then, our machine learning model is employed to assist human experts in manually identifying the remaining SATDs. Our experiments on 10 software projects show that both models yield statistically significant improvement in effectiveness over the state-of-the-art automated and semi-automated models. Specifically, DebtFree can reduce the labeling effort by 99% in mode1 (unlabeled training data), and up to 63% in mode2 (labeled training data) while improving the current active learner's F1 relatively to almost 100%.

show abstract

“…Meng at el. [5] have compared the effectiveness of unsupervised and supervised prediction models for effort-aware file-level defect prediction and suggested that unsupervised models do not perform statistically significantly better than state-of-art supervised model under within-project setting. Chen at el.…”

Section: Background and Related Workmentioning

confidence: 99%

Software Defect Prediction Via Deep Learning

Khan*¹,

Albahli²,

Albattah³

et al. 2020

IJITEE

View full text Add to dashboard Cite

Existing models on defect prediction are trained on historical limited data which has been studied from a variety of pioneering and researchers. Cross-project defect prediction, which is often reuse data from other projects, works well when the data of training models is completely sufficient to meet the project demands. However, current studies on software defect prediction require some degree of heterogeneity of metric values that does not always lead to accurate predictions. Inspired by the current research studies, this paper takes the benefit with the state-of-the-art of deep learning and random forest to perform various experiments using five different datasets. Our model is ideal for predicting of defects with 90% accuracy using 10-fold cross-validation. The achieved results show that Random Forest and Deep learning are giving more accurate predictions with compared to Bayes network and SVM on all five datasets. We also derived Deep Learning that can be competitive classifiers and provide more robust for detecting defect prediction.

show abstract

File-Level Defect Prediction: Unsupervised vs. Supervised Models

Cited by 57 publications

References 45 publications

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

Effort-aware just-in-time defect identification in practice: a case study at Alibaba

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

Software Defect Prediction Via Deep Learning

Contact Info

Product

Resources

About