REPD: Source code defect prediction as anomaly detection

Afric, Petar; Šikić, Lucija; Kurdija, Adrian Satja; Silic, Mario

doi:10.1016/j.jss.2020.110641

Cited by 10 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many kinds of models have been used to tackle SDP. Researchers have proposed using supervised models [63], [64], [65], semi-supervised models [66], [67], unsupervised models [23], tasks specific models such as BugCache [22] and even approaching the problem as an anomaly detection problem [20], [21].…”

Section: B Metrics Prediction Granularity and Approaches To Sdpmentioning

confidence: 99%

Empirical Study: How Issue Classification Influences Software Defect Prediction

et al. 2023

Self Cite

View full text Add to dashboard Cite

Software defect prediction aims to identify potentially defective software modules to better allocate limited quality assurance resources. Practitioners often do this by utilizing supervised models trained using historical data. This data is gathered by mining version control and issue tracking systems. Version control commits are linked to issues they address. If the linked issue is classified as a bug report, the change is considered as bug fixing. The problem arises from the fact that issues are often incorrectly classified within issue tracking systems. This introduces noise into the gathered datasets. In this paper, we investigate the influence issue classification has on software defect prediction dataset quality and resulting model performance. To do this, we mine data from 7 popular open-source repositories, create issue classification and software defect prediction datasets for each of them. We investigate issue classification using four different methods; a simple keyword heuristic, an improved keyword heuristic, the FastText model and the RoBERTa model. Our results show that using the RoBERTa model for issue classification produces the best software defect prediction datasets, containing on average 14.3641% of mislabeled instances. SDP models trained on such datasets achieve superior performance, to those trained on SDP datasets created using other issue classification methods, in 65 out of 84 experiments, with 55 of them being statistically relevant. Furthermore, in 17 out of 28 experiments we could not show a statistically relevant performance difference between SDP models trained on RoBERTa derived software defect prediction datasets and those created using manually labeled issues.

show abstract

Section: B Metrics Prediction Granularity and Approaches To Sdpmentioning

confidence: 99%

Empirical Study: How Issue Classification Influences Software Defect Prediction

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…is part will elaborate the model proposed in this paper. We regard software defect prediction as anomaly detection with similar intuition and motivation as study works of Nahar Neela et al [11] and Afric et al [12], but our model is completely different from theirs.…”

Section: Our Approachmentioning

confidence: 99%

“…erefore, we need to discard the solution that treats the defect prediction problem as a classification problem. Motivated to study works of Nahar Neela et al [11] and Afric et al [12], we decided to treat software defect prediction as an anomaly detection.…”

Section: Introductionmentioning

confidence: 99%

A Software Defect Prediction Approach Based on BiGAN Anomaly Detection

Zhang

Jiang

2022

Scientific Programming

View full text Add to dashboard Cite

Software defect prediction usually is regarded as a classification problem, but classification models will face the class imbalance problem. Although there are many methods to solve the class imbalance problem, there is no method that can fundamentally solve the problem currently. In addition, supervised learning algorithms are always used to train defect prediction models, but obtaining a large amount of high-quality labelled data requires a lot of time and labor cost. In order to solve the class imbalance problem and eliminate the disadvantage of supervised learning, this paper attempts to predict software defects from a new perspective of anomaly detection. We propose an Anomaly Detection Model Based on BiGAN for Software Defect Prediction (ADGAN-SDP). The model proposed in this paper not only does not need to consider the class imbalance problem but also uses a semi-supervised method to train the model. Eight classification-based software defect prediction models are used as the baseline models and compared with ADGAN-SDP model. We evaluate ADGAN-SDP on 19 projects from NASA, AEEEM, and ReLink repositories. The experimental results show that the ADGAN-SDP model, which has a higher recall, outperforms all baseline models. It is suggested that the anomaly detection approach can be applied to the software defect prediction to fundamentally solve the class imbalance problem.

show abstract

“…Other studies have tackled the problem of class-imbalance in defect prediction data (Khoshgoftaar et al, 2010;Wang and Yao, 2013) by treating this problem as an anomaly detection one, where defective instances are considered as anomalies (Neela et al, 2017;Afric et al, 2020). Neela et al (Neela et al, 2017) construct an anomaly detection approach which incorporates both univariate and multivariate Gaussian distributions to model non-defective software modules.…”

Section: Related Workmentioning

confidence: 99%

“…Afric et al (Afric et al, 2020) present a Reconstruction Error Probability Distribution (REPD) model for within-project defect prediction, and assess its effectiveness on five NASA datasets (i.e., CM1, JM1, KC1, KC2, and PC1) obtained from the PROMISE repository (Sayyad Shirabad and Menzies, 2005). This approach is used to handle point and collective anomalies and it is compared to five ML models (i.e., Gaussian NB, LR, k-NN, decision tree, and Hybrid SMOTE-Ensemble).…”

Section: Related Workmentioning

confidence: 99%

Investigating the Use of One-Class Support Vector Machine for Software Defect Prediction

Moussa¹,

Azar²,

Sarro³

2022

Preprint

View full text Add to dashboard Cite

Early software defect identification is considered an important step towards software quality assurance. Software defect prediction aims at identifying software components that are likely to cause faults before a software is made available to the end-user. To date, this task has been modeled as a twoclass classification problem, however its nature also allows it to be formulated as a one-class classification task.Preliminary results obtained in prior work show that One-Class Support Vector Machine (OCSVM) can outperform two-class classifiers for defect prediction. If confirmed, these results would overcome the data imbalance problem researchers have for long attempted to tackle in this field.In this paper, we further investigate whether learning from one class only is sufficient to produce effective defect prediction models by conducting a thorough large-scale empirical study investigating 15 real-world software projects, three validation scenarios, eight classifiers, robust evaluation measures and statistical significance tests. The results reveal that OCSVM is more suitable for cross-version and cross-project, rather than for within-project defect prediction, thus suggesting it performs better with heterogeneous data. While, we cannot conclude that OCSVM is the best classifier (Random Forest performs best herein), our results show interesting findings that open up further research avenues for training accurate defect prediction classifiers when defective instances are scarce or unavailable.

show abstract

REPD: Source code defect prediction as anomaly detection

Cited by 10 publications

References 18 publications

Empirical Study: How Issue Classification Influences Software Defect Prediction

Empirical Study: How Issue Classification Influences Software Defect Prediction

A Software Defect Prediction Approach Based on BiGAN Anomaly Detection

Investigating the Use of One-Class Support Vector Machine for Software Defect Prediction

Contact Info

Product

Resources

About