MAHAKIL: Diversity Based Oversampling Approach to Alleviate the Class Imbalance Issue in Software Defect Prediction

Bennin, Kwabena Ebo; Keung, Jacky; Phannachitta, Passakorn; Monden, Akito; Mensah, Solomon

doi:10.1109/tse.2017.2731766

Cited by 231 publications

(94 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It then picks of those rows y i at random and creates a new example at a randomly selected distance between x and y i . Some recent results report that off-the-shelf SMOTE can be improved by some local tuning [3], [12]. We do not use such local tuning since recently is has been shown that such tunings are outperformed by FFTs (see below).…”

Section: Smotementioning

confidence: 99%

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Standard automatic methods for recognizing problematic development commits can be greatly improved via the incremental application of human+artificial expertise. In this approach, called EMBLEM, an AI tool first explore the software development process to label commits that are most problematic. Humans then apply their expertise to check those labels (perhaps resulting in the AI updating the support vectors within their SVM learner). We recommend this human+AI partnership, for several reasons. When a new domain is encountered, EMBLEM can learn better ways to label which comments refer to real problems. Also, in studies with 9 open source software projects, labelling via EMBLEM's incremental application of human+AI is at least an order of magnitude cheaper than existing methods (≈ eight times). Further, EMBLEM is very effective. For the data sets explored here, EMBLEM better labelling methods significantly improved Popt20 and G-score performance in nearly all the projects studied here. TABLE 1This paper argues against using keywords like these as a method for labelling a commit as "buggy'.

show abstract

Section: Smotementioning

confidence: 99%

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Quite often researchers propose one specific use of a machine learning model in software defect prediction, such as Naï ve Bayes [10], and use the work of Lessmann et al [4] to justify their choice of classifier(s) [53] [38] showed that RF was significantly better than 21 other prediction models." [29], although Lessmann et al do not make any such assertion [4]. This study shows that, although it is still unclear which classifier performs the best, researchers should justify the use and validity of their choice of classifier [48].…”

Section: Comparison With Other Studiesmentioning

confidence: 62%

Evaluating Software Defect Prediction Performance: An Updated Benchmarking Study

Lessmann

Baesens

2019

SSRN Journal

View full text Add to dashboard Cite

Accurately predicting faulty software units helps practitioners target faulty units and prioritize their efforts to maintain software quality. Prior studies use machine-learning models to detect faulty software code. We revisit past studies and point out potential improvements. Our new study proposes a revised benchmarking configuration. The configuration considers many new dimensions, such as class distribution sampling, evaluation metrics, and testing procedures. The new study also includes new datasets and models. Our findings suggest that predictive accuracy is generally good. However, predictive power is heavily influenced by the evaluation metrics and testing procedure (frequentist or Bayesian approach). The classifier results depend on the software project. While it is difficult to choose the best classifier, researchers should consider different dimensions to overcome potential bias.

show abstract

“…Of the advanced over-sampling methods, some typical or recently proposed examples are the SMOTE (Chawla et al 2002), MWMOTE (majority weighted minority over-sampling) (Barua et al 2014), graph-based over-sampling (Pé rez-Ortiz et al 2015), diversity-based over-sampling (Bennin et al 2018), and generative adversarial network-based over-sampling (Douzas and Bacao 2018) techniques. Among them, the SMOTE is the most established and widely used over-sampling method, which creates synthetic minority samples by interpolating existing minority samples along the line segments joining the k minority class nearest neighbours.…”

Section: Resampling Techniques For Class Imbalancementioning

confidence: 99%

Malicious web domain identification using online credibility and performance data by considering the class imbalance issue

Chiong

Pranata

et al. 2019

IMDS

View full text Add to dashboard Cite

Purpose -Malicious web domain identification is of significant importance to the security protection of Internet users. With online credibility and performance data, this paper aims to investigate the use of machine learning techniques for malicious web domain identification by considering the class imbalance issue (i.e., there are more benign web domains than malicious ones). Design/methodology/approach -We propose an integrated resampling approach to handle class imbalance by combining the Synthetic Minority Over-sampling TEchnique (SMOTE) and Particle Swarm Optimisation (PSO), a population-based meta-heuristic algorithm. We use the SMOTE for over-sampling and PSO for under-sampling. Findings -By applying eight well-known machine learning classifiers, the proposed integrated resampling approach is comprehensively examined using several imbalanced web domain datasets with different imbalance ratios. Compared to five other well-known resampling approaches, experimental results confirm that the proposed approach is highly effective. Practical implications -This study not only inspires the practical use of online credibility and performance data for identifying malicious web domains, but also provides an effective resampling approach for handling the class imbalance issue in the area of malicious web domain identification. Originality/value -Online credibility and performance data is applied to build malicious web domain identification models using machine learning techniques. An integrated resampling approach is proposed to address the class imbalance issue. The performance of the proposed approach is confirmed based on real-world datasets with different imbalance ratios.

show abstract

MAHAKIL: Diversity Based Oversampling Approach to Alleviate the Class Imbalance Issue in Software Defect Prediction

Cited by 231 publications

References 63 publications

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Evaluating Software Defect Prediction Performance: An Updated Benchmarking Study

Malicious web domain identification using online credibility and performance data by considering the class imbalance issue

Contact Info

Product

Resources

About