Attribute Selection and Imbalanced Data: Problems in Software Defect Prediction

Khoshgoftaar, Taghi M.; Gao, Kehan; Seliya, Naeem

doi:10.1109/ictai.2010.27

Cited by 118 publications

(74 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…From our experience in this research, the problem of attributes selection was a key aspect to obtain satisfactory results, which was also confirmed by [16], [17], and [18]. With appropriate attributes identified, the better accuracy could be achieved for a smaller sets of attributes with a simple appropriate classifier.…”

Section: Resultssupporting

confidence: 56%

A case study on machine learning model for code review expert system in software engineering

Madera¹,

Tomon²

2017

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Abstract-Code review is a key tool for quality assurance in software development. It is intended to find coding mistakes overlooked during development phase and lower risk of bugs in final product. In large and complex projects accurate code review is a challenging task. As code review depends on individual reviewer predisposition there is certain margin of source code changes that is not checked as it should. In this paper we propose machine learning approach for pointing project artifacts that are significantly at risk of failure. Planning and adjusting quality assurance (QA) activities could strongly benefit from accurate estimation of software areas endangered by defects. Extended code review could be directed there. The proposed approach has been evaluated for feasibility on large medical software project. Significant work was done to extract features from heterogeneous production data, leading to good predictive model. Our preliminary research results were considered worthy of implementation in the company where the research has been conducted, thus opening the opportunities for the continuation of the studies.

show abstract

Section: Resultssupporting

confidence: 56%

A case study on machine learning model for code review expert system in software engineering

Madera¹,

Tomon²

2017

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

show abstract

“…OSS just like the other undersampling techniques however, does not explicitly avoid the inclusion of outliers in the final training set. Also, the performance of OSS in most empirical studies, particularly in software defect prediction using the NASA MDP datasets where the presence of noise and repeated instances is evident, is not encouraging as it has been consistently outperformed by other techniques such as RUS and SMOTE [4], [25], and [26].…”

Section: Related Workmentioning

confidence: 99%

New Cluster Undersampling Technique for Class Imbalance Learning

Sowah¹,

Agebure²,

Mills³

et al. 2016

IJMLC

View full text Add to dashboard Cite

Abstract-Adequately learning from datasets that are highly imbalance has become one of the most challenging tasks in Data Mining and Machine Learning disciplines. Most datasets from high risk application areas are often adversely affected by the class imbalance problem due to the limited occurrence of positive examples. This paper presents a new undersampling technique, called Cluster Undersampling Technique (CUST) that has the capability of further improving the performance of classification algorithms when learning from imbalance datasets. The performance of CUST is evaluated by using it to undersample 16 real world class imbalance datasets prior to building classification models using C4.5 decision tree and OneR algorithms. The performance of the models are compared to the performance of random undersampling and oversampling, synthetic minority oversampling, one-sided selection, and cluster-based undersampling. The experimental results using area under receiver-operating characteristic curve and geometric mean showed that CUST resulted in higher performance and is statistically better compared to well-known techniques.

show abstract

“…Four sampling methods were considered in their work, namely, random undersampling, random oversampling, synthetic minority over-sampling (SMOTE) and Wilson editing. Khoshgoftaar et al [22] presented a process involving a random undersampling technique for addressing uneven class distribution to select important attributes in software engineering. Gao et al [13] proposed a new technique, called SelectRUSBoost, which is a form of ensemble learning that incorporates random undersampling into feature selection to alleviate class imbalance.…”

Section: Feature Selection and Class Imbalancementioning

confidence: 99%

A new transferred feature selection algorithm for customer identification

Zhu¹,

Niu²,

Xiao³

et al. 2016

Neural Comput & Applic

View full text Add to dashboard Cite

Class imbalance brings great challenges to feature selection in customer identification and most of current feature selection approaches cannot produce good prediction on the minority class. A number of studies have attempted to solve this issue by using resampling techniques. However, resampling techniques only use the in-domain information and they cannot achieve good performance when the imbalance is caused by the absolute rarity of the minority class. In this paper, we focus on the issue of feature selection with class imbalance caused by absolute rarity. By introducing the idea of transfer learning, we develop a transferred feature selection method based on the group method of data handling neural network (GMDH). The proposed ensemble neural network extracts information of similar customers from related domains to deal with the information scarcity of the minority class in the target domain. Experiments are done on a real-world application from cigarette company. The results indicate that the new method gives better predictive performance than other benchmark feature selection methods, especially the predictive accuracy of minority high-value customers. At the same time, the new algorithm can help to identify important features that distinguish highvalue customers from low-value ones.

show abstract

Attribute Selection and Imbalanced Data: Problems in Software Defect Prediction

Cited by 118 publications

References 20 publications

A case study on machine learning model for code review expert system in software engineering

A case study on machine learning model for code review expert system in software engineering

New Cluster Undersampling Technique for Class Imbalance Learning

A new transferred feature selection algorithm for customer identification

Contact Info

Product

Resources

About