The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

Abstract

Effort‐Aware Defect Prediction (EADP) methods sort software modules based on the defect density and guide the testing team to inspect the modules with high defect density first. Previous studies indicated that some feature selection methods could improve the performance of Classification‐Based Defect Prediction (CBDP) models, and the Correlation‐based feature subset selection method with the Best First strategy (CorBF) performed the best. However, the practical benefits of feature selection methods on EADP performance are still unknown, and blindly employing the best‐performing CorBF method in CBDP to pre‐process the defect datasets may not improve the performance of EADP models but possibly result in performance degradation. To assess the impact of the feature selection techniques on EADP, a total of 24 feature selection methods with 10 classifiers embedded in a state‐of‐the‐art EADP model (CBS+) on the 41 PROMISE defect datasets were examined. We employ six evaluation metrics to assess the performance of EADP models comprehensively. The results show that (1) The impact of the feature selection methods varies in classifiers and datasets. (2) The four wrapper‐based feature subset selection methods with forwards search, that is, AdaBoost with Forwards Search, Deep Forest with Forwards Search, Random Forest with Forwards Search, and XGBoost with Forwards Search (XGBF) are better than other methods across the studied classifiers and the used datasets. And XGBF with XGBoost as the embedded classifier in CBS+ performs the best on the datasets. (3) The best‐performing CorBF method in CBDP does not perform well on the EADP task. (4) The selected features vary with different feature selection methods and different datasets, and the features noc (number of children), ic (inheritance coupling), cbo (coupling between object classes), and cbm (coupling between methods) are frequently selected by the four wrapper‐based feature subset selection methods with forwards search. (5) Using AdaBoost, deep forest, random forest, and XGBoost as the base classifiers embedded in CBS+ can achieve the best performance. In summary, we recommend the software testing team should employ XGBF with XGBoost as the embedded classifier in CBS+ to enhance the EADP performance.

The impact of feature selection techniques on aging-related bug prediction models: An empirical investigation

“…The results indicated that the feature selection technique could enhance the performance of the KNN, MLP, and LR classifiers, with the KNN classifier achieving the best performance with seven features. Li et al [48] give an investigation to discuss how the feature selection method impacts the effort-aware defect prediction task. They construct experiments on 41 PROMISE datasets with 24 feature selection methods and 10 classifiers.…”

On the relative value of imbalanced learning for code smell detection

“…In our empirical study, we use the three threshold-dependent evaluation metrics (Precision, Recall, and F-measure (F1)) and one threshold-independent evaluation metric (Matthews correlation coefficient, MCC) to evaluate the performance of CSD models. The metrics are widely used in both software engineering studies [64][65][66][67][68][69][70][71] and artificial intelligence researches. [72][73][74][75] In the binary classification problem, these four evaluation metrics can be calculated according to a confusion matrix, as shown in Table 4.…”

A compositional model for cross-project effort-aware defect prediction

“…This helps select suitable algorithms based on software project requirements, addressing ranking instability in EADP studies. Li et al 19 investigated the impact of feature selection methods on EADP. They employed six effort-aware metrics to assess the EADP models' performance comprehensively.…”

“…Consequently, Software Defect Prediction (SDP) techniques have garnered significant attention as they enable the optimal utilization of limited resources. Software testing teams construct SDP models based on historical software data to predict the defect proneness of software modules to be inspected 3 . This allows them to allocate testing resources more effectively or prioritize the inspection of those modules that are predicted to have defects, thereby facilitating the efficient allocation of software testing resources.…”

“…While research on CPEADP is actively ongoing, there is still a need for further improvement in methods to enhance performance in cross-project scenarios. This is because the differences in data distributions in cross-project settings can significantly impact model performance 3 . Traditional machine learning algorithms assume that training and testing data come from the same data distribution, which is challenging to achieve in many practical applications.…”