On effort-aware metrics for defect prediction

Çarka, Jonida; Esposito, Matteo; Falessi, Davide

doi:10.1007/s10664-022-10186-7

Cited by 16 publications

(13 citation statements)

References 117 publications

(95 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Qu et al [62] proposed integrating developer information into EADP to enhance performance. Carka et al [63] proposed to assess the EADP performance using the normalised PofB, which sorted software modules according to the predicted defect densities. Huang et al [17] proposed the CBS+ algorithm for EADP.…”

Section: Effort-aware Defect Predictionmentioning

confidence: 99%

See 1 more Smart Citation

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

Keung

et al. 2023

IET Software

View full text Add to dashboard Cite

Effort‐Aware Defect Prediction (EADP) methods sort software modules based on the defect density and guide the testing team to inspect the modules with high defect density first. Previous studies indicated that some feature selection methods could improve the performance of Classification‐Based Defect Prediction (CBDP) models, and the Correlation‐based feature subset selection method with the Best First strategy (CorBF) performed the best. However, the practical benefits of feature selection methods on EADP performance are still unknown, and blindly employing the best‐performing CorBF method in CBDP to pre‐process the defect datasets may not improve the performance of EADP models but possibly result in performance degradation. To assess the impact of the feature selection techniques on EADP, a total of 24 feature selection methods with 10 classifiers embedded in a state‐of‐the‐art EADP model (CBS+) on the 41 PROMISE defect datasets were examined. We employ six evaluation metrics to assess the performance of EADP models comprehensively. The results show that (1) The impact of the feature selection methods varies in classifiers and datasets. (2) The four wrapper‐based feature subset selection methods with forwards search, that is, AdaBoost with Forwards Search, Deep Forest with Forwards Search, Random Forest with Forwards Search, and XGBoost with Forwards Search (XGBF) are better than other methods across the studied classifiers and the used datasets. And XGBF with XGBoost as the embedded classifier in CBS+ performs the best on the datasets. (3) The best‐performing CorBF method in CBDP does not perform well on the EADP task. (4) The selected features vary with different feature selection methods and different datasets, and the features noc (number of children), ic (inheritance coupling), cbo (coupling between object classes), and cbm (coupling between methods) are frequently selected by the four wrapper‐based feature subset selection methods with forwards search. (5) Using AdaBoost, deep forest, random forest, and XGBoost as the base classifiers embedded in CBS+ can achieve the best performance. In summary, we recommend the software testing team should employ XGBF with XGBoost as the embedded classifier in CBS+ to enhance the EADP performance.

show abstract

Section: Effort-aware Defect Predictionmentioning

confidence: 99%

“…Carka et al. [63] proposed to assess the EADP performance using the normalised PofB, which sorted software modules according to the predicted defect densities. Huang et al.…”

Section: Related Workmentioning

confidence: 99%

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

Keung

et al. 2023

IET Software

View full text Add to dashboard Cite

show abstract

“…Ulan et al [66] proposed an unsupervised EADP method based on weighted metric aggregation. Carka et al [67] proposed to evaluate the EADP performance using normalised PofB, which ranked software modules based on predicted defect densities. Zhao et al [21], Xu et al [68] and Cheng et al [22] proposed three JIT EADP methods for Android applications.…”

Section: Effort-aware Defect Predictionmentioning

confidence: 99%

“…Carka et al. [67] proposed to evaluate the EADP performance using normalised PofB, which ranked software modules based on predicted defect densities. Zhao et al.…”

Section: Related Workmentioning

confidence: 99%

Revisiting ‘revisiting supervised methods for effort‐aware cross‐project defect prediction’

Yang

Keung

et al. 2023

IET Software

View full text Add to dashboard Cite

Effort-aware cross-project defect prediction (EACPDP), which uses cross-project software modules to build a model to rank within-project software modules based on the defect density, has been suggested to allocate limited testing resource efficiently. Recently, Ni et al. proposed an EACPDP method called EASC, which used all cross-project modules to train a model without considering the data distribution difference between cross-project and within-project data. In addition, Ni et al. employed the different defect density calculation strategies when comparing EASC and baseline methods. To explore the effective defect density calculation strategies and methods on EACPDP, the authors compare four data filtering methods and five transfer learning methods with EASC using four commonly used defect density calculation strategies. The authors use three classification evaluation metrics and seven effort-aware metrics to assess the performance of methods on 11 PROMISE datasets comprehensively. The results show that (1) The classification before sorting (CBS+) defect density calculation strategy achieves the best overall performance. (2) Using balanced distribution adaption (BDA) and joint distribution adaptation (JDA) with the K-nearest neighbour classifier to build the EACPDP model can find 15% and 14.3% more defective modules and 11.6% and 8.9% more defects while achieving the acceptable initial false alarms (IFA). (3) Better comprehensive classification performance of the methods can bring better EACPDP performance to some extent. (4) A flexible adjustment of the defect threshold λ of the CBS+ strategy contribute to different goals. In summary, the authors recommend researchers and practitioners use to BDA and JDA with the CBS+ strategy to build the EACPDP model. K E Y W O R D Sdata mining, quality assurance, software engineering, software maintenance, software metrics, software quality | INTRODUCTIONComputer software is widely used in various industries in society today, which may fail with quality problems. As software plays an increasingly important role in various fields, ensuring software reliability is one of the issues people are more concerned about [1][2][3]. Software defects are a potential sources of errors, failures, and crashes of associated systems. However, the increase in the scale of the software makes defect inspection and fixing more time-consuming. Worse yet, softwareThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

show abstract

“…Cabral and Minku., (2022) proposed a new JIT-SDP approach, which provided higher and more stable predictive performance (i.e., reliable) over time. Carka et al, (2022) investigated the trend and effectiveness of Effort-aware JIT-SDP, and proposed and evaluated the standardization of PofB.…”

Section: Effort-aware Software Defect Predictionmentioning

confidence: 99%

EANDL: A High-performance and Low-cost Effort-Aware Just-in-Time Software Defect Prediction Model

Chen,

Wang,

Song

et al. 2023

Preprint

View full text Add to dashboard Cite

Just-in-time software defect prediction (JIT-SDP) is a fine-grained software defect prediction technology and its predictive entity is code changes. Effort-aware just-in-time software defect prediction (Effort-aware JIT-SDP) is a defect prediction technology that takes the cost of detection into consideration with the goal of detecting more defect changes with less cost. To solve the problem of poor prediction performance of Effort aware JIT-SDP model, an Effort-aware JIT-SDP model called EANDL based on NSGA3, Differential Evolution and Logistic Regression is proposed. Firstly, two evaluation metrics called DPR and LDSOD are proposed to evaluate the sorting performance. Secondly, the prediction model is constructed by using logistic regression and maximizing two evaluation metrics using NSGA3-DE so as to determine the optimal coefficient vector of logistic regression. In addition, two metrics called ACE and ACEP to measure the effort consumption of Effort-aware JIT-SDP models are proposed. Finally, intensive simulation experiments have been conducted on six open-source datasets to demonstrate the performance of the proposed method. Empirical results demonstrate that EANDL outperforms eight typical and classical supervised and unsupervised models in terms of sorting performance, effort consumption, and response time.

show abstract

On effort-aware metrics for defect prediction

Cited by 16 publications

References 117 publications

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

The impact of feature selection techniques on effort‐aware defect prediction: An empirical study

Revisiting ‘revisiting supervised methods for effort‐aware cross‐project defect prediction’

EANDL: A High-performance and Low-cost Effort-Aware Just-in-Time Software Defect Prediction Model

Contact Info

Product

Resources

About