A Search-based Training Algorithm for Cost-aware Defect Prediction

Abstract

Research has yielded approaches to predict future defects in software artifacts based on historical information, thus assisting companies in effectively allocating limited development resources and developers in reviewing each others' code changes. Developers are unlikely to devote the same effort to inspect each software artifact predicted to contain defects, since the effort varies with the artifacts' size (cost) and the number of defects it exhibits (effectiveness). We propose to use Genetic Algorithms (GAs) for training prediction models to maximize their cost-effectiveness. We evaluate the approach on two well-known models, Regression Tree and Generalized Linear Model, and predict defects between multiple releases of six open source projects. Our results show that regression models trained by GAs significantly outperform their traditional counterparts, improving the cost-effectiveness by up to 240%. Often the top 10% of predicted lines of code contain up to twice as many defects. ABSTRACTResearch has yielded approaches to predict future defects in software artifacts based on historical information, thus assisting companies in effectively allocating limited development resources and developers in reviewing each others' code changes. Developers are unlikely to devote the same effort to inspect each software artifact predicted to contain defects, since the effort varies with the artifacts' size (cost) and the number of defects it exhibits (effectiveness). We propose to use Genetic Algorithms (GAs) for training prediction models to maximize their cost-effectiveness. We evaluate the approach on two well-known models, Regression Tree and Generalized Linear Model, and predict defects between multiple releases of six open source projects. Our results show that regression models trained by GAs significantly outperform their traditional counterparts, improving the cost-effectiveness by up to 240%. Often the top 10% of predicted lines of code contain up to twice as many defects.

Fine-grained just-in-time defect prediction

“…Another threat regards how we assess the costeffectiveness of the models experimented. As done in previous research [9,64,2,33,49,50], we measure the inspection cost in terms of lines of code to be inspected by a reviewer. LOC has been evaluated as a valid proxy measure [2] since it is correlated with code and cognitive complexity [56].…”

Heterogeneous fault prediction with cost‐sensitive domain adaptation

“…Heterogeneous fault prediction is very promising, as it permits potentially all heterogeneous data of software projects to be used for fault predic- For the future work, we would like to use more software project data that contains both open-source and commercial proprietary projects to validate the generalization ability of our CLSUP approach. In addition, we will try to address the HFP problem with other popular techniques, such as deep learning [51] and search-based optimization methods [4,53].…”

“…They developed a multiobjective fault predictor to train logistic regression (LR) and decision trees models by using a genetic algorithm, which is a cost‐effectiveness model. Later on, Panichella et al presented to use genetic algorithms for training prediction models to maximize their cost‐effectiveness. They applied regression tree and generalized linear model to predict defects between multiple releases of six open source projects.…”

Progress on approaches to software defect prediction

“…Panichella et al [128] presented to train defect predictors with the maximisation of their cost‐effectiveness through GAs. They employed regression tree and generalised linear model to build defect prediction models, and found that regression models trained by GAs achieve better performance than their traditional counterparts.…”