Investigating the effect of “defect co-fix” on quality assurance resource allocation: A search-based approach

Hemmati, Hadi; Nagappan, Meiyappan; Hassan, Ahmed E.

doi:10.1016/j.jss.2014.11.040

Cited by 8 publications

(7 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the results are similar. Hemmati et al [20] use a similar variant of ROC that considers the percentage of defects detected versus the percentage of lines of code. Arisholm and Briand [21] propose an approach similar to a ROC curve.…”

Section: Cost Metricsmentioning

confidence: 99%

“…In the following, we use the labels 1 = Def ective and 0 = Clean, i.e., the prediction model classifies artifacts into clean and defective artifacts. With the exception of a publication by Hemmati et al [20], the current state of the art assumes each artifact that is correctly labeled as defective, predicts all defects in that affect an artifact correctly. However, this is not necessarily the case, because a defect d ∈ D may affect multiple artifacts.…”

Section: Specification Of Defect Predictionmentioning

confidence: 99%

“…This problem affects all current defect prediction research. To the best of our knowledge, there is only the work by Hemmati et al [20], that exploits the n-to-m relationship for the generation of a ranking of files. However, Hemmati et al evaluate their study based on the density of remaining defects per file, i.e., they use an effort-aware 1-to-m metric for the the evalution of their results.…”

mentioning

confidence: 99%

“…This is possible, because our cost model only requires a binary function for labeling of single artifacts as defined in Equation (2). Regardless, algorithms that would already consider the nto-m relationship while fitting a prediction model, as is done by Hemmati et al [20], are likely to perform better.…”

mentioning

confidence: 99%

See 3 more Smart Citations

On the Costs and Profit of Software Defect Prediction

Herbold

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Defect prediction can be a powerful tool to guide the use of quality assurance resources. However, while lots of research covered methods for defect prediction as well as methodological aspects of defect prediction research, the actual cost saving potential of defect prediction is still unclear. Within this article, we close this research gap and formulate a cost model for software defect prediction. We derive mathematically provable boundary conditions that must be fulfilled by defect prediction models such that there is a positive profit when the defect prediction model is used. Our cost model includes aspects like the costs for quality assurance, the costs of post-release defects, the possibility that quality assurance fails to reveal predicted defects, and the relationship between software artifacts and defects. We initialize the cost model using different assumptions, perform experiments to show trends of the behavior of costs on real projects. Our results show that the unrealistic assumption that defects only affect a single software artifact, which is a standard practice in the defect prediction literature, leads to inaccurate cost estimations. Moreover, the results indicate that thresholds for machine learning metrics are also not suited to define success criteria for software defect prediction.Index Terms-Defect prediction, costs, return on investment ! • S. Herbold is with the

show abstract

Section: Cost Metricsmentioning

confidence: 99%

Section: Specification Of Defect Predictionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

On the Costs and Profit of Software Defect Prediction

Herbold

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…As while testing the software or before testing all risks must be mitigated by proper assessment and that only can lead to software quality improvement. Number of remaining defects can be one of the effective measure for evaluation of QA prioritization 2 . For maintaining quality of the system we should be able to dynamically update the level of non reviewed files dependent on co fixes done while doing review of previous files .All the software artifacts i.e.…”

mentioning

confidence: 99%

Tools, Strategies & Models for Incorporating Software Quality Assurance in Risk Oriented Testing

Malik

Singh

2017

Orient. J. Chem

View full text Add to dashboard Cite

Evolution of software is cumbersome process and also needs many iterations of software testing for satisfying some quality criteria. Software quality assurance activities must be effectively used for the proper software quality management and to achieve good product quality .Effective quality management is related to Value Engineering and Risk Management. In the present paper we will study relevance of quality assurance tools, strategies and models while doing risk based testing for the proper product function orientation .By analysing risks we can get to know how much we need to do the testing of software and further can assure the software quality.

show abstract

Exploring the relationship between performance metrics and cost saving potential of defect prediction models

Herbold

2022

Empir Software Eng

View full text Add to dashboard Cite

Context: Performance metrics are a core component of the evaluation of any machine learning model and used to compare models and estimate their usefulness. Recent work started to question the validity of many performance metrics for this purpose in the context of software defect prediction. Objective: Within this study, we explore the relationship between performance metrics and the cost saving potential of defect prediction models. We study whether performance metrics are suitable proxies to evaluate the cost saving capabilities and derive a theory for the relationship between performance metrics and cost saving potential. Methods: We measure performance metrics and cost saving potential in defect prediction experiments. We use a multinomial logit model, decision, and random forest to model the relationship between the metrics and the cost savings. Results: We could not find a stable relationship between cost savings and performance metrics. We attribute the lack of the relationship to the inability of performance metrics to account for the property that a small proportion of very large software artifacts are the main driver of the costs. fact that performance metrics are incapable of accurately considering the costs associate with individual artifacts, which is required due to the exponential distribution of artifact sizes. Conclusion: Any defect prediction study interested in finding the best prediction model, must consider cost savings directly, because no reasonable claims regarding the economic benefits of defect prediction can be made otherwise.

show abstract

Investigating the effect of “defect co-fix” on quality assurance resource allocation: A search-based approach

Cited by 8 publications

References 37 publications

On the Costs and Profit of Software Defect Prediction

On the Costs and Profit of Software Defect Prediction

Tools, Strategies & Models for Incorporating Software Quality Assurance in Risk Oriented Testing

Exploring the relationship between performance metrics and cost saving potential of defect prediction models

Contact Info

Product

Resources

About