An Empirical Analysis of the Correlation between CK Metrics, Test Coverage and Mutation Score

Cruz, Robinson Crusoé Da; Eler, Marcelo Medeiros

doi:10.5220/0006312703410350

Cited by 7 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to generate the instance space of automated testing and investigate the research questions formulated in Section 1, the following steps are required [24], [37]:…”

Section: Instance Space Analysismentioning

confidence: 99%

Instance Space Analysis of Search-Based Software Testing

Neelofar

Smith‐Miles

Muñoz

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Search-based software testing (SBST) is now a mature area, with numerous techniques developed to tackle the challenging task of software testing. SBST techniques have shown promising results and have been successfully applied in the industry to automatically generate test cases for large and complex software systems. Their effectiveness, however, has been shown to be problem dependent. In this paper, we revisit the problem of objective performance evaluation of SBST techniques in light of recent methodological advances -in the form of Instance Space Analysis (ISA) -enabling the strengths and weaknesses of SBST techniques to be visualised and assessed across the broadest possible space of problem instances (software classes) from common benchmark datasets. We identify features of SBST problems that explain why a particular instance is hard for an SBST technique, reveal areas of hard and easy problems in the instance space of existing benchmark datasets, and identify the strengths and weaknesses of state-of-the-art SBST techniques. In addition, we examine the diversity and quality of common benchmark datasets used in experimental evaluations.

show abstract

“…In order to generate the instance space of automated testing and investigate the research questions formulated in Section 1, the following steps are required [24], [37]:…”

Section: Instance Space Analysismentioning

confidence: 99%

Instance Space Analysis of Search-Based Software Testing

Neelofar

Smith‐Miles

Muñoz

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Afterward, the statistical test, independent t-test, was used to determine whether the ensemble meta-estimator, VoR, performed better than base regressors. Some researchers consider coverage metrics such as branch coverage as a basis for predicting test effectiveness [36,70].…”

Section: Prediction Models Evaluationmentioning

confidence: 99%

An ensemble meta-estimator to predict source code testability

Zakeri‐Nasrabadi

Parsa

2022

Applied Soft Computing

View full text Add to dashboard Cite

“…For each model, we measure and report four typical error metrics, including MAE, MSE, Root Mean-Squared Error (RMSE), Mean-Squared Logarithmic Error (MSLgE), and Median Absolute Error (MdAE). These metrics are given by Equations ( 8)- (12). In all equations, y ˆi is the predicted value of the ith sample, y i is the corresponding actual value for y ˆi and n is the number of samples.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…They concluded that code coverage is an appropriate means to determine test effectiveness. Cruz and Eler [12] have investigated the correlation between test coverage, mutation score, and object-oriented systems metrics. Their preliminary results show that some C&K metrics [13] strongly affects line coverage and mutation score.…”

Section: Test Effectiveness Measurementmentioning

confidence: 99%

Learning to predict test effectiveness

Zakeri‐Nasrabadi

Parsa

2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

The high cost of the test can be dramatically reduced, provided that the coverability as an inherent feature of the code under test is predictable. This article offers a machine learning model to predict the extent to which the test could cover a class in terms of a new metric called Coverageability. The prediction model consists of an ensemble of four regression models. The learning samples consist of feature vectors, where features are source code metrics computed for a class. The samples are labeled by the Coverageability values computed for their corresponding classes. We offer a mathematical model to evaluate test effectiveness in terms of size and coverage of the test suite generated automatically for each class. We extend the size of the feature space by introducing a new approach to define submetrics in terms of existing source code metrics. Using feature importance analysis on the learned prediction models, we sort sources code metrics in the order of their impact on the test effectiveness. As a result of which we found the class strict cyclomatic complexity as the most influential source code metric. Our experiments with our prediction models on a large corpus of Java projects containing about 23,000 classes demonstrate the Mean Absolute Error (MAE) of 0.032, Mean-Squared Error (MSE) of 0.004, and an R 2 score of 0.855. Compared with the state-of-the-art coverage prediction models, our models improve MAE, MSE, and an R 2 score by 5.78%, 2.84%, and 20.71%, respectively.

show abstract

An Empirical Analysis of the Correlation between CK Metrics, Test Coverage and Mutation Score

Cited by 7 publications

References 15 publications

Instance Space Analysis of Search-Based Software Testing

Instance Space Analysis of Search-Based Software Testing

An ensemble meta-estimator to predict source code testability

Learning to predict test effectiveness

Contact Info

Product

Resources

About