Classifying generated white-box tests: an exploratory study

Honfi, Dávid; Micskei, Zoltán

doi:10.1007/s11219-019-09446-5

Cited by 6 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conducted empirical research to analyze test case understandability by exploiting the subset of test case evaluation result (450 instances) from Honfi's study [44] that involved 30 developers and 15 white box generated test cases. We extracted 20 test code metrics from the generated test case and six developer-related metrics from the preliminary survey.…”

Section: Discussionmentioning

confidence: 99%

“…On the developers' side, their performance when classifying white-box test cases has been investigated in terms of whether the output is true (pass) or false (fail) [10]. The experiment involved 106 developers who were asked to classify the outputs of the test cases generated using several methods.…”

Section: Figure 1 Example Of Optimized Test Casementioning

confidence: 99%

“…In a study conducted by [9], it was reported that developers spend almost 50% of their time trying to understand and analyze the outputs of automated testing tools. It has also been found that the misclassification rate is as high as 20% when the developer has to determine whether the results of the test case execution produced successful or failed outputs [10].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Test Case Understandability Model

2020

View full text Add to dashboard Cite

Several automated test case generation techniques have been proposed to date, although the adoption of such techniques in the industry remains low. A key factor that has contributed to this low adoption rate is the difficulty experienced by the developer in terms of reading and understanding automatically generated test cases. For this reason, it is essential to construct a test case understandability model for improving the generated test case. In the present paper, we extracted 20 test case metrics, six developer related metrics and two understandability proxies from a white-box test case classification experiment. Based on these metrics, we employed classification and regression algorithms to build test case understandability model. From the experiment, we can conclude that combined metrics always exhibit better discriminatory performance in classification models as well as a higher correlation in regression models when compared to a model that involved only test case metrics or developer metrics. INDEX TERMS test case, understandability model, automated test case generation

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Figure 1 Example Of Optimized Test Casementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Test Case Understandability Model

2020

View full text Add to dashboard Cite

show abstract

“…Several examples in testing software-intensive CPSs [5], [6], [18], [19], [27], [28] highlight this realistic aspect. Furthermore, the usability of automatically generated tests may be hindered by test cases that are not realistic (i.e., strange and difficult to comprehend for developers) [29]. Problem statement.…”

Section: Introductionmentioning

confidence: 99%

Automated Generation of Consistent Graph Models With Multiplicity Reasoning

Marussy

Semeráth

Varró

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Advanced tools used in model-based systems engineering (MBSE) frequently represent their models as graphs. In order to test those tools, the automated generation of well-formed (or intentionally malformed) graph models is necessitated which is often carried out by solver-based model generation techniques. In many model generation scenarios, one needs more refined control over the generated unit tests to focus on the more relevant models. Type scopes allow to precisely define the required number of newly generated elements, thus one can avoid the generation of unrealistic and highly symmetric models having only a single type of elements. In this paper, we propose a 3-valued scoped partial modeling formalism, which innovatively extends partial graph models with predicate abstraction and counter abstraction. As a result, well-formedness constraints and multiplicity requirements can be evaluated in an approximated way on incomplete (unfinished) models by using advanced graph query engines with numerical solvers (e.g. IP or LP solvers). Based on the refinement of 3-valued scoped partial models, we propose an efficient model generation algorithm that generates models that are both well-formed and satisfy the scope requirements. We show that the proposed approach scales significantly better than existing SAT-solver techniques or the original graph solver without multiplicity reasoning. We illustrate our approach in a complex design-space exploration case study of collaborating satellites introduced by researchers at NASA Jet Propulsion Lab.

show abstract