An empirical study to quantify the characteristics of Java programs that may influence symbolic execution from a unit testing perspective

Eler, Marcelo Medeiros; Endo, André Takeshi; Durelli, Vinícius H. S.

doi:10.1016/j.jss.2016.03.020

Cited by 7 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Challenges However, like other hard problems, the practical application of test input generation faces also several challenges . Common important ones to deal with include path explosion (exponentially increasing number of possible execution paths), complex and external arithmetic functions, floating‐point calculations, pointer operations, interaction with the environment, and multi‐threading ; predicates with only a flag variable, nested conditions and enumerations ; complex strings and objects with internal state ; nonfunctional requirements ; exception‐dependent paths ; and calling native code or interacting with nonnative libraries.…”

Section: Overviewmentioning

confidence: 99%

“…Most of these 32 features (e.g., nested or anonymous classes and inheritance or conditional statements) are included in the current work; only assertions and volatile fields are missing. Similarly, Eler et al analyzed how frequent loops, different basic types (integers, arrays, objects…), external calls, and exceptions are in 147 open source Java projects, which are all included in the selected features. However, it should be noted that frequency does not imply importance.…”

Section: Features To Comparementioning

confidence: 99%

“…Shamshiri et al performed experiments with Randoop , EvoSuite , and AgitarOne on real faults from the Defects4J dataset. Eler et al measured how frequent are some of the challenging features in 147 open source Java projects.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Evaluating code‐based test input generator tools

Cseppento

Micskei

2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary In recent years, several tools have been developed to automatically select test inputs from the code of the system under test. However, each of these tools has different advantages, and there is a little detailed feedback available on the actual capabilities of the various tools. To evaluate test input generators, this paper collects a set of programming language concepts that should be handled by the tools and maps these core concepts and challenging features like handling the environment or multi‐threading to 363 code snippets, respectively. These snippets would serve as inputs for the tools. Next, the paper presents SETTE, an automated framework to execute and evaluate these snippets. Using SETTE, multiple experiments were performed on five Java and one .NET‐based tools using symbolic execution, search‐based, and random techniques. The test suites' coverage, size, generation time, and mutation score were compared. The results highlight the strengths and weaknesses of each tool and approach and identify hard code parts that are difficult to tackle for most of the tools. We hope that this research could serve as actionable feedback to tool developers and help practitioners assess the readiness of test input generation.

show abstract