Evaluating Symbolic Execution-Based Test Tools

Cseppento, Lajos; Micskei, Zoltán

doi:10.1109/icst.2015.7102587

Cited by 17 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model based software and systems engineering [114,115,121,122]: techniques and tools for requirements engineering, design, analysis, optimization, deployment, testing [116], maintenance, domain-specific modeling languages, model transformations [123] and code generation). Formal methods [118,119,124] The Intelligent Systems Group is competent and dedicated to research in ambient intelligent systems [127], heterogeneous information processing, data analysis, medical image analysis [126], modeling complex systems [128], bioinformatics, biomedical informatics [125].…”

Section: Main Research Fieldsmentioning

confidence: 99%

BME VIK Annual Research Report on Electrical Engineering and Computer Science 2015

Vajk¹,

Harsányi

Poppe

et al. 2016

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

PrefaceThroughout László Vajta, Dean of BME VIK János Levendovszky, Deputy Dean of BME VIK Department of Automation and Applied InformaticsDepartment of Automation and Applied Informatics (AUT) is one of the largest and dynamically developing departments at BME with diverse scope competences such as control theory, embedded systems, software modelling, applied software development, Internet of Things, power electronics, mechatronics, and many others. The department's main activities are education, research and development. Training versatile electrical and software engineers with solid practical knowledge is our top priority. Our profile also includes developing high quality software and hardware solutions for industry partners. All of our activities are backed by strong research background.In 2015, the department has organized the 21 st European Wireless (EW) Conference. The EW 2015 conference hosted more than 140 participants from 30 different countries. The 2015 edition of EW was aimed at addressing a key theme on "5G and beyond."The next sections summarize the key research results of the department in year 2015. IoT and Smart CityThe Internet of Things (IoT) is transforming the surrounding physical objects into an ecosystem of information that enriches our everyday life. The IoT represents the convergence of advances in miniaturization, wireless connectivity, and increased data storage, and is driven by various sensors. Application and service development methods and frameworks are required to support the realization of solutions covering data collection, transmission, data processing, analysis, reporting, and advanced querying. Our SensorHUB framework URBMOBI is a mobile environmental sensor that (i) provides temporally and spatially distributed environmental data, (ii) fulfills the need for monitoring at various places without the costs for a large number of fixed measurement stations, (iii) integrates small and precise sensors in a system that can be operated on buses, trams, or other vehicles, (iv) concentrates on urban heat and thermal comfort, and (v) aims at providing climate services and integration with real-time climate models.utilizes the state-of-the-art open source technologies and provides a unified tool chain for IoT related application and ser-URBMOBI project has been worked out between 2013 and 2015 by the following consortium: RWTH Aachen University (Germany), Netherlands Organization for Applied Scientific Research TNO (Netherlands), ARIA Technologies (France), Budapest University of Technology and Economics (Hungary), and Meteorological and Environmental Earth Observation (Italy).The SOLSUN (Sustainable Outdoor Lighting & Sensory Urban Networks) project is about to demonstrate how intelligent city infrastructure can be created in a cost-effective and sustainable way by reusing existing street lighting as the communications backbone. We apply different technologies and methods to reduce energy consumption at the same time as turning streetlights into nodes on a scalable network that is al...

show abstract

Section: Main Research Fieldsmentioning

confidence: 99%

BME VIK Annual Research Report on Electrical Engineering and Computer Science 2015

Vajk¹,

Harsányi

Poppe

et al. 2016

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

show abstract

“…Our initial paper concentrated on tools using symbolic execution. This paper extends the scope of the work by including search‐based and random testing; extending the features with handling environment, reflection, multi‐threading, and native code; adding a new tool and a previously manually evaluated tool to the automatic evaluation framework; performing a large number of experiments; and extending the analysis from only code coverage to several other properties.…”

Section: Introductionmentioning

confidence: 99%

Evaluating code‐based test input generator tools

Cseppento

Micskei

2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary In recent years, several tools have been developed to automatically select test inputs from the code of the system under test. However, each of these tools has different advantages, and there is a little detailed feedback available on the actual capabilities of the various tools. To evaluate test input generators, this paper collects a set of programming language concepts that should be handled by the tools and maps these core concepts and challenging features like handling the environment or multi‐threading to 363 code snippets, respectively. These snippets would serve as inputs for the tools. Next, the paper presents SETTE, an automated framework to execute and evaluate these snippets. Using SETTE, multiple experiments were performed on five Java and one .NET‐based tools using symbolic execution, search‐based, and random techniques. The test suites' coverage, size, generation time, and mutation score were compared. The results highlight the strengths and weaknesses of each tool and approach and identify hard code parts that are difficult to tackle for most of the tools. We hope that this research could serve as actionable feedback to tool developers and help practitioners assess the readiness of test input generation.

show abstract

“…The resulting test cases may thus set the input objects to states not reachable through the programming interfaces [17], and may result in test cases not yielding feasible behaviours of the program. For example, the test code of Figure 4 assigns the numeric value 7 to list.size (line 9), which is a correct solution for the constraint List.size>3 in the path condition, but violates the invariant of the current list structure that contains only 4 nodes (excluding the sentinel node list.header).…”

Section: Building Heapmentioning

confidence: 99%

“…Symbolic techniques, like JPF [53], jPET [1], JBSE [10,11], and BLISS [48], identify input states to execute code elements, but rely on direct heap manipulation to instantiate the data structures that comprise these input states. Test cases that instantiate the input data structures by directly manipulating the heap memory may either not compile, since they directly access private fields, or create unreachable program states, since they bypass the program functions that preserve the invariants [17], and are often hardly readable and understandable by developers [18].…”

Section: Introductionmentioning

confidence: 99%

Combining symbolic execution and search-based testing for programs with complex heap inputs

Braione

Denaro

Mattavelli

et al. 2017

Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Despite the recent improvements in automatic test case generation, handling complex data structures as test inputs is still an open problem. Search-based approaches can generate sequences of method calls that instantiate structured inputs to exercise a relevant portion of the code, but fall short in building inputs to execute program elements whose reachability is determined by the structural features of the input structures themselves. Symbolic execution techniques can effectively handle structured inputs, but do not identify the sequences of method calls that instantiate the input structures through legal interfaces. In this paper, we propose a new approach to automatically generate test cases for programs with complex data structures as inputs. We use symbolic execution to generate path conditions that characterise the dependencies between the program paths and the input structures, and convert the path conditions to optimisation problems that we solve with search-based techniques to produce sequences of method calls that instantiate those inputs. Our preliminary results show that the approach is indeed effective in generating test cases for programs with complex data structures as inputs, thus opening a promising research direction. CCS CONCEPTS• Software and its engineering → Software testing and debugging; Formal software verification; KEYWORDSAutomatic test case generation, Symbolic execution, Search-based software engineering ACM Reference format:

show abstract

Evaluating Symbolic Execution-Based Test Tools

Cited by 17 publications

References 18 publications

BME VIK Annual Research Report on Electrical Engineering and Computer Science 2015

BME VIK Annual Research Report on Electrical Engineering and Computer Science 2015

Evaluating code‐based test input generator tools

Combining symbolic execution and search-based testing for programs with complex heap inputs

Contact Info

Product

Resources

About