Symbolic execution for software testing in practice

Abstract

We present results for the "Impact Project Focus Area" on the topic of symbolic execution as used in software testing. Symbolic execution is a program analysis technique introduced in the 70s that has received renewed interest in recent years, due to algorithmic advances and increased availability of computational power and constraint solving technology. We review classical symbolic execution and some modern extensions such as generalized symbolic execution and dynamic test generation. We also give a preliminary assessment of the use in academia, research labs, and industry.

make test-zesti: A symbolic execution solution for improving regression testing

“…Symbolic execution is a technique that has received much attention in recent years in the context of software testing [10] due to its ability to automatically explore multiple program paths and reason about the program's behaviour along each of them. Tools such as KLEE [9], SAGE [17], JPF-SE [1], BitBlaze [33] and Pex [37] are just some of the symbolic execution engines currently used successfully in academia and in industry.…”

Goal-oriented dynamic test generation

“…In particular, we explore innovative techniques to improve automated test input generation approaches [29], [121] to strengthen structural coverage testing, which has long been advocated by the software industry to assess test adequacy [20], [56], [117], [132] but is still limited, even with the recent development of advanced techniques [105], [128], [138], [142]. We then enhance security vulnerability detection techniques [64], [93] by integrating static and dynamic program analysis techniques to actively search for security vulnerability defects.…”

Relational symbolic execution of SQL code for unit testing of database programs

“…While different approaches exist towards the automatic generation of test data, symbolic execution [5] has been recognised as a state-of-art technique for so-called white-box structural test data generation [6,7,8,9,10]. In such an approach, the idea is to generate test data that in some way cover a sufficiently large part of the control-flow graph of the code unit under test [4].…”

“…This test generation process is repeated until test data have been generated for a sufficiently large and diverse number of paths through the control-flow graph, according to some coverage criterion [4]. Test data generation based on symbolic execution is now at the core of various popular open-source and commercial testing tools, some being used in an industrial context, notably at Microsoft and NASA [9].…”