The search for proof and the search for counterexamples (bugs) are complementary activities that need to be pursued concurrently in order to maximize the practical success rate of verification tools. While this is well-understood in safety verification, the current focus of liveness verification has been almost exclusively on the search for termination proofs. A counterexample to termination is an infinite program execution. In this paper, we propose a method to search for such counterexamples. The search proceeds in two phases. We first dynamically enumerate lasso-shaped candidate paths for counterexamples, and then statically prove their feasibility. We illustrate the utility of our nontermination prover, called TNT, on several nontrivial examples, some of which require bitlevel reasoning about integer representations.

No abstract

We present CESI, an algorithm that combines exhaustive enumeration of test inputs from a structured domain with symbolic execution driven test generation. CESI is a hybrid of two predominant techniques: specification-based enumerative test generation (which exhaustively generates all possible inputs satisfying some constraint) and symbolic directed test generation (which explores program paths based on symbolic path constraint solving). We target programs whose valid inputs are determined by some context free grammar. We introduce symbolic grammars, where the original tokens are replaced with symbolic constants, that link enumerative grammar-based input generation with symbolic directed testing. Symbolic grammars abstract the concrete input syntax, thus reducing the set of input strings that must be enumerated exhaustively. For each enumerated input string, which may contain symbolic constants, symbolic execution based test generation instantiates the constants based on program execution paths. The "template" generated by enumerating valid strings reduces the burden on the symbolic execution to generate syntactically valid inputs and hence exercise interesting code paths. Together, symbolic grammars provide a link between exhaustive enumeration of valid inputs and execution-directed symbolic test generation. In preliminary experiments, CESI is better than if both enumerative and symbolic techniques are used alone.

The search for proof and the search for counterexamples (bugs) are complementary activities that need to be pursued concurrently in order to maximize the practical success rate of verification tools. While this is well-understood in safety verification, the current focus of liveness verification has been almost exclusively on the search for termination proofs. A counterexample to termination is an infinite program execution. In this paper, we propose a method to search for such counterexamples. The search proceeds in two phases. We first dynamically enumerate lasso-shaped candidate paths for counterexamples, and then statically prove their feasibility. We illustrate the utility of our nontermination prover, called TNT, on several nontrivial examples, some of which require bitlevel reasoning about integer representations.

Abstract. Automatic symbolic techniques to generate test inputs, for example, through concolic execution, suffer from path explosion: the number of paths to be symbolically solved for grows exponentially with the number of inputs. In many applications though, the inputs can be partitioned into "non-interfering" blocks such that symbolically solving for each input block while keeping all other blocks fixed to concrete values can find the same set of assertion violations as symbolically solving for the entire input. This can greatly reduce the number of paths to be solved (in the best case, from exponentially many to linearly many in the number of inputs). We present an algorithm that combines test input generation by concolic execution with dynamic computation and maintenance of information flow between inputs. Our algorithm iteratively constructs a partition of the inputs, starting with the finest (all inputs separate) and merging blocks if a dependency is detected between variables in distinct input blocks during test generation. Instead of exploring all paths of the program, our algorithm separately explores paths for each block (while fixing variables in other blocks to random values). In the end, the algorithm outputs an input partition and a set of test inputs such that (a) inputs in different blocks do not have any dependencies between them, and (b) the set of tests provides equivalent coverage with respect to finding assertion violations as full concolic execution. We have implemented our algorithm in the Splat test generation tool. We demonstrate that our reduction is effective by generating tests for four examples in packet processing and operating system code.