Tool-assisted unit-test generation and selection based on operational abstractions

Abstract

Abstract. Unit testing, a common step in software development, presents a challenge. When produced manually, unit test suites are often insufficient to identify defects. The main alternative is to use one of a variety of automatic unit-test generation tools: these are able to produce and execute a large number of test inputs that extensively exercise the unit under test. However, without a priori specifications, programmers need to manually verify the outputs of these test executions, which is generally impractical. To reduce this cost, unit-test selection techniques may be used to help select a subset of automatically generated test inputs. Then programmers can verify their outputs, equip them with test oracles, and put them into the existing test suite. In this paper, we present the operational violation approach for unit-test generation and selection, a black-box approach without requiring a priori specifications. The approach dynamically generates operational abstractions from executions of the existing unit test suite. These operational abstractions guide test generation tools to generate tests to violate them. The approach selects those generated tests violating operational abstractions for inspection. These selected tests exercise some new behavior that has not been exercised by the existing tests. We implemented this approach by integrating the use of Daikon (a dynamic invariant detection tool) and Parasoft Jtest (a commercial Java unit testing tool), and conducted several experiments to assess the approach.

Carving and Replaying Differential Unit Test Cases from System Test Cases

“…Some of these tools aim to assess software robustness (e.g., whether an exception is thrown [13]). Others utilize some type of specification such as pre and postconditions or operational abstractions, to guide the test case generation and actually check whether the test outcome meets the expectation results [7], [11], [27], [39]. Interestingly enough, a part of JTest called JTest Tracer can be used to monitor a deployed application in real time and capture inputs to generate realistic JUnit test cases [22], a process somewhat similar to carving.…”

Automatic extraction of abstract-object-state machines from unit-test executions

“…After Rostra generates test inputs and exports them into JUnit [4] test classes, we run these test classes with our previously developed Jusc [23] tool, a Java unit-test selection tool based on residual structural coverage [14], to output a path trace file after program executions terminate. We developed a tool to postprocess the collected path trace file to collect branch coverage information.…”

Alternating GUI Test Generation and Execution

“…They use feedback from the application's run-time state to generate additional test cases, e.g., in the form of outcomes of programmer-supplied predicates in the code to cover all non-isomorphic inputs [2], operational abstractions to cover increased program behaviors [3,16], partially generated non-exception-throwing method-call sequences to generate longer sequences [14], and as input to a fitness function to guide genetic search [5,9].…”