EvoSpex: An Evolutionary Algorithm for Learning Postconditions

Abstract

Software reliability is a primary concern in the construction of software, and thus a fundamental component in the definition of software quality. Analyzing software reliability requires a specification of the intended behavior of the software under analysis, and at the source code level, such specifications typically take the form of assertions. Unfortunately, software many times lacks such specifications, or only provides them for scenario-specific behaviors, as assertions accompanying tests. This issue seriously diminishes the analyzability of software with respect to its reliability.In this paper, we tackle this problem by proposing a technique that, given a Java method, automatically produces a specification of the method's current behavior, in the form of postcondition assertions. This mechanism is based on generating executions of the method under analysis to obtain valid pre/post state pairs, mutating these pairs to obtain (allegedly) invalid ones, and then using a genetic algorithm to produce an assertion that is satisfied by the valid pre/post pairs, while leaving out the invalid ones. The technique, which targets in particular methods of referencebased class implementations, is assessed on a benchmark of open source Java projects, showing that our genetic algorithm is able to generate post-conditions that are stronger and more accurate, than those generated by related automated approaches, as evaluated by an automated oracle assessment tool. Moreover, our technique is also able to infer an important part of manually written rich postconditions in verified classes, and reproduce contracts for methods whose class implementations were automatically synthesized from specifications. I .

Fuzzing Class Specifications

“…Regarding GAssert and EvoSpex, we followed the same methodology described in their papers [31,38], using exactly the same configuration parameters for the evolutionary processes of each technique. Moreover, to account for the randomness of each approach, for each of the 43 Java methods we inferred postconditions with each tool a total of 10 times.…”

“…GAssert and EvoSpex. GAssert [38] and EvoSpex [31] are recently proposed specification inference techniques. As Daikon, these tools execute a test suite of the program under analysis and observe executions to infer specifications that are consistent with the observations.…”

“…It is then limited to simple assertions (e.g., no direct support for quantification), or requires the developer to manually extend the assertion language. GAssert [38] and EvoSpex [31], two recently proposed techniques for contract inference, try to address this limitation of Daikon by supporting more expressive assertion languages, but their extensions focus on specific kinds of constraints: GAssert focuses on logical/arithmetic constraints (no quantified expressions) and EvoSpex focuses on object navigation constraints (only very simple logical and arithmetic operators are supported). Moreover, as both techniques are based on evolutionary search, they are difficult to extend or adapt to support further expressions, as the evolutionary algorithms are targeted for the specific languages supported by the corresponding tools.…”

“…Techniques for inferring class specifications exist [7,14,31,38], but their expressiveness is limited. Daikon [14], the baseline that other techniques use, supports a restricted set of templates, from which assertions are generated.…”

Synthesizing contracts correct modulo a test generator

“…The work by Jahangirova et al [2016] also uses mutation testing to infer assertions, and iterates over rounds with the programmer. The work by Molina et al [2021] similarly infers program postconditions using an evolutionary algorithm to improve postconditions over generations. It mutates valid input-output pairs derived from test generation to produce potential invalid pairs, albeit in an unsound way.…”