Instrumenting a weakest precondition calculus for counterexample generation

Abstract

A major issue in the activity of deductive program verification is to understand why automated provers fail to discharge a proof obligation. To help the user understand the problem and decide what needs to be fixed in the code or the specification, it is essential to provide means to investigate such a failure. We present our approach for the design and the implementation of counterexample generation, exhibiting values for the variables of the program where a given part of the specification fails to be validated. To produce a counterexample, we exploit the ability of SMT solvers to propose, when a proof of a formula is not found, a counter-model. Turning such a counter-model into a counterexample for the initial program is not trivial because of the many transformations leading from a particular piece of code and its specification to a set of proof goals given to external provers. using automated theorem provers, in particular those of the Satisfiability Modulo Theories (SMT) family such as Alt-Ergo [5], CVC4 [6] and Z3 [7]. These theorem provers are used as black-boxes that, given a VC, may produce three kinds of results:1. The prover answers something meaning "yes, the VC is valid". 2. The prover answers anything else, meaning "I don't know", in other words the prover is not able to prove the VC for some reason. 3. The prover runs for too long a time (seemingly infinitely) or runs out of memory.The case where the prover runs for too long a time is handled in practice by setting a given time limit, so that the prover process is killed when exceeding this limit. The cases 2 and 3 are the same from the user's perspective: the VC is not proved. Note that we do not distinguish a case where the prover would answer "no it is not valid", because the VCs typically involve undecidable logic features (e.g. non-linear integer arithmetic, first-order quantification) so provers are in practice incomplete: there is no way for them to be sure that a given VC is not provable. A major issue in the activity of deductive verification is thus understanding the reasons for a proof failure. There are various reasons why it may fail:

Automated Verification of Temporal Properties of Ladder Programs

“…A detailed introduction to Why3 is given in our extended report [4]. Among the recent features of Why3 of particular interest for our work are the ability to generate loop invariants and to produce counterexamples when a proof fails [8]. Indeed, the first of these features had to be improved in order to support our work on Ladder programs, this is a contribution that some of us made to Why3 [4].…”

How the Analyzer can Help the User Help the Analyzer

“…Ideally, a counterexample captures in an understandable executable trace why a property cannot be proved, by exhibiting a consistent example where the property does not hold. In reality, after multiple person-year efforts to develop and improve counterexamples in the context of SPARK [7], there is still much to be desired here.…”

CISE3: Verificação de aplicações com consistência fraca em Why3

“…Em adição, a programadora ao especificar o estado da aplicação juntamente com os seus invariantes de integridade, o Why3 consegue verificar se uma operação os pode quebrar ou não, dada a sua implementação. Nos casos em que o Why3 não consiga provar alguma asserção no programa, a plataforma é capaz de apresentar um contraexemplo [4]. Dito isto, é possível afirmar que o Why3 é capaz de realizar a análise de safety, sem ser necessário realizar qualquer alteração interna à plataforma.…”