Program Checking with Less Hassle

Abstract

Abstract. The simple and often imprecise specifications that programmers may write are a significant limit to a wider application of rigorous program verification techniques. Part of the reason why non-specialists find writing good specification hard is that, when verification fails, they receive little guidance as to what the causes might be, such as implementation errors or inaccurate specifications. To address these limitations, this paper presents two-step verification, a technique that combines implicit specifications, inlining, and loop unrolling to provide improved user feedback when verification fails. Two-step verification performs two independent verification attempts for each program element: one using standard modular reasoning, and another one after inlining and unrolling; comparing the outcomes of the two steps suggests which elements should be improved. Two-step verification is implemented in AutoProof, our static verifier for Eiffel programs integrated in EVE (the Eiffel Verification Environment) and available online. The Trouble with SpecsThere was a time when formal verification required heroic efforts and was the exclusive domain of a small group of visionaries. That time is now over; but formal techniques still seem a long way from becoming commonplace. If formal verification techniques are to become a standard part of the software development process-and they are-we have to understand and remove the obstacles that still prevent non-specialists from using them.A crucial issue is specification. Program correctness is a relative notion, in that a program is correct not in absolute terms but only relative to a given specification of its expected behavior; in other words, verified programs are only as good as their specification. Unfortunately, many programmers are averse to writing specifications, especially formal ones, for a variety of reasons that mostly boil down a benefit-to-effort ratio perceived as too low. Writing formal specifications may require specialized skills and experience; and the concrete benefits are dubious in environments that value productivity, assessed through simple quantitative measures, more than quality. Why should programmers subject themselves to the taxing exercise of writing specifications in addition to implementations, if there is not much in it for them other than duplicated work?

Your Proof Fails? Testing Helps to Find the Reason

“…RELATED WORK Understanding proof failures. A two-step verification in [15] compares the proof failures of an Eiffel program with those of its variant where called functions are inlined and loops are unrolled. It reports code and contract revision suggestions from this comparison.…”

AutoProof: Auto-Active Functional Verification of Object-Oriented Programs

“…Dealing with inconclusive error reports in incomplete tools is a practical hurdle to usability that can spoil user experience-especially for novices. To improve user feedback in case of failed verification attempts, AutoProof implements a collection of heuristics known as "two-step verification" [35]. When they are enabled, each failed verification attempt is transparently followed by a second step that is in general unsound (as it uses under-approximations such as loop unrolling) but helps discern whether failed verification is due to real errors or just to insufficiently detailed annotations.…”

To Run What No One Has Run Before: Executing an Intermediate Verification Language

“…2 already mentioned the Boogie Verification Debugger (BVD, [14]); the Spec# debugger [21] implements similar functionalities which construct concrete counterexamples from failed Boogie runs. Two-step verification [27] compares verification with different semantics (based on unrolling and inlining) to attribute verification failures to either inconsistent or incomplete specifications.…”