Automating Proof Steps of Progress Proofs: Comparing Vampire and Dafny

Abstract

Developing provably sound type systems is a non-trivial task which, as of today, typically requires expert skills in formal methods and a considerable amount of time. Our Veritas [4] project aims at providing support for the development of soundness proofs of type systems and efficient type checker implementations from specifications of type systems. To this end, we investigate how to best automate typical steps within type soundness proofs. In this paper, we focus on progress proofs for type systems of domain-specific languages. As a running example for such a type system, we model a subset SQL and augment it with a type system. We compare two different approaches for automating proof steps of the progress proofs for this type system against each other: firstly, our own tool Veritas, which translates proof goals and specifications automatically to TPTP [14] and calls Vampire [9] on them, and secondly, the programming language Dafny [7], which translates proof goals and specifications to the intermediate verification language Boogie 2 [6] and calls the SMT solver Z3 [10] on them. We find that Vampire and Dafny are equally well-suited for automatically proving simple steps within progress proofs.

Extrinsically typed operational semantics for functional languages

“…The seminal work of Schürmann and Pfenning shows that the type soundness of nontrivial functional languages can be automatically established in the LF-based theorem prover Twelf [Pfenning and Schürmann 1999;Schürmann 2000]. The Veritas tool automates the verification of the type soundness of languages by compiling them into a first order theorem prover, and by checking that suitable formulae hold [Grewe 2019;Grewe et al 2016Grewe et al , 2017Grewe et al , 2015. The way our meta type system checks type preservation stems directly from these works, but our approach to progress is different.…”

Using Vampire with Support for Algebraic Datatypes in Type Soundness Proofs

“…the remaining set cases SQL-Progress-icase2-case1, SQL-Progress-icase2-case2 etc.) were proven by Vampire 3.0 [4]). The most likely explanation here is that the casc mode of Vampire 3.0 contained a different strategy portfolio than the one of Vampire 4.1.…”

“…Intuitively, progress says that if a query can be typed, it is either already a table value (with a well-typed table), or the reduction semantics can reduce the query at least one step further without getting stuck. We described the property and the progress proof in a previous publication at the Vampire workshop [4].…”

“…In previous work, we describe the details of this translation [5]. Our case studies so far showed that Vampire [8] is well able to prove individual proof steps from progress and preservation proofs automatically [4], as well as proof goals for exploring language specifications [5], such as goals that test the concrete behavior of a type system for specific programs. On the other hand, we also frequently ran into seemingly trivial proof problems that previous Vampire versions could not prove, even with a high timeout.…”