Exploration of language specifications by compilation to first-order logic

Abstract

Exploration of language specifications helps to discover errors and inconsistencies early during the development of a programming language. We propose exploration of language specifications via application of existing automated first-order theorem provers (ATPs). To this end, we translate language specifications and exploration tasks to first-order logic, which many ATPs accept as input. However, there are several different strategies for compiling a language specification to first-order logic, and even small variations in the translation may have a large impact on the time it takes ATPs to find proofs. In this paper, we first present a systematic empirical study on how to best compile language specifications to first-order logic such that existing ATPs can solve typical exploration tasks efficiently. We have developed a compiler product line that implements 36 different compilation strategies and used it to feed language specifications to 4 existing first-order theorem provers. As benchmarks, we developed language specifications for typed SQL and for a Questionnaire Language (QL), with 50 exploration goals each. Our study empirically confirms that the choice of a compilation strategy greatly influences prover performance in general and shows which strategies are advantageous for prover performance. Second, we extend our empirical study with 4 domain-specific strategies for axiom selection and find that axiom selection does not influence prover performance in our benchmark specifications.

Automating Proof Steps of Progress Proofs: Comparing Vampire and Dafny

“…The full code of our example and the implementation of Veritas is available at https://github.com/stg-tud/type-pragmatics/ tree/master/Veritas. For further details on our current encoding scheme and the specification of typed SQL, please refer to our recent paper "Exploration of Language Specifications by Compilation to First-order Logic" [3]. We model underspecified types via the keyword "open data", which we introduced into SPL since our presentation at the 2nd Vampire Workshop [5].…”

Using Vampire with Support for Algebraic Datatypes in Type Soundness Proofs

“…This embedded DSL allows for specifying the syntax of a language via closed algebraic datatypes, the dynamic semantics (reduction semantics) of the language via simple recursive, immutable functions, and the static semantics (type system) as well as properties to prove via typing rules and formulas. In a previous publication, we thoroughly describe our input format and different variants of how we translate the format to untyped and typed first-order logic ( [5]), where we encode algebraic datatypes ourselves by generating axioms which describe the respective term algebras.…”

“…SQL test specification For the present work, we use a type system specification described in one of our previous publications [5]: The language we model is a small subset of SQL, consisting of basic table values, simple select ... from ... where ... queries without joins and Cartesian products (i.e. combinations of table selection (of table rows) and projection (of table columns)), and queries for forming the union, intersection, and difference of two tables with the same table schema.…”

“…Previous versions of VeriTaS translated a core specification language with algebraic datatypes and recursive functions to TPTP [11], encoding algebraic datatypes by axiomatizing term algebras. In previous work, we describe the details of this translation [5]. Our case studies so far showed that Vampire [8] is well able to prove individual proof steps from progress and preservation proofs automatically [4], as well as proof goals for exploring language specifications [5], such as goals that test the concrete behavior of a type system for specific programs.…”

“…In previous work, we describe the details of this translation [5]. Our case studies so far showed that Vampire [8] is well able to prove individual proof steps from progress and preservation proofs automatically [4], as well as proof goals for exploring language specifications [5], such as goals that test the concrete behavior of a type system for specific programs. On the other hand, we also frequently ran into seemingly trivial proof problems that previous Vampire versions could not prove, even with a high timeout.…”