POPLMark reloaded: Mechanizing proofs by logical relations

Abstract

We propose a new collection of benchmark problems in mechanizing the metatheory of programming languages, in order to compare and push the state of the art of proof assistants. In particular, we focus on proofs using logical relations (LRs) and propose establishing strong normalization of a simply typed calculus with a proof by Kripke-style LRs as a benchmark. We give a modern view of this well-understood problem by formulating our LR on well-typed terms. Using this case study, we share some of the lessons learned tackling this problem in different dependently typed proof environments. In particular, we consider the mechanization in Beluga, a proof environment that supports higher-order abstract syntax encodings and contrast it to the development and strategies used in general-purpose proof assistants such as Coq and Agda. The goal of this paper is to engage the community in discussions on what support in proof environments is needed to truly bring mechanized metatheory to the masses and engage said community in the crafting of future benchmarks.

Formalization of metatheory of the Lambda Calculus in constructive type theory using the Barendregt variable convention

“…Further work in the line of the present paper should precisely address the issue of serious comparison between the various competing notations at hand. Such work has been initiated in Berghofer and Urban (2007) and Abel et al (2019). We think that our approaches to named, classical syntax are worth being considered among the serious candidates.…”

Formal metatheory of second-order abstract syntax

“…Benchmarks. The PoplMaRK challenge [Aydemir et al 2005;Abel et al 2019] sets out a collection of criteria according to which metatheory-formalisation efforts can be compared. Several submissions use Coq as the target language, in some cases involving code generation from a second-order signature [Aydemir et al 2008;Vouillon 2011;Lee et al 2012;Polonowski 2013;Keuchel et al 2016;.…”

Harpoon: Mechanizing Metatheory Interactively

“…STLC strong normalization [1] Larger development (310 commands), all forms of case analysis as above.…”

“…We have used Harpoon (see https://beluga-lang.readthedocs.io/) on a wide range of representative examples from the Beluga library: normalization proofs for the simply-typed lambda calculus [6], benchmarks for reasoning about binders [9,10], and the recent POPLMark Reloaded challenge [1]. These examples involve numerous concerns that arise in proof development, and cover all the domainspecific abstractions that Beluga provides.…”