Verifying an Effect-Handler-Based Define-By-Run Reverse-Mode AD Library

Abstract

By exploiting a number of relatively subtle programming language features, including dynamically-allocated mutable state, first-class functions, and effect handlers, reverse-mode automatic differentiation can be implemented as a library. One outstanding question, however, is: with which logical tools can one specify what this code is expected to compute and verify that it behaves as expected? We answer this question by using a modern variant of Separation Logic to specify and verify a minimalist (but concise and elegant) reverse-mode automatic differentiation library. We view this result as an advanced exercise in program verification, with potential future applications to more realistic automatic differentiation systems.

Dual-Numbers Reverse AD, Efficiently

“…Approaches utilising non-local control flow. Another category of approaches to AD recently taken by the PL community are those that rely on forms of non-local control flow such as delimited continuations [Wang and Rompf 2018] or effect handlers [de Vilhena and Pottier 2021;Sigal 2021]. These techniques are different in the sense that they generate code that is not purely functional.…”