Towards random and enumerative testing for OCaml and WhyML properties

Abstract

Deductive program verication greatly improves software quality, but proving formal specications is dicult, and this activity can only be partially automated. It is therefore relevant to supplement deductive verication tools, such as Why3, with the ability to test the properties to be veried. We present a methodological study and a prototype for the random and enumerative testing of properties written either in the Why3 input language WhyML or in the OCaml programming language used by Why3 to run programs written in WhyML. An originality is that we propose enumerative testing based on data generators themselves written in WhyML and formally veried with Why3. Another specicity is that the development eort is reduced by exploiting Why3's extraction mechanism to OCaml and an existing random testing tool for OCaml. These design choices are applied in a propotypal implementation of a tool, called AutoCheck. The prototype and the paper are designed with simplicity and usability in mind, in order to make them accessible to the widest audience. Starting from the most elementary cases, a tutorial illustrates the implemented features with many examples presented in increasing complexity order.