Using ACL2 To Teach Students About Software Testing

Abstract

We report on our experience using ACL2 in the classroom to teach students about software testing. The course COSC 2300 at the University of Wyoming is a mostly traditional Discrete Mathematics course, but with a clear focus on computer science applications. For instance, the section on logic and proofs is motivated by the desire to write proofs about computer software. We emphasize that the importance of software correctness falls along a spectrum with casual programs on one end and mission-critical ones on the other. Corresponding to this spectrum is a variety of tools, ranging from unit tests, randomized testing of properties, and even formal proofs. In this paper, we describe one of the major activities, in which students use the ACL2 Sedan's counter-example generation facility to investigate properties of various existing checksum algorithms used in error detection. Students are challenged to state the relevant properties correctly, so that the counter-example generation tool is used effectively in all cases, and ACL2 can find formal proofs automatically in some of those. IntroductionAt the University of Wyoming, we have been teaching a Discrete Structures course (COSC 2300) every semester since the fall of 2016. The course covers all the typical topics from Discrete Mathematics, but with an emphasis on Computer Science applications to make it more relevant to CS majors, who make up the vast majority of students. The course is also heavily influenced by an Honors Course for non-majors developed at the University of Oklahoma by Rex Page [11,12]. One of those influences is the use of ACL2 to achieve some of the course learning outcomes.To be clear, competence in ACL2 is not one of those learning outcomes. Instead, students are expected to master the usual topics in discrete mathematics: basic proof techniques, induction, inductive data structures (e.g., lists and trees), foundational mathematical structures (e.g., sets, relations, functions), modular arithmetic, and basic combinatorics. What ACL2 brings to the table is in connecting the abstract concepts of discrete mathematics with the more practical aspects of software development.Again to be clear, we make no attempt to teach software verification in a sophomore-level course. Rather, we emphasize that there is a spectrum of reliability needs for software. On one end are simple programs that few people genuinely care about, like simple iPhone games. On the other end are missionand even life-critical software, like aerospace, automotive, or health applications. So as software developers, we have a wide range of tools that can be used to improve the reliability of our software.At the lower end stand unit test cases, which are widely acknowledged to be a best practice in software development. Students learn about unit tests in our class, and also in other classes as they progress through their degree. In ACL2, we expose unit tests through the macro check-expect, a common testing framework that allows the programmer to provide an expression and its expected retur...