The CADE ATP System Competition — CASC

Abstract

The CADE ATP System Computer (CASC) evaluates the performance of sound, fully automatic, classical logic, ATP systems. The evaluation is in terms of the number of problems solved, the number of acceptable proofs and models produced, and the average runtime for problems solved, in the context of a bounded number of eligible problems chosen from the TPTP problem library, and a specified time limit for each solution attempt. The CADE-22 ATP System Competition (CASC-22) was held on 5th August 2009. The design of the competition and it's rules, and information regarding the competing systems, are provided in this report.

Testing a Saturation-Based Theorem Prover: Experiences and Challenges

“…In SMT-COMP 2016 there were 603 conflicts (solvers returning different results) on 73 benchmarks caused by three solvers giving incorrect results for various reasons. 5 In the CASC competition [25], there is a period of testing where soundness is checked and resolved, and there have been a number of solvers later disqualified from the competition due to unsoundness. In our experience, adding a new feature to a theorem prover is a highly complex task and it is easy to introduce unsoundness, or general incorrectness, especially in areas of the code that are encountered during proof search infrequently.…”

Do Portfolio Solvers Harm?

“…[13], often combined with an SS-portfolio approach. Leading competition versions of solvers for the "main" divisions of the first-order logic theorem proving competition CASC [27] namely E [24], iProver [16] and Vampire [18] are all SS-portfolio solver instances. E subsequently runs several different superposition strategies found by a machine learning approach.…”

Online Runtime Verification Competitions: How To Possibly Deal With Their Issues (position paper)

“…The SAT [1], SMT-COMP [2] and CASC [21] competitions respectively focus on comparing SAT solvers, SMT solvers and automated theorem provers (ATPs). Each of them takes benefit of uniform common formats supported by every contestant tool (e.g.…”