Competition on Software Verification and Witness Validation: SV-COMP 2023

Abstract

The 12th edition of the Competition on Software Verification (SV-COMP 2023) is again the largest overview of tools for software verification, evaluating 52 verification systems from 34 teams from 10 countries. Besides providing an overview of the state of the art in automatic software verification, the goal of the competition is to establish standards, provide a platform for exchange to developers of such tools, educate PhD students on reproducibility approaches and benchmarking, and provide computing resources to developers that do not have access to compute clusters. The competition consisted of 23 805 verification tasks for C programs and 586 verification tasks for Java programs. The specifications include reachability, memory safety, overflows, and termination. This year, the competition introduced a new competition track on witness validation, where validators for verification witnesses are evaluated with respect to their quality.

Korn—Software Verification with Horn Clauses (Competition Contribution)

“…The random sampler is very effective-in SV-COMP 2023 it discovered all 210 violations reported by Korn, of which 204 are found within 2 seconds. Sampling of small non-zero values is crucial, e.g., Ackermann02.c falsifies with input vector [2,0]; using all zero inputs still finds 57 of these 210 violations.…”

“…Verification conditions are generated in the fragment of SMT-LIB of the HORN logic. 2 Korn can invoke any compliant solver as a backend either using its standard input or a file to communicate the verification task. There is explicit support for Z3 [12], Eldarica [15] to pass e.g.…”

“…The tool archive packaged for SV-COMP 2023 is part of the official tools artifact [4] and also available separately [9]. The official competition results [3] are complemented with our post-competition evaluation, based on commit 92e6732 and are available at [8].…”

CEGAR-PT: A Tool for Abstraction by Program Transformation

“…2) Verifier: Calls to a software verifier are encapsulated using COVERITEAM [18], which provides a common interface for using off-the-shelf verifiers. It allows the usage of at least all 45 verifiers for C that participated in the competition on software verification [19]. The verdict that the verifier produces corresponds to the transformed program P .…”

Symbiotic-Witch 2: More Efficient Algorithm and Witness Refutation

“…Data Availability Statement. All data of SV-COMP 2023 are archived as described in the competition report [3] and available on the competition web site. This includes the verification tasks, results, witnesses, scripts, and instructions for reproduction.…”