Automated Testing and Debugging of SAT and QBF Solvers

Brummayer, Robert; Lonsing, Florian; Biere, Armin

doi:10.1007/978-3-642-14186-7_6

Cited by 63 publications

(53 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One step in this direction is the application of delta-debugging [27] to reduce the input to a simpler form to aid debugging efforts. This approach has been explored for SAT/QBF solvers [5,1] applied to both the input problem and the parameter space.…”

Section: Analysability Of Unsound Proofsmentioning

confidence: 99%

“…A useful area of research would be the automatic generation of problems, or fuzzing of existing problems, to cover such dimensions. In this direction we could borrow from successful results in SAT/QBF solving [5,6].…”

Section: Achieving Better Coverage With Random Testingmentioning

confidence: 99%

“…In SMT-COMP 2016 there were 603 conflicts (solvers returning different results) on 73 benchmarks caused by three solvers giving incorrect results for various reasons. 5 In the CASC competition [25], there is a period of testing where soundness is checked and resolved, and there have been a number of solvers later disqualified from the competition due to unsoundness. In our experience, adding a new feature to a theorem prover is a highly complex task and it is easy to introduce unsoundness, or general incorrectness, especially in areas of the code that are encountered during proof search infrequently.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Testing a Saturation-Based Theorem Prover: Experiences and Challenges

2017

View full text Add to dashboard Cite

Abstract. This paper attempts to address the question of how best to assure the correctness of saturation-based automated theorem provers using our experience with developing the theorem prover Vampire. We describe the techniques we currently employ to ensure that Vampire is correct and use this to motivate future challenges that need to be addressed to make this process more straightforward and to achieve better correctness guarantees.

show abstract

Section: Analysability Of Unsound Proofsmentioning

confidence: 99%

Section: Achieving Better Coverage With Random Testingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Testing a Saturation-Based Theorem Prover: Experiences and Challenges

2017

View full text Add to dashboard Cite

show abstract

“…Fuzzing has been used to test all kinds of software including SAT solvers [10]. Inspired by the utility of fuzzers, we introduce StringFuzz and describe its value as an exploratory testing tool.…”

Section: Introductionmentioning

confidence: 99%

StringFuzz: A Fuzzer for String Solvers

Blotsky

Mora

Berzish

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper, we introduce StringFuzz: a modular SMT-LIB problem instance transformer and generator for string solvers. We supply a repository of instances generated by StringFuzz in SMT-LIB 2.0/2.5 format. We systematically compare Z3str3, CVC4, Z3str2, and Norn on groups of such instances, and identify those that are particularly challenging for some solvers. We briefly explain our observations and show how StringFuzz helped discover causes of performance degradations in Z3str3.

show abstract

“…Clearly, solvers are not immune to bugs [15,16] that may lead to wrong results. Fortunately, there is a simple way to certify the output of such solvers and prevent the potentially dangerous propagation of an error in them.…”

Section: Introductionmentioning

confidence: 99%

Greedy pebbling for proof space compression

Fellner

Paleo

2017

Int J Softw Tools Technol Transfer

View full text Add to dashboard Cite

Automated reasoning tools for the verification and synthesis of software often produce proofs to allow independent certification of the correctness of the produced solutions. As proofs can be large, this paper considers the problem of compressing proofs with respect to their space, which is approximately proportional to the memory necessary to check them. Proof checking with a small amount of available memory is analogous to playing a pebbling game with a small number of pebbles. This paper exploits this analogy and describes novel algorithms for playing a pebbling game. The sequence of moves executed in the pebbling game then corresponds to an improved topological ordering of the nodes of the proof, leading to smaller memory consumption when the proof is checked. Because the number of possible pebbling strategies and topological orderings is too large, brute-force approaches to find optimal solutions are impractical, and hence, the new pebbling algorithms proposed here are based on heuristics for finding good, though not necessarily optimal, solutions. The algorithms are evaluated on the task of compressing the space of thousands of propositional resolution proofs generated by SAT-and SMT-solvers.

show abstract

Automated Testing and Debugging of SAT and QBF Solvers

Cited by 63 publications

References 31 publications

Testing a Saturation-Based Theorem Prover: Experiences and Challenges

Testing a Saturation-Based Theorem Prover: Experiences and Challenges

StringFuzz: A Fuzzer for String Solvers

Greedy pebbling for proof space compression

Contact Info

Product

Resources

About