Data sampling through collaborative filtering for algorithm selection

Benchmarking is a crucial phase when developing algorithms. This also applies to solvers for the SAT (propositional satisfiability) problem. Benchmark selection is about choosing representative problem instances that reliably discriminate solvers based on their runtime. In this paper, we present a dynamic benchmark selection approach based on active learning. Our approach predicts the rank of a new solver among its competitors with minimum runtime and maximum rank prediction accuracy. We evaluated this approach on the Anniversary Track dataset from the 2022 SAT Competition. Our selection approach can predict the rank of a new solver after about 10 % of the time it would take to run the solver on all instances of this dataset, with a prediction accuracy of about 92 %. We also discuss the importance of instance families in the selection process. Overall, our tool provides a reliable way for solver engineers to determine a new solver’s performance efficiently.

show abstract

Data sampling through collaborative filtering for algorithm selection

Cited by 8 publications

References 32 publications

Active Matrix Completion for Algorithm Selection

Active Matrix Completion for Algorithm Selection

Algorithm Selection on Adaptive Operator Selection: A Case Study on Genetic Algorithms

Active Learning for SAT Solver Benchmarking

Contact Info

Product

Resources

About