Instance Generation via Generator Instances

Abstract

Access to good benchmark instances is always desirable when developing new algorithms, new constraint models, or when comparing existing ones. Handwritten instances are of limited utility and are timeconsuming to produce. A common method for generating instances is constructing special purpose programs for each class of problems. This can be better than manually producing instances, but developing such instance generators also has drawbacks. In this paper, we present a method for generating graded instances completely automatically starting from a class-level problem specification. A graded instance in our present setting is one which is neither too easy nor too difficult for a given solver. We start from an abstract problem specification written in the Essence language and provide a system to transform the problem specification, via automated type-specific rewriting rules, into a new abstract specification which we call a generator specification. The generator specification is itself parameterised by a number of integer parameters; these are used to characterise a certain region of the parameter space. The solutions of each such generator instance form valid problem instances. We use the parameter tuner irace to explore the space of possible generator parameters, aiming to find parameter values that yield graded instances. We perform an empirical evaluation of our system for five problem classes from CSPlib, demonstrating promising results.

Discriminating Instance Generation from Abstract Specifications: A Case Study with CP and MIP

“…Most studies in this area tackle one problem class at a time. We here extend [1] to automatically produce instances for any problem class. The goal is to automate finding instances that are particularly suited to one algorithm but not another, and to study characteristics of these instances.…”

“…When developing a model of a combinatorial problem class, a set of representative instances drawn from the class is essential for evaluating the model's performance. Recently, Akgn et al [1] demonstrated how to generate instances automatically from the Essence 1 specification of a problem class [10]. The instances generated are graded : neither too difficult nor too easy relative to a given solver and resource limits.…”

“…A problem specification can be automatically converted to an instance generator specification by Conjure [1]. First, the given statements that declare parameters are converted into find statements that declare corresponding decision variables.…”

“…Second, the where statements are converted to such that statements. This process is explained in more detail in [1].…”

Automatic Streamlining for Constrained Optimisation

“…At the moment the instance generator has to be manually created, and is the only part of the whole system that is not automated. However, this issue has been tackled in a recent work [2] within the same pipeline, which can be integrated into our system in the future. To keep the computational cost manageable, we require a set of relatively easy (but not trivial) instances for the training phase, which we define as solvable by MINION [19] on an unstreamlined model within a time limit of [10, 300] seconds.…”