Algorithm runtime prediction: Methods &amp; evaluation

Hutter, Frank; Xu, Lin; Hoos, Holger H.; Leyton‐Brown, Kevin

doi:10.1016/j.artint.2013.10.003

Cited by 360 publications

(344 citation statements)

References 83 publications

Supporting

Mentioning

340

Contrasting

Order By: Relevance

“…In very recent work, we comprehensively studied EPMs based on a variety of modeling techniques that have been used for performance prediction over the years, including ridge regression [15], neural networks [23], Gaussian processes [20], regression trees [24], and random forests [22]. Overall, we found random forests and approximate Gaussian processes to perform best.…”

Section: Empirical Performance Modelsmentioning

confidence: 99%

“…Random forests (and also regression trees) were particularly strong for very heterogeneous benchmark sets, since their tree-based mechanism automatically groups similar inputs together and does not allow widely different inputs to interfere with the predictions for a given group. Another benefit of the tree-based methods is that hundreds of training data points already sufficed to yield competitive performance predictions in joint input spaces induced by as many as 76 algorithm parameters and 148 instance features [22]. This strong performance suggests that the functions being modeled must be relatively simple, for example by depending at most very weakly on most inputs.…”

Section: Empirical Performance Modelsmentioning

confidence: 99%

“…We used the same benchmark distributions and features as in previous work [22] and only describe them on a high level here. For MIP, we used two instance distributions from computational sustainability (RCW and CORLAT), one from winner determination in combinatorial auctions (REG), two unions of these (CR := CORLAT ∪ RCW and CRR := CORLAT ∪ REG ∪ RCW), and a large and diverse set of publicly available MIP instances (BIGMIX).…”

Section: Benchmark Instances and Their Featuresmentioning

confidence: 99%

“…Since then, these predictive models have been extended to model the dependency of performance on (often categorical) algorithm parameters, to make probabilistic predictions, and to work effectively with large amounts of training data [20,11,21,12,22].…”

Section: Empirical Performance Modelsmentioning

confidence: 99%

See 3 more Smart Citations

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Hutter

Hoos

Leyton‐Brown

2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Most state-of-the-art algorithms for large scale optimization expose free parameters, giving rise to combinatorial spaces of possible configurations. Typically, these spaces are hard for humans to understand. In this work, we study a model-based approach for identifying a small set of both algorithm parameters and instance features that suffices for predicting empirical algorithm performance well. Our empirical analyses on a wide variety of hard combinatorial problem benchmarks (spanning SAT, MIP, and TSP) show that-for parameter configurations sampled uniformly at random-very good performance predictions can typically be obtained based on just two key parameters, and that similarly, few instance features and algorithm parameters suffice to predict the most salient algorithm performance characteristics in the combined configuration/feature space. We also use these models to identify settings of these key parameters that are predicted to achieve the best overall performance, both on average across instances and in an instance-specific way. This serves as a further way of evaluating model quality and also provides a tool for further understanding the parameter space. We provide software for carrying out this analysis on arbitrary problem domains and hope that it will help algorithm developers gain insights into the key parameters of their algorithms, the key features of their instances, and their interactions.

show abstract

Section: Empirical Performance Modelsmentioning

confidence: 99%

Section: Empirical Performance Modelsmentioning

confidence: 99%

Section: Benchmark Instances and Their Featuresmentioning

confidence: 99%

Section: Empirical Performance Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Hutter

Hoos

Leyton‐Brown

2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…For comprehensive surveys about algorithm selection and runtime prediction, we refer the interested reader to [171,108]. There are also several doctoral dissertations related to the AS problem, namely: [175,105,36,69,58,120,130].…”

Section: Related Workmentioning

confidence: 99%

Portfolio approaches in constraint programming

Amadini

2015

Constraints

View full text Add to dashboard Cite

Recent research has shown that the performance of a single, arbitrarily efficient algorithm can be significantly outperformed by using a portfolio of -possibly onaverage slower-algorithms. Within the Constraint Programming (CP) context, a portfolio solver can be seen as a particular constraint solver that exploits the synergy between the constituent solvers of its portfolio for predicting which is (or which are) the best solver(s) to run for solving a new, unseen instance.In this thesis we examine the benefits of portfolio solvers in CP. Despite portfolio approaches have been extensively studied for Boolean Satisfiability (SAT) problems, in the more general CP field these techniques have been only marginally studied and used. We conducted this work through the investigation, the analysis and the construction of several portfolio approaches for solving both satisfaction and optimization problems. We focused in particular on sequential approaches, i.e., single-threaded portfolio solvers always running on the same core.We started from a first empirical evaluation on portfolio approaches for solving Constraint Satisfaction Problems (CSPs), and then we improved on it by introducing new data, solvers, features, algorithms, and tools. Afterwards, we addressed the more general Constraint Optimization Problems (COPs) by implementing and testing a number of models for dealing with COP portfolio solvers. Finally, we have come full circle by developing sunny-cp: a sequential CP portfolio solver that turned out to be competitive also in the MiniZinc Challenge, the reference competition for CP solvers.iii iv

show abstract

Machine learning‐based radiotherapy time prediction and treatment scheduling management

Xie,

Xu,

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

PurposeThe utility efficiency of medical devices is important, especially for countries such as China, which have a large population and shortage of medical care resources. Radiotherapy devices are among the most valuable and specialized equipment categories and carry enormous treatment loads. In this study, a novel method is proposed to improve the efficiency of a radiotherapy device (linac). Although scheduling management with accurate prediction of the entire treatment time included in each appointment, arrange a reasonable time duration for appointments and save time between patient shifts effectively. Tasks belonging to the treatment and non‐treatment groups can be assigned more flexibly based on the availability of time.Material and methodsData from 1665 patients, including patient positioning time (PT) and treatment time (TT), were collected in collaboration with the Radiotherapy Center of the Department of Oncology at the Second Affiliated Hospital of Kunming Medical University from November 2020 to August 2021. The features related to PT and TT were extracted and used to train the machine learning‐based model to predict PT and TT in independent patients. The prediction results were subsequently applied to a minute‐based scheduling tool.ConclusionArtificial intelligence is a promising approach to solve abstract problems with a specialized knowledge background. The results of this study show encouraging prediction outcomes in relation to effective scheduling management and could improve the efficiency of the linac. This successful trial broadens the meaning of medical data and potential future research directions in radiotherapy.

show abstract

Algorithm runtime prediction: Methods & evaluation

Cited by 360 publications

References 83 publications

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Portfolio approaches in constraint programming

Machine learning‐based radiotherapy time prediction and treatment scheduling management

Contact Info

Product

Resources

About