Hybrid Ranking and Regression for Algorithm Selection

Hanselle, Jonas; Tornede, Alexander; Wever, Marcel; Hüllermeier, Eyke

doi:10.1007/978-3-030-58285-2_5

“…For example, putting more emphasis on the "dyadic" nature of instances (consisting of a hardware and a software component), one may think of leveraging methods specialized on dyadic prediction [16]. Another idea is to tackle the problem as a ranking instead of a regression problem, which is meaningful if predictions are only used to compare configurations with each other [10]. Last but not least, the aspect of explainability might be of interest: In addition to making a performance prediction, the learner should be able to provide an explanation of that prediction, for example by quantifying the importance of features or highlighting components that appear to be critical.…”

Section: Discussionmentioning

confidence: 99%

Performance Prediction for Hardware-Software Configurations: A Case Study for Video Games

Peeters

¹

,

Melnikov

²

,

Hüllermeier

³

2021

Lecture Notes in Computer Science

Self Cite

0

View full text Add to dashboard Cite

Performance prediction for hardware-software configurations is a relevant and practically important problem. With an increasing availability of data in the form of performance measurements, this problem becomes amenable to machine learning, i.e., the data-driven construction of predictive models. In this paper, we propose a learning method that is specifically tailored to the task of performance prediction and takes two important characteristics of this problem into account: (i) prior knowledge in the form of monotonicity constraints, suggesting that certain properties of hard-or software can influence performance only positively or negatively, and (ii) strong differences in the precision and reliability of performance measurements available as training data. We evaluate our method on a real-world dataset from the domain of performance prediction in video games, which we specifically collected for this purpose.

show abstract

“…Methods for ranking and comparing include using empirical hardness models Leyton-Brown et al (2009) or more general surrogates as in GGA++ (Ansótegui et al, 2015) or SMAC , statistics (López-Ibánez et al, 2016), the TrueSkill mechanism (Fitzgerald et al, 2015), bandits (El Mesaoudi-Paul et al, 2020b). Nonetheless, there is undoubtedly still room for improvement, using perhaps new preference learning techniques (as have been used for AS in Hanselle et al (2020)) or deep learning models.…”

Section: Novel Ac Methodologiesmentioning

confidence: 99%

A Survey of Methods for Automated Algorithm Configuration

Schede¹,

Brandt²,

Tornede³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There is currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry. Finally, our review provides researchers and practitioners with a look at future research directions in the field of AC.

show abstract

“…Methods for ranking and comparing include using empirical hardness models Leyton-Brown et al (2009) or more general surrogates as in GGA++ (Ansótegui et al, 2015) or SMAC ), statistics (López-Ibánez et al, 2016, the TrueSkill mechanism (Fitzgerald et al, 2015), bandits (El Mesaoudi-Paul et al, 2020b). Nonetheless, there is undoubtedly still room for improvement, using perhaps new preference learning techniques (as have been used for AS in Hanselle et al (2020)) or deep learning models.…”

Section: Novel Ac Methodologiesmentioning

confidence: 99%

A Survey of Methods for Automated Algorithm Configuration

Schede

¹

,

Brandt

²

,

Tornede

³

et al. 2022

jair

Self Cite

View full text Add to dashboard Cite

Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There is currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry. Finally, our review provides researchers and practitioners with a look at future research directions in the field of AC.

show abstract

Hybrid Ranking and Regression for Algorithm Selection

Cited by 8 publications

References 19 publications

Performance Prediction for Hardware-Software Configurations: A Case Study for Video Games

Performance Prediction for Hardware-Software Configurations: A Case Study for Video Games

A Survey of Methods for Automated Algorithm Configuration

A Survey of Methods for Automated Algorithm Configuration

Contact Info

Product

Resources

About