Algorithm Selection as a Bandit Problem with Unbounded Losses

Gagliolo, Matteo; Schmidhuber, Jürgen

doi:10.1007/978-3-642-13800-3_7

Cited by 26 publications

(21 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The use of MAB algorithms to solve the EvE dilemma has been investigated in the context of selecting between different algorithm portfolios to solve decision problems [18], and in the framework of Adaptive Operator Selection by the authors. For the latter case, the Upper Confidence Bound (UCB) technique [1] was used, being referred to as the original (or basic) MAB algorithm in the following.…”

Section: Introductionmentioning

confidence: 99%

Analyzing bandit-based adaptive operator selection mechanisms

Fialho

Costa

Schoenauer

et al. 2010

Ann Math Artif Intell

135

123

View full text Add to dashboard Cite

Several techniques have been proposed to tackle the Adaptive Operator Selection (AOS) issue in Evolutionary Algorithms. Some recent proposals are based on the Multi-Armed Bandit (MAB) paradigm: each operator is viewed as one arm of a MAB problem, and the rewards are mainly based on the fitness improvement brought by the corresponding operator to the individual it is applied to. However, the AOS problem is dynamic, whereas standard MAB algorithms are known to optimally solve the exploitation versus exploration trade-off in static settings. An original dynamic variant of the standard MAB Upper Confidence Bound algorithm is proposed here, using a sliding time window to compute both its exploitation and exploration terms. In order to perform sound comparisons between AOS algorithms, artificial scenarios have been proposed in the literature. They are extended here toward smoother transitions between different reward settings. The resulting original testbed also includes a real evolutionary algorithm that is applied to the well-known Royal Road problem. It is used here to perform a thorough analysis of the behavior of AOS algorithms, to assess their sensitivity with respect to their own hyper-parameters, and to propose a sound comparison of their performances.

show abstract

Section: Introductionmentioning

confidence: 99%

Analyzing bandit-based adaptive operator selection mechanisms

Fialho

Costa

Schoenauer

et al. 2010

Ann Math Artif Intell

135

123

View full text Add to dashboard Cite

show abstract

“…For the same set-up, an allocation strategy is proposed in [16] based on updating dynamically the belief over the run-time distribution. Finally, when a set of time allocation strategies are available and the optimization problem is to be solved several times, one can use the standard multi-armed bandit framework as in [9,10,11].…”

Section: Introductionmentioning

confidence: 99%

Efficient Multi-start Strategies for Local Search Algorithms

Kocsis

György

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Local search algorithms for global optimization often suffer from getting trapped in a local optimum. The common solution for this problem is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resources (in particular, processing time) to the instances depending on their behavior. Hence, a multi-start strategy has to decide (dynamically) when to allocate additional resources to a particular instance and when to start new instances. In this paper we propose a consistent multi-start strategy that assumes a convergence rate of the local search algorithm up to an unknown constant, and in every phase gives preference to those instances that could converge to the best value for a particular range of the constant. Combined with the local search algorithm SPSA (Simultaneous Perturbation Stochastic Approximation), the strategy performs remarkably well in practice, both on synthetic tasks and on tuning the parameters of learning algorithms.

show abstract

“…Further references on algorithm selection can be found in [8,9]. Literature on parallel computing, grid computing, distributed computing [6,1,18] is focused on allocation of dynamically changing computational resources, in a transparent and fault tolerant manner.…”

Section: Related Workmentioning

confidence: 99%

“…In [9], basing on [3], we introduced EXP3LIGHT-A, a BPS which guarantees a bound on regret when the maximum loss is unknown a priori. Note that any bound on the regret of the chosen BPS will determine a bound on the regret of GAMBLETA with respect to the best time allocator.…”

Section: Algorithm 1 Gambleta(at Bps) Gambling Time Allocatormentioning

confidence: 99%

“…Released CPUs are then reallocated by the front-end to solve the following problem instance. At the upper level, the BPS (EXP3LIGHT-A from [9]) is used as in GAMBLETA, using the total CPU time as a loss (i. e., jt if j CPUs are used for a wall-clock time t), in order to favor time allocators that do not use more CPUs than necessary.…”

Section: Allocating Multiple Cpusmentioning

confidence: 99%

See 1 more Smart Citation

Towards Distributed Algorithm Portfolios

Gagliolo

Schmidhuber

Advances in Soft Computing

Self Cite

View full text Add to dashboard Cite

Summary. In recent work we have developed an online algorithm selection technique, in which a model of algorithm performance is learned incrementally while being used. The resulting exploration-exploitation trade-off is solved as a bandit problem. The candidate solvers are run in parallel on a single machine, as an algorithm portfolio, and computation time is shared among them according to their expected performances. In this paper, we extend our technique to the more interesting and practical case of multiple CPUs.

show abstract

Algorithm Selection as a Bandit Problem with Unbounded Losses

Cited by 26 publications

References 30 publications

Analyzing bandit-based adaptive operator selection mechanisms

Analyzing bandit-based adaptive operator selection mechanisms

Efficient Multi-start Strategies for Local Search Algorithms

Towards Distributed Algorithm Portfolios

Contact Info

Product

Resources

About