Multi-fidelity Gaussian Process Bandit Optimisation

Kandasamy, Kirthevasan; Dasarathy, Gautam; Oliva, Junier B.; Schneider, Jeff; Póczos, Barnabás

doi:10.48550/arxiv.1603.06288

Cited by 5 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Maximum Information Gain. As in previous works on GP (Chowdhury and Gopalan, 2017;Kandasamy et al, 2016), our results will depend on the maximum information gain (Srinivas et al, 2009) between function measurements and the function values, defined as below: Definition 1. Suppose A ⊆ X is a subset of feature space, and Ã = {x 1 , ..., x n } ⊆ A is a finite subset of A.…”

Section: The Gaussian Process Back Endmentioning

confidence: 99%

“…We build our work on GP-UCB (Srinivas et al, 2009), a method for optimizing unknown functions under the Gaussian process (GP) assumption by optimizing the Upper Confidence Bound (UCB). Closest to our setting is a line of recent research on multi-fidelity GP optimization (Kandasamy et al, 2016(Kandasamy et al, , 2017Sen et al, 2018), which assumes that we can query the target functions at multiple fidelities of different costs and precisions. We detail the relation and difference of our setting with multi-fidelity optimization in Section 3.6.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zeroth Order Non-convex optimization with Dueling-Choice Bandits

Xu,

Joshi,

Singh

et al. 2019

Preprint

View full text Add to dashboard Cite

We consider a novel setting of zeroth order non-convex optimization, where in addition to querying the function value at a given point, we can also duel two points and get the point with the larger function value. We refer to this setting as optimization with dueling-choice bandits since both direct queries and duels are available for optimization. We give the COMP-GP-UCB algorithm based on GP-UCB (Srinivas et al., 2009), where instead of directly querying the point with the maximum Upper Confidence Bound (UCB), we perform a constrained optimization and use comparisons to filter out suboptimal points. COMP-GP-UCB comes with theoretical guarantee of O( Φ √ T ) on simple regret where T is the number of direct queries and Φ is an improved information gain corresponding to a comparison based constraint set that restricts the search space for the optimum. In contrast, in the direct query only setting, Φ depends on the entire domain. Finally, we present experimental results to show the efficacy of our algorithm.

show abstract

Section: The Gaussian Process Back Endmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Zeroth Order Non-convex optimization with Dueling-Choice Bandits

Xu,

Joshi,

Singh

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, the multi-fidelity setting has been theoretically studied in online problems [40,1,33,18]. In some recent works [19,16,17], UCB like algorithms with Bayesian Gaussian process assumptions on f have been analyzed in a multi-fidelity black-box optimization setting. Also relevant to this work are the bandit based techniques for hyper-parameter optimization such as [28,14].…”

Section: Related Workmentioning

confidence: 99%

“…We use multi-fidelity versions of commonly used benchmark functions in the black-box optimization literature. These multi-fidelity versions have been previously used in [17,34].…”

Section: A More On Mfpoo (Algorithm 2)mentioning

confidence: 99%

See 1 more Smart Citation

Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach

Sen¹,

Kandasamy²,

Shakkottai³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of black-box optimization of a noisy function in the presence of low-cost approximations or fidelities, which is motivated by problems like hyper-parameter tuning. In hyperparameter tuning evaluating the black-box function at a point involves training a learning algorithm on a large data-set at a particular hyper-parameter and evaluating the validation error. Even a single such evaluation can be prohibitively expensive. Therefore, it is beneficial to use low-cost approximations, like training the learning algorithm on a sub-sampled version of the whole data-set. These low-cost approximations/fidelities can however provide a biased and noisy estimate of the function value. In this work, we incorporate the multi-fidelity setup in the powerful framework of noisy black-box optimization through tree-like hierarchical partitions. We propose a multi-fidelity bandit based tree-search algorithm for the problem and provide simple regret bounds for our algorithm. Finally, we validate the performance of our algorithm on real and synthetic datasets, where it outperforms several benchmarks.

show abstract

Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization

Marco

Berkenkamp

Hennig

et al. 2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

101

View full text Add to dashboard Cite

In practice, the parameters of control policies are often tuned manually. This is time-consuming and frustrating. Reinforcement learning is a promising alternative that aims to automate this process, yet often requires too many experiments to be practical. In this paper, we propose a solution to this problem by exploiting prior knowledge from simulations, which are readily available for most robotic platforms. Specifically, we extend Entropy Search, a Bayesian optimization algorithm that maximizes information gain from each experiment, to the case of multiple information sources. The result is a principled way to automatically combine cheap, but inaccurate information from simulations with expensive and accurate physical experiments in a cost-effective manner. We apply the resulting method to a cart-pole system, which confirms that the algorithm can find good control policies with fewer experiments than standard Bayesian optimization on the physical system only.

show abstract

Multi-fidelity Gaussian Process Bandit Optimisation

Cited by 5 publications

References 0 publications

Zeroth Order Non-convex optimization with Dueling-Choice Bandits

Zeroth Order Non-convex optimization with Dueling-Choice Bandits

Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach

Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization

Contact Info

Product

Resources

About