Convergence properties of the expected improvement algorithm with fixed mean and covariance functions

Vázquez, Emmanuel; Bect, Julien

doi:10.1016/j.jspi.2010.04.018

Cited by 157 publications

(121 citation statements)

References 13 publications

Supporting

Mentioning

117

Contrasting

Order By: Relevance

“…In particular, a number of approximations have been introduced and it is of interest whether the proposed strategy of selecting points according to the MEI will find good solutions and whether it will converge to the global optimum. Recent work [12] gives positive convergence results when the prior function is a Gaussian Process (GP) with fixed mean and covariance [13]. These results hold under fairly general conditions which apply to the BO algorithm on which our work is based.…”

Section: Bayesian Optimizationsupporting

confidence: 52%

Incorporating Domain Models into Bayesian Optimization for RL

Wilson

Fern

Tadepalli

2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In many Reinforcement Learning (RL) domains there is a high cost for generating experience in order to evaluate an agent's performance. An appealing approach to reducing the number of expensive evaluations is Bayesian Optimization (BO), which is a framework for global optimization of noisy and costly to evaluate functions. Prior work in a number of RL domains has demonstrated the effectiveness of BO for optimizing parametric policies. However, those approaches completely ignore the state-transition sequence of policy executions and only consider the total reward achieved. In this paper, we study how to more effectively incorporate all of the information observed during policy executions into the BO framework. In particular, our approach uses the observed data to learn approximate transitions models that allow for Monte-Carlo predictions of policy returns. The models are then incorporated into the BO framework as a type of prior on policy returns, which can better inform the BO process. The resulting algorithm provides a new approach for leveraging learned models in RL even when there is no planner available for exploiting those models. We demonstrate the effectiveness of our algorithm in four benchmark domains, which have dynamics of variable complexity. Results indicate that our algorithm effectively combines model based predictions to improve the data efficiency of model free BO methods, and is robust to modeling errors when parts of the domain cannot be modeled successfully.

show abstract

Section: Bayesian Optimizationsupporting

confidence: 52%

Incorporating Domain Models into Bayesian Optimization for RL

Wilson

Fern

Tadepalli

2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Expected Improvement has also been shown to converge under additional mild assumptions [14], but its main advantage is a closed-form expression, that doesn't require numerical integration:…”

Section: Theoremmentioning

confidence: 99%

Optimising the Active Muon Shield for the SHiP Experiment at CERN

Baranov

Burnaev

Ustyuzhanin

et al. 2017

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Abstract. The SHiP experiment is designed to search for very weakly interacting particles beyond the Standard Model which are produced in a 400 GeV/c proton beam dump at the CERN SPS. The critical challenge for this experiment is to keep the Standard Model background level negligible. In the beam dump, around 10 11 muons will be produced per second. The muon rate in the spectrometer has to be reduced by at least four orders of magnitude to avoid muoninduced backgrounds. It is demonstrated that new improved active muon shield may be used to magnetically deflect the muons out of the acceptance of the spectrometer.

show abstract

“…The minimization of the Objective Function can be performed referring to many techniques, said to be global or local depending on whether they converge to global or local minima. For instance, the following types of methods can be highlighted: -the local gradient-based methods (Tarantola, 2005), which include quasi-Newton and Gauss-Newton families; -the global evolutionary and genetic methods (Hansen, 2006); -the global response surface-based methods (Schonlau, 1997;Villemonteix et al, 2009;Vazquez and Bect, 2010). In the remainder of this paper, we propose to use the known simulation results to approximate p Y(x) (y) from M. Feraille / An Optimization Strategy Based on the Maximization of Matching-Targets' Probability for Unevaluated Results several Gaussian processes, instead of considering a Dirac probability density function.…”

Section: Probabilistic Inverse Problem Frameworkmentioning

confidence: 99%

“…As described in Schonlau (1997), Villemonteix et al (2009) and Vazquez and Bect (2010), Gaussian process properties, with characterization of mean and covariance, can also be used for local or global optimization. In this case, the required OF is approximated by a Gaussian process and an improvement function corresponding to the potential reduction of the OF is defined.…”

Section: Introductionmentioning

confidence: 99%

An Optimization Strategy Based on the Maximization of Matching-Targets’ Probability for Unevaluated Results

Feraille

2013

Oil Gas Sci. Technol. – Rev. IFP Energies nouvelles

View full text Add to dashboard Cite

Re´sume´-Optimisation par maximisation de la probabilite´d'atteindre les cibles pour des re´sultats non e´value´s -La me´thode pre´sente´e dans cet article rentre dans le cadre de l'optimisation probabiliste classique. La nouveaute´consiste a`construire un processus Gaussien pour chaque re´sultat souhaite´(i.e. associe´a`chaque cible spe´cifie´e) et de les utiliser ensuite pour estimer les densite´s de probabilite´des re´sultats non e´value´s. Ces densite´s sont alors prises en compte dans le calcul de la densite´a posteriori des parame`tres a`optimiser. Cette approche est adapte´e lorsque chaque e´valuation de la fonction est couˆteuse en temps de calcul. Une description de´taille´e de cette me´thode d'optimisation est propose´e ainsi que son utilisation sur plusieurs cas tests.Abstract -An Optimization Strategy Based on the Maximization of Matching-Targets' Probability for Unevaluated Results -The Maximization of Matching-Targets' Probability for Unevaluated Results (MMTPUR), technique presented in this paper, is based on the classical probabilistic optimization framework. The numerical function values that have not been evaluated are considered as stochastic functions. Thus, a Gaussian process uncertainty model is built for each required numerical function result (i.e., associated with each specified target) and is used to estimate probability density functions for unevaluated results. Parameter posterior distributions, used within the optimization process, then take into account these probabilities. This approach is particularly adapted when, getting one evaluation of the numerical function is very time consuming. In this paper, we provide a detailed outline of this technique. Finally, several test cases are developed to stress its potential.

show abstract

Convergence properties of the expected improvement algorithm with fixed mean and covariance functions

Cited by 157 publications

References 13 publications

Incorporating Domain Models into Bayesian Optimization for RL

Incorporating Domain Models into Bayesian Optimization for RL

Optimising the Active Muon Shield for the SHiP Experiment at CERN

An Optimization Strategy Based on the Maximization of Matching-Targets’ Probability for Unevaluated Results

Contact Info

Product

Resources

About