Stochastic Methods for Single Objective Global Optimization

Abstract

I. INTRODUCTIONStochastic (or randomized) optimization schemes were among the first algorithms proposed to numerically solve unconstrained global optimization problems. We here consider stochastic optimization schemes that 1) comprise a randomized mechanism to iteratively generate candidate solutions and 2) do not require gradient or Hessian information about the objective function. In many areas of modern science and engineering such methods, also referred to as metaheuristics, simulation optimizers, or randomized search heuristics, are indispensable in the black-box (or simulation) optimization context where higher-order information about the objective is not available or does not exist. In the past five decades an enormous number of these general-purpose optimizers has been proposed and developed across many different scientific fields. This makes it increasingly difficult to keep track of true novelties and relevant improvements in the field. In this introductory chapter we highlight some of the key concepts and algorithmic paradigms in single objective stochastic global optimization. We first present a general problem definition and introduce the landscape concept for black-box optimization. We then present a generic global optimization framework that fits a large number of present days algorithms and heuristics. We then discuss popular instances of stochastic global optimization schemes, ranging from mathematical programming to evolutionary computation. We also present an alternative view point on stochastic optimization borrowed from the realm of computational statistics. We show that many modern optimization heuristics strongly resemble some of the most successful Markov Chain Monte Carlo schemes used in statistical inference. We close this chapter with a review of several standard test environments for stochastic global optimization schemes and provide links for further reading.

Distilling Robust Design Principles of Biocircuits Using Mixed Integer Dynamic Optimization

“…Regarding future work, we plan to refine the robustness metric used here, which could over-estimate the viable space for specific nonlinear relationships among parameters. In addition, it is worth mentioning that we suspect there might be significant conceptual similarities between the sampling in our optimization-based approach and those using computational statistics techniques due to the connections between the latter and stochastic global optimization methods [19]. We believe that this topic deserves a more systematic and detailed comparative analysis.…”

Advanced mathematical modelling tools for food processing and food safety assessment

“…Hay que tener en cuenta que, con estos métodos estocásticos, la convergencia a la optimalidad global no está garantizada, pero varios estudios empíricos han demostrado que suelen ser los mejores métodos para muchas clases de problemas (Mller, 2014;Moles, Mendes, et al, 2003).…”