Curiosity-driven optimization

Schaul, Tom; Sun, Yu; Wierstra, Daan; Gomez, Fausino; Schmidhuber, Jürgen

doi:10.1109/cec.2011.5949772

Cited by 13 publications

(11 citation statements)

References 28 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Novelty search [106] is a possible starting point, but it raises a new question: in what space do we measure novelty? Another approach is to define and formalize interestingness and define a curiosity-driven search process [160,149] or to rely on more indirect selective pressures as in co-evolution [172,64,65] or environment-driven pressures [27]. Considering the search process as a creative process also raises another important question: on what scientific method do we rely?…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Doncieux

Mouret

2014

Evol. Intel.

View full text Add to dashboard Cite

Evolutionary robotics is often viewed as the application of a family of black-box optimization algorithms -evolutionary algorithms -to the design of robots, or parts of robots. When considering evolutionary robotics as black-box optimization, the selective pressure is mainly driven by a user-defined, black-box fitness function, and a domain-independent selection procedure. However, most evolutionary robotics experiments face similar challenges in similar setups: the selective pressure, and, in particular, the fitness function, is not a pure user-defined black box. The present review shows that, because evolutionary robotics experiments share common features, selective pressures for evolutionary robotics are a subject of research on their own. The literature has been split into two categories: goal refiners, aimed at changing the definition of a good solution, and process helpers, designed to help the search process. Two subcategories are further considered: task-specific approaches, which require knowledge on how to solve the task and taskagnostic ones, which do not need it. Besides highlighting the diversity of the approaches and their respective goals, the present review shows that many task-agnostic process helpers have been proposed during the last years, thus bringing us closer to the goal of a fully automated robot behavior design process.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Doncieux

Mouret

2014

Evol. Intel.

View full text Add to dashboard Cite

show abstract

“…Methods that store the locally best individual (such as differential evolution) also need the flexibility to select from previous generations. Furthermore, several recently proposed techniques such as Novelty Search [9], Curiosity Search [15], and Evolutionary Annealing [10] store members from each population in an archive, making them available for selection.…”

Section: Selection Rulesmentioning

confidence: 99%

A measure-theoretic analysis of stochastic optimization

Lockett

Miikkulainen

2013

Proceedings of the Twelfth Workshop on Foundations of Genetic Algorithms XII

View full text Add to dashboard Cite

This paper proposes a measure-theoretic framework to study iterative stochastic optimizers that provides theoretical tools to explore how the optimization methods may be improved. Within this framework, optimizers form a closed, convex subset of a normed vector space, implying the existence of a distance metric between any two optimizers and a meaningful and computable spectrum of new optimizers between them. It is shown how the formalism applies to evolutionary algorithms in general. The analytic property of continuity is studied in the context of genetic algorithms, revealing the conditions under which approximations such as metamodeling or surrogate methods may be effective. These results demonstrate the power of the proposed analytic framework, which can be used to propose and analyze new techniques such as controlled convex combinations of optimizers, meta-optimization of algorithm parameters, and more.

show abstract

“…Due to issues of infinite regress, this optimizer is likely to be uncomputable in many cases. Methods that perform an inner optimization have been proposed previously; the inner optimization loop can be computationally expensive [22].…”

Section: Estimating Model-optimal Choicesmentioning

confidence: 99%

Model-optimal optimization by solving bellman equations

Lockett

2014

Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

This paper analytically identifies the points chosen by the best black-box optimization methods, measured by assigning values to the history of points examined by stochastic optimization methods applied to randomly chosen but static (single) objective functions. Optimizers that make these "best" choices are called model-optimal. Model-optimal optimizers may not exist for a given model of static objectives but can always be approximated. If the search domain and the fitness range are both compact, or if other more abstract conditions are satisfied, it can be proven that model-optimal optimizers exist and at least one of them is deterministic. Model-optimality is studied by treating the average performance as a Bellman equation. The overall performance of an optimizer can be assessed based on the sequence of points selected for evaluation. Each choice introduces an error, and the overall performance is in every case just the sum of these errors. It might be possible to extend these results to certain stochastic or dynamic objectives, but they probably do not apply in adaptive or coevolutionary environments. The properties of model-optimality may make it possible to assess the tightness of existing bounds on average performance, and it may be possible to approximate model-optimal optimization decisions directly.

show abstract

Curiosity-driven optimization

Cited by 13 publications

References 28 publications

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

A measure-theoretic analysis of stochastic optimization

Model-optimal optimization by solving bellman equations

Contact Info

Product

Resources

About