Bayesian Active Learning for Sensitivity Analysis

Pfingsten, Tobias

doi:10.1007/11871842_35

Cited by 13 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, in Bayesian active learning, the expected utility is considered by averaging over possible outcomes. 11 Informationbased criteria as proposed by MacKay [29], Krause et al [27] and Pfingsten [42], for example, or their combination with expected outcomes as discussed by Verdinelli and Kadane [54] and Chaloner and Verdinelli [9] are commonly used to define utility functions. Solely maximizing an expected information gain tends to select states far away from the current state set.…”

Section: Bayesian Active Learningmentioning

confidence: 99%

Gaussian process dynamic programming

2009

View full text Add to dashboard Cite

a b s t r a c tReinforcement learning (RL) and optimal control of systems with continuous states and actions require approximation techniques in most interesting cases. In this article, we introduce Gaussian process dynamic programming (GPDP), an approximate value function-based RL algorithm. We consider both a classic optimal control problem, where problem-specific prior knowledge is available, and a classic RL problem, where only very general priors can be used. For the classic optimal control problem, GPDP models the unknown value functions with Gaussian processes and generalizes dynamic programming to continuous-valued states and actions. For the RL problem, GPDP starts from a given initial state and explores the state space using Bayesian active learning. To design a fast learner, available data have to be used efficiently. Hence, we propose to learn probabilistic models of the a priori unknown transition dynamics and the value functions on the fly. In both cases, we successfully apply the resulting continuous-valued controllers to the under-actuated pendulum swing up and analyze the performances of the suggested algorithms. It turns out that GPDP uses data very efficiently and can be applied to problems, where classic dynamic programming would be cumbersome.

show abstract

Section: Bayesian Active Learningmentioning

confidence: 99%

Gaussian process dynamic programming

2009

View full text Add to dashboard Cite

show abstract

“…Originally introduced for reinforcement learning [1], [9], the curiosity framework has been used for active learning [10], [11], to explain certain patterns of human visual attention better than previous approaches [12], and to explain concepts such as beauty, attention and creativity [3], [13].…”

Section: Curiosity-driven Optimizationmentioning

confidence: 99%

Curiosity-driven optimization

Schaul

Sun

Wierstra

et al. 2011

2011 IEEE Congress of Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Abstract-The principle of artifi cial curiosity directs active exploration towards the most informative or most interesting data. We show its usefulness for global black box optimization when data point evaluations are expensive. Gaussian process regression is used to model the fi tness function based on all available observations so far. For each candidate point this model estimates expected fi tness reduction, and yields a novel closed-form expression of expected information gain. A new type of Pareto-front algorithm continually pushes the boundary of candidates not dominated by any other known data according to both criteria, using multi-objective evolutionary search. This makes the exploration-exploitation trade-off explicit, and permits maximally informed data selection. We illustrate the robustness of our approach in a number of experimental scenarios.

show abstract

“…For example, Gramacy et al (2004) used nonstationary Gaussian process trees to explore a computational fluid dynamics simulation of a NASA reentry vehicle. Pfingsten (2006) used a Bayesian active learning technique to assist in analyzing micro-mechanical sensors. A research area known as Design and Analysis of Computer Experiments (DACE) (Sacks et al, 1989) uses statistical methods, including kriging, to construct surrogates to deterministic computer models.…”

Section: Related Workmentioning

confidence: 99%

Active learning for directed exploration of complex systems

Burl

Wang

2009

Proceedings of the 26th Annual International Conference on Machine Learning

View full text Add to dashboard Cite

Physics-based simulation codes are widely used in science and engineering to model complex systems that would be infeasible to study otherwise. Such codes provide the highest-fidelity representation of system behavior, but are often so slow to run that insight into the system is limited. For example, conducting an exhaustive sweep over a d-dimensional input parameter space with ksteps along each dimension requires k d simulation trials (translating into k d CPU-days for one of our current simulations). An alternative is directed exploration in which the next simulation trials are cleverly chosen at each step. Given the results of previous trials, supervised learning techniques (SVM, KDE, GP) are applied to build up simplified predictive models of system behavior. These models are then used within an active learning framework to identify the most valuable trials to run next. Several active learning strategies are examined including a recently-proposed information-theoretic approach. Performance is evaluated on a set of thirteen synthetic oracles, which serve as surrogates for the more expensive simulations and enable the experiments to be replicated by other researchers.

show abstract

Bayesian Active Learning for Sensitivity Analysis

Cited by 13 publications

References 16 publications

Gaussian process dynamic programming

Gaussian process dynamic programming

Curiosity-driven optimization

Active learning for directed exploration of complex systems

Contact Info

Product

Resources

About