Nesterov-aided Stochastic Gradient Methods using Laplace Approximation for Bayesian Design Optimization

Carlon, André Gustavo; Dia, Ben Mansour; Espath, Luis FR; López, Rosalía Rodríguez; Tempone, Raúl

doi:10.48550/arxiv.1807.00653

Cited by 2 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, surrogate models and approximations of the marginal likelihood could be combined with the full-model LMIS estimator using a control variate or multilevel Monte Carlo approach, to further reduce the number of full model evaluations while preserving consistency. Another clear extension is to pair LMIS estimators with optimization methods designed for stochastic objectives (e.g., [20,40,32,8]) to facilitate search over the design space.…”

Section: Discussionmentioning

confidence: 99%

A layered multiple importance sampling scheme for focused optimal Bayesian experimental design

Feng,

Marzouk

2019

Preprint

View full text Add to dashboard Cite

We develop a new computational approach for "focused" optimal Bayesian experimental design with nonlinear models, with the goal of maximizing expected information gain in targeted subsets of model parameters. Our approach considers uncertainty in the full set of model parameters, but employs a design objective that can exploit learning trade-offs among different parameter subsets. We introduce a new layered multiple importance sampling scheme that provides consistent estimates of expected information gain in this focused setting. This sampling scheme yields significant reductions in estimator bias and variance for a given computational effort, making optimal design more tractable for a wide range of computationally intensive problems.

show abstract

Section: Discussionmentioning

confidence: 99%

A layered multiple importance sampling scheme for focused optimal Bayesian experimental design

Feng,

Marzouk

2019

Preprint

View full text Add to dashboard Cite

show abstract

Section: Choice Of Mldlmc Parametersmentioning

confidence: 99%

“…The model for the average work per sample of g , denoted by W (g ), is assumed to follow (11). Then, for a fixed κ ∈ (0, 1), the level L and M can be estimated from the bias constraint (36), given Assumption 1 and work model (11). Given L and κ, minimizing the work (44), subject to the bias (36) and statistical (39) constraint with probability 1 − α, gives us the number of samples on level for the MLDLMC estimator (cf.…”

Section: Choice Of Mldlmc Parametersmentioning

confidence: 99%

“…Given Assumption 1, the work model (11), and that η w ≥ min (η s , γ) /2 holds, the first term of the total work model (46) will be the dominant one asymptotically [17]. Therefore, we omit the contribution of the second term in the following asymptotic analysis.…”

Section: Optimal Work Analysismentioning

confidence: 99%

“…Recent work on Bayesian alphabetical optimal experimental design includes [2,1,3,14,40,42]. Efficient optimization strategies on continuous design spaces include stochastic gradient methods (e.g., [11,23,22]) and the approximate coordinate exchange algorithm (e.g., [30,33]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design

Beck

Dia

Espath

et al. 2020

Numerical Meth Engineering

View full text Add to dashboard Cite

An optimal experimental set-up maximizes the value of data for statistical inferences and predictions. An optimal set-up is particularly important for experiments that are time consuming or expensive to perform. In the context of partial differential equations (PDEs), multilevel methods have been proven to dramatically reduce the computational complexity of their single-level counterparts. Here, two multilevel methods, which efficiently compute the expected information gain using a Kullback-Leibler divergence measure in simulation-based Bayesian optimal experimental design, are proposed. The first method is a multilevel double loop Monte Carlo (MLDLMC) with importance sampling that greatly reduces the computational work of the inner loop. The second proposed method is a multilevel double loop stochastic collocation (MLDLSC) with importance sampling, which performs a high-dimensional integration by deterministic quadrature on sparse grids. In both methods, the Laplace approximation is used as an effective means of importance sampling, and the optimal values of the method parameters are determined by minimizing the average computational work, subject to a desired error tolerance. The computational efficiencies of the methods are demonstrated by computing the expected information gain from an electrical impedance tomography experiment where the fiber orientation in composite laminate materials are inferred through Bayesian inversion. MLDLSC performs better than MLDLMC when the regularity of the underlying computational model, with respect to the additive noise and the unknown parameters, can be exploited.

show abstract

Nesterov-aided Stochastic Gradient Methods using Laplace Approximation for Bayesian Design Optimization

Cited by 2 publications

References 0 publications

A layered multiple importance sampling scheme for focused optimal Bayesian experimental design

A layered multiple importance sampling scheme for focused optimal Bayesian experimental design

Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design

Contact Info

Product

Resources

About