Large-scale Bayesian optimal experimental design with derivative-informed projected neural network

Wu, Keyi; O’Leary-Roseberry, Thomas; Chen, Peng; Ghattas, Omar

doi:10.48550/arxiv.2201.07925

Cited by 1 publication

(2 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scalable solution of highdimensional outer-loop problems often requires derivative information, since this information detects map sensitivities that can make problems effectively low-dimensional. This property has been observed and used in many outer-loop problems such as model reduction for sampling and deep learning [4,5,13,34,36], optimization under uncertainty [3,15,17], Bayesian inverse problems [6,9,10,11,12,14,16,18,21,23,26,44], and Bayesian optimal experimental design [2,20,39,40,41].…”

Section: State Modelmentioning

confidence: 99%

“…In this case there is less variance in the prior distribution, but more sensitivity to the diffusion operator due to the forcing terms and the cubic nonlinearity. The third test case is a convection-reaction-diffusion (CRD) problem where the parameter shows up in a nonlinear reaction term [36,41]. In this case the parameters for the distribution are δ = 1.0, γ = 0.1.…”

Section: Poisson Problemmentioning

confidence: 99%

See 1 more Smart Citation

Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning

O’Leary-Roseberry¹,

Chen²,

Villa³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Neural operators have gained significant attention recently due to their ability to approximate high-dimensional parametric maps between function spaces. At present, only parametric function approximation has been addressed in the neural operator literature. In this work we investigate incorporating parametric derivative information in neural operator training; this information can improve function approximations, additionally it can be used to improve the approximation of the derivative with respect to the parameter, which is often the key to scalable solution of high-dimensional outer-loop problems (e.g. Bayesian inverse problems). Parametric Jacobian information is formally intractable to incorporate due to its high-dimensionality, to address this concern we propose strategies based on reduced SVD, randomized sketching and the use of reduced basis surrogates. All of these strategies only require only O(r) Jacobian actions to construct sample Jacobian data, and allow us to reduce the linear algebra and memory costs associated with the Jacobian training from the product of the input and output dimensions down to O(r 2 ), where r is the dimensionality associated with the dimension reduction technique.Numerical results for parametric PDE problems demonstrate that the addition of derivative information to the training problem can significantly improve the parametric map approximation, particularly given few data. When Jacobian actions are inexpensive compared to the parametric map, this information can be economically substituted for parametric map data. Additionally we show that Jacobian error approximations improve significantly with the introduction of Jacobian training data. This result opens the door to the use of derivative-informed neural operators (DINOs) in outer-loop algorithms where they can amortize the additional training data cost via repeated evaluations.

show abstract