Output-Weighted Sampling for Multi-Armed Bandits with Extreme Payoffs

Yang, Yibo; Blanchard, Antoine; Sapsis, Themistoklis P.; Perdikaris, Paris

doi:10.48550/arxiv.2102.10085

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, while we focus on Bayesian experimental design, slight adjustments to the choice of acquisition function allow for Bayesian Optimization tasks [22] or for approaching other metrics of interest (e.g. mean squared error).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Discovering and forecasting extreme events via active learning in neural operators

Pickering

Guth²,

Karniadakis³

et al. 2022

Nat Comput Sci

View full text Add to dashboard Cite

Extreme events in society and nature, such as pandemic spikes or rogue waves, can have catastrophic consequences. Characterizing extremes is difficult as they occur rarely, arise from seemingly benign conditions, and belong to complex and often unknown infinite-dimensional systems. Such challenges render attempts at characterizing them as moot. We address each of these difficulties by combining novel training schemes in Bayesian experimental design (BED) with an ensemble of deep neural operators (DNOs). This model-agnostic framework pairs a BED scheme that actively selects data for quantifying extreme events with an ensemble of DNOs that approximate infinite-dimensional nonlinear operators.We find that not only does this framework clearly beat Gaussian processes (GPs) but that 1) shallow ensembles of just two members perform best; 2) extremes are uncovered regardless of the state of initial data (i.e. with or without extremes); 3) our method eliminates "double-descent" phenomena; 4) the use of batches of suboptimal acquisition points compared to step-by-step global optima does not hinder BED performance; and 5) Monte Carlo acquisition outperforms standard minimizers in high-dimensions.Together these conclusions form the foundation of an AI-assisted experimental infrastructure that can efficiently infer and pinpoint critical situations across many domains, from physical to societal systems.

show abstract

Section: Discussionmentioning

confidence: 99%

“…where L is a diagonal matrix containing the lengthscales for each dimension and the GP hyperparameters appearing in the covariance function (σ 2 f and L in (22) are trained by maximum likelihood estimation).…”

Section: Gaussian Process Regressionmentioning

confidence: 99%

Discovering and forecasting extreme events via active learning in neural operators

Pickering

Guth²,

Karniadakis³

et al. 2022

Nat Comput Sci

View full text Add to dashboard Cite

show abstract

“…Although neural network architectures are attractive for approximating nonlinear regression tasks, their complexity rids them of analytical expressions. This does not allow for a traditional Bayesian treatment of uncertainty in the underlying surrogate model -a key property present for GP regression (see equation 22). Knowledge of the uncertainty of a surrogate model allows one to target model deficiencies as seen in the parameter space.…”

Section: Ensemble Of Neural Network For Uncertainty Quantificationmentioning

confidence: 99%

Discovering and forecasting extreme events via active learning in neural operators

Pickering¹,

Guth²,

Karniadakis³

et al. 2022

Preprint

View full text Add to dashboard Cite

Extreme events in society and nature, such as pandemic spikes or rogue waves, can have catastrophic consequences. Characterizing extremes is difficult as they occur rarely, arise from seemingly benign conditions, and belong to complex and often unknown infinite-dimensional systems. Such challenges render attempts at characterizing them as moot. We address each of these difficulties by combining novel training schemes in Bayesian experimental design (BED) with an ensemble of deep neural operators (DNOs). This model-agnostic framework pairs a BED scheme that actively selects data for quantifying extreme events with an ensemble of DNOs that approximate infinite-dimensional nonlinear operators. We find that not only does this framework clearly beat Gaussian processes (GPs) but that 1) shallow ensembles of just two members perform best; 2) extremes are uncovered regardless of the state of initial data (i.e. with or without extremes); 3) our method eliminates "double-descent" phenomena; 4) the use of batches of suboptimal acquisition points compared to step-by-step global optima does not hinder BED performance; and 5) Monte Carlo acquisition outperforms standard minimizers in high-dimensions. Together these conclusions form the foundation of an AI-assisted experimental infrastructure that can efficiently infer and pinpoint critical situations across many domains, from physical to societal systems.

show abstract

“…the likelihood ratio assigns to each point in the input space a measure of relevance by weighting how likely that point is to be observed "in the wild" (through the input density p b ) against its expected impact on the magnitude of the output (through the output density p µ ). As such, the likelihood ratio serves as an attention mechanism which steers the algorithm toward the extremes [31,41].…”

Section: Acquisition Functionsmentioning

confidence: 99%

“…specifically target extreme minima of the latent function. As in previous work [31,41,42], we approximate the likelihood ratio with a Gaussian mixture model in order to make the integral in (13) analytic.…”

Section: Acquisition Functionsmentioning

confidence: 99%

Bayesian optimization for active flow control

et al. 2021

View full text Add to dashboard Cite

A key question in flow control is that of the design of optimal controllers when the control space is high-dimensional and the experimental or computational budget is limited. We address this formidable challenge using a particular flavor of machine learning and present the first application of Bayesian optimization to the design of open-loop controllers for fluid flows. We consider a range of acquisition functions, including the recently introduced output-informed criteria of Blanchard and Sapsis (2021), and evaluate performance of the Bayesian algorithm in two iconic configurations for active flow control: computationally, with drag reduction in the fluidic pinball; and experimentally, with mixing enhancement in a turbulent jet. For these flows, we find that Bayesian optimization identifies optimal controllers at a fraction of the cost of other optimization strategies considered in previous studies. Bayesian optimization also provides, as a by-product of the optimization, a surrogate model for the latent cost function, which can be leveraged to paint a complete picture of the control landscape. The proposed methodology can be used to design open-loop controllers

show abstract

Output-Weighted Sampling for Multi-Armed Bandits with Extreme Payoffs

Cited by 4 publications

References 0 publications

Discovering and forecasting extreme events via active learning in neural operators

Discovering and forecasting extreme events via active learning in neural operators

Discovering and forecasting extreme events via active learning in neural operators

Bayesian optimization for active flow control

Contact Info

Product

Resources

About