Randomized Multilevel Monte Carlo for Embarrassingly Parallel Inference

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalising constant, and where even the un-normalised target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics-based model governed by an intractable partial differential equation (PDE) appears in the likelihood. A multi-index sequential Monte Carlo (MISMC) method is used to construct ratio estimators which provably enjoy the complexity improvements of multi-index Monte Carlo (MIMC) as well as the efficiency of sequential Monte Carlo (SMC) for inference. In particular, the proposed method provably achieves the canonical complexity of $$\hbox {MSE}^{-1}$$ MSE - 1 , while single-level methods require $$\hbox {MSE}^{-\xi }$$ MSE - ξ for $$\xi >1$$ ξ > 1 . This is illustrated on examples of Bayesian inverse problems with an elliptic PDE forward model in 1 and 2 spatial dimensions, where $$\xi =5/4$$ ξ = 5 / 4 and $$\xi =3/2$$ ξ = 3 / 2 , respectively. It is also illustrated on more challenging log-Gaussian process models, where single-level complexity is approximately $$\xi =9/4$$ ξ = 9 / 4 and multilevel Monte Carlo (or MIMC with an inappropriate index set) gives $$\xi = 5/4 + \omega $$ ξ = 5 / 4 + ω , for any $$\omega > 0$$ ω > 0 , whereas our method is again canonical. We also provide novel theoretical verification of the product-form convergence results which MIMC requires for Gaussian processes built in spaces of mixed regularity defined in the spectral domain, which facilitates acceleration with fast Fourier transform methods via a cumulant embedding strategy, and may be of independent interest in the context of spatial statistics and machine learning.

show abstract

Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

Jasra

Law

Walton

et al. 2023

Found Comput Math

View full text Add to dashboard Cite

show abstract

Bayesian Estimation of Oscillator Parameters: Toward Anomaly Detection and Cyber-Physical System Security

Lukens

Passian

Yoginath

et al. 2022

Sensors

View full text Add to dashboard Cite

Cyber-physical system security presents unique challenges to conventional measurement science and technology. Anomaly detection in software-assisted physical systems, such as those employed in additive manufacturing or in DNA synthesis, is often hampered by the limited available parameter space of the underlying mechanism that is transducing the anomaly. As a result, the formulation of anomaly detection for such systems often leads to inverse or ill-posed problems, requiring statistical treatments. Here, we present Bayesian inference of unknown parameters associated with a generic actuator considered as a representative vital element of a cyber-physical system. Via a series of experimental input-output measurements, a transfer function for the actuator is obtained numerically, which serves as our model for the proposed method. Linear, nonlinear, and delayed dynamics may be assumed for the actuator response. By devising a code-based malicious signal, we study the efficacy of Bayesian inference for its potential to produce a detection, including uncertainty quantification, with a remarkably small number of input data points. Our approach should be adaptable to a variety of real-time cyber-physical anomaly detection scenarios.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Randomized Multilevel Monte Carlo for Embarrassingly Parallel Inference

Cited by 2 publications

References 42 publications

Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

Bayesian Estimation of Oscillator Parameters: Toward Anomaly Detection and Cyber-Physical System Security

Contact Info

Product

Resources

About