The frontier of simulation-based inference

Abstract

Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving additional momentum to the field. Finally, we describe how the frontier is expanding so that a broad audience can appreciate the profound influence these developments may have on science.

Classifying evolutionary forces in language change using neural networks

“…Crucially, the quality of the inference depends on whether the statistics are able to sufficiently summarize the observations, but it is often unclear which statistics are capable of doing so. A promising solution to this problem, again, can be found in machine learning algorithms (and in particular, neural networks), which allow us to work with high(er)-dimensional representations of the data, and thus circumvent the problem of summary statistics (Cranmer et al, 2015(Cranmer et al, , 2019Dinev & Gutmann, 2018;Gutmann et al, 2018;Hermans et al, 2019;Papamakarios et al, 2018). One such technique is the application of networks inspired by generative adversarial networks, which are trained to discriminate between data generated by parameter point θ 0 from data simulated with θ 1 (Cranmer et al, 2019;Hermans et al, 2019).…”

“…A promising solution to this problem, again, can be found in machine learning algorithms (and in particular, neural networks), which allow us to work with high(er)-dimensional representations of the data, and thus circumvent the problem of summary statistics (Cranmer et al, 2015(Cranmer et al, , 2019Dinev & Gutmann, 2018;Gutmann et al, 2018;Hermans et al, 2019;Papamakarios et al, 2018). One such technique is the application of networks inspired by generative adversarial networks, which are trained to discriminate between data generated by parameter point θ 0 from data simulated with θ 1 (Cranmer et al, 2019;Hermans et al, 2019). We consider it a fruitful and exciting future line of research to investigate whether these new neural inference techniques can be combined with the neural network of the TSC, in order to improve the detection ofand, by extension, our understanding ofevolutionary forces in language and cultural change.…”

Combining Machine Learning and Simulation to a Hybrid Modelling Approach: Current and Future Directions

“…A prominent integration type of machine learning techniques into simulation is the identification of simpler models, such as surrogate models [11,12,16,26]. These are approximate and cheap to evaluate models that are particularly of interest when the solution of the original, more precise model is very time-or resource-consuming.…”

sbi: A toolkit for simulation-based inference

“…For more general simulators, however, evaluating the likelihood of data given parameters might be computationally intractable. Traditional algorithms for this 'likelihood-free' setting (Cranmer, Brehmer, & Louppe, 2020) are based on Monte-Carlo rejection (Pritchard, Seielstad, Perez-Lezaun, & Feldman, 1999;Sisson, Fan, & Tanaka, 2007), an approach known as Approximate Bayesian Computation (ABC). More recently, algorithms based on neural networks have been developed (Greenberg, Nonnenmacher, & Macke, 2019;Hermans, Begy, & Louppe, 2020;Lueckmann et al, 2017;Papamakarios & Murray, 2016;Papamakarios, Sterratt, & Murray, 2019).…”