Likelihood-Free Inference with Deep Gaussian Processes

Aushev, Alexander; Pesonen, Henri; Heinonen, Markus; Corander, Jukka; Kaski, Samuel

doi:10.48550/arxiv.2006.10571

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our benchmark, in its current form, has several limitations. First, the algorithms considered here do not cover the entire spectrum of SBI algorithms: We did not include sequential algorithms using active learning or Bayesian Optimization Aushev et al, 2020), or 'gray-box' algorithms, which use additional information about or from the simulator (e.g., . We focused on approaches using neural networks for density estimation and did not compare to alternatives using Gaussian Processes (e.g., Meeds and Welling, 2014;Wilkinson, 2014).…”

Section: Limitationsmentioning

confidence: 99%

Benchmarking Simulation-Based Inference

Lueckmann,

Boelts,

Greenberg

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in probabilistic modelling have led to a large number of simulation-based inference algorithms which do not require numerical evaluation of likelihoods. However, a public benchmark with appropriate performance metrics for such 'likelihood-free' algorithms has been lacking. This has made it difficult to compare algorithms and identify their strengths and weaknesses. We set out to fill this gap: We provide a benchmark with inference tasks and suitable performance metrics, with an initial selection of algorithms including recent approaches employing neural networks and classical Approximate Bayesian Computation methods. We found that the choice of performance metric is critical, that even state-of-the-art algorithms have substantial room for improvement, and that sequential estimation improves sample efficiency. Neural network-based approaches generally exhibit better performance, but there is no uniformly best algorithm. We provide practical advice and highlight the potential of the benchmark to diagnose problems and improve algorithms. The results can be explored interactively on a companion website.All code is open source, making it possible to contribute further benchmark tasks and inference algorithms.

show abstract

Section: Limitationsmentioning

confidence: 99%

Benchmarking Simulation-Based Inference

Lueckmann,

Boelts,

Greenberg

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The price is that they require significantly more simulations than Bayesian optimisation (BO) approaches, as seen in Sect. 4.3 of Aushev et al (2020).…”

Section: Sequential Neural Estimationmentioning

confidence: 98%

Likelihood-free inference in state-space models with unknown dynamics

Aushev,

Tran,

Pesonen

et al. 2023

Stat Comput

View full text Add to dashboard Cite

Likelihood-free inference (LFI) has been successfully applied to state-space models, where the likelihood of observations is not available but synthetic observations generated by a black-box simulator can be used for inference instead. However, much of the research up to now has been restricted to cases in which a model of state transition dynamics can be formulated in advance and the simulation budget is unrestricted. These methods fail to address the problem of state inference when simulations are computationally expensive and the Markovian state transition dynamics are undefined. The approach proposed in this manuscript enables LFI of states with a limited number of simulations by estimating the transition dynamics and using state predictions as proposals for simulations. In the experiments with non-stationary user models, the proposed method demonstrates significant improvement in accuracy for both state inference and prediction, where a multi-output Gaussian process is used for LFI of states and a Bayesian neural network as a surrogate model of transition dynamics.

show abstract

“…BO finds applications in various scientific and industrial domains, e.g., machine learning for hyperparameter optimization [37], [38], modeling of population genetics [39], spreading of pathogens [40], atomic structure of materials [41], [42], as well as cosmology [43], and establishes as a state-of-the-art method in lower-dimensional problems [17], [44]. However, the BO performances and its computational efficiency decline as the dimensionality of a problem increases [37], [45]- [47], which is the case with the calibration of large-scale ABM that features a large number of behavioral parameters to be tuned.…”

Section: B Bayesian Optimizationmentioning

confidence: 99%

A Bayesian Optimization Approach for Calibrating Large-Scale Activity-Based Transport Models

Agriesti,

Kuzmanovski,

Hollmén

et al. 2023

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

Addressing complexity in transportation in cases such as disruptive trends or disaggregated management strategies has become increasingly important. This in turn is resulting in the rising adoption of Agent-Based and Activity-Based modeling. Still, a broad adoption is hindered by the high complexity and computational needs. For example, hundreds of parameters are involved in the calibration of Activity-Based models focused on behavioral theory, to properly frame the required detailed socio-economical characteristics. To address this challenge, this paper presents a novel Bayesian Optimization approach that incorporates a surrogate model defined as an improved Random Forest to automate the calibration process of the behavioral parameters. The presented solution calibrates the largest set of parameters yet, according to the literature, by combining state-of-the-art methods. To the best of the authors' knowledge, this is the first work in which such a high dimensionality is tackled in sequential model-based algorithm configuration theory. The proposed method is tested in the city of Tallinn, Estonia, for which the calibration of 477 behavioral parameters is carried out. The calibration process results in a satisfactory performance for all the major indicators, the OD matrix average mismatch is equal to 15.92 vehicles per day while the error for the overall number of trips is equal to 4%.

show abstract

Likelihood-Free Inference with Deep Gaussian Processes

Cited by 3 publications

References 0 publications

Benchmarking Simulation-Based Inference

Benchmarking Simulation-Based Inference

Likelihood-free inference in state-space models with unknown dynamics

A Bayesian Optimization Approach for Calibrating Large-Scale Activity-Based Transport Models

Contact Info

Product

Resources

About