Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

Abstract

Bayesian experimental design (BED) is a framework that uses statistical models and decision making under uncertainty to optimise the cost and performance of a scientific experiment. Sequential BED, as opposed to static BED, considers the scenario where we can sequentially update our beliefs about the model parameters through data gathered in the experiment. A class of models of particular interest for the natural and medical sciences are implicit models, where the data generating distribution is intractable, but sampling from it is possible. Even though there has been a lot of work on static BED for implicit models in the past few years, the notoriously difficult problem of sequential BED for implicit models has barely been touched upon. We address this gap in the literature by devising a novel sequential design framework for parameter estimation that uses the Mutual Information (MI) between model parameters and simulated data as a utility function to find optimal experimental designs, which has not been done before for implicit models. Our approach uses likelihood-free inference by ratio estimation to simultaneously estimate posterior distributions and the MI. During the sequential BED procedure we utilise Bayesian optimisation to help us optimise the MI utility. We find that our framework is efficient for the various implicit models tested, yielding accurate parameter estimates after only a few iterations.

Training deep neural density estimators to identify mechanistic models of neural dynamics

“…These results show how SNPE allows fast and accurate identification of biophysical model parameters on new data, and how SNPE can be deployed for applications requiring rapid automated inference, such as high-throughput screening-assays, closed-loop paradigms (e.g. for adaptive experimental manipulations or stimulus-selection [ Kleinegesse and Gutmann, 2019 ]), or interactive software tools.…”

An Information-Theoretic Framework for Optimal Design: Analysis of Protocols for Estimating Soft Tissue Parameters in Biaxial Experiments

“…Figure 4 shows that the best combinations that maximise a balanced criterion, such as I(K 1 ; Z) + I(K 2 ; Z) − I(K 1 ; K 2 |Z), are a trade-off between the combinations of angles that maximise I(K 1 ; Z) and I(K 2 ; Z) individually. For example, when five angles are considered, the angles that maximise I(K 1 ; Z) are φ a = [66, 72, 78, 84, 90] and those that maximise I(K 2 ; Z) are φ b = [30,36,42,48,54], while the combination that maximises I(K 1 ; Z) + I(K 2 ; Z) − I(K 1 ; K 2 |Z) is [30,36,48,78,90], which has two angles from φ a and three angles from φ b . It should be noted that such a trade-off between maximising individual parameter gains is still significantly different to a uniform discretisation; 5.…”

“…Thus, a large area of future work is related to the development of efficient and robust MI and CMI estimators. Note that several approaches are being proposed by researchers to solve this problem; see for example [40][41][42][43][44][45][46]. Lastly, a thorough comparison against classical optimal design methods…”

Stir to Pour: Efficient Calibration of Liquid Properties for Pouring Actions

“…Our work is limited to studying the effect of a fixed set of stirring actions that were selected after careful analysis in simulation. We believe an interesting avenue for future work lies in studying how such actions can be generated automatically using information-based metrics [36], [37], such as maximizing the information gain after each stir. Another interesting direction is to relax the current assumption of knowledge about the shapes and containers and including uncertainty in the estimation of the stick configuration.…”