Sampling Errors in Nested Sampling Parameter Estimation

Abstract

Sampling errors in nested sampling parameter estimation differ from those in Bayesian evidence calculation, but have been little studied in the literature. This paper provides the first explanation of the two main sources of sampling errors in nested sampling parameter estimation, and presents a new diagrammatic representation for the process. We find no current method can accurately measure the parameter estimation errors of a single nested sampling run, and propose a method for doing so using a new algorithm for dividing nested sampling runs. We empirically verify our conclusions and the accuracy of our new method.Comment: Very minor changes. 22 pages + appendix, 10 figures. Accepted by Bayesian Analysi

nestcheck: diagnostic tests for nested sampling calculations

“…This section provides a brief overview of the nested sampling algorithm and the sampling errors involved in the process -0 termination direction of iteration mean step size ≈ 1/n log X L(X)X L(X) samples Figure 1. Illustration of nested sampling with a constant number of live points n (reproduced from Higson et al 2018). The algorithm samples an exponentially shrinking fraction of the prior X as it moves towards increasing likelihoods.…”

“…for more details see Higson et al (2018). A comparison of nested sampling with other sampling methods is beyond of the scope of this paper; for this we refer the reader to Allison & Dunkley (2014) and Murray (2007).…”

“…Methods for numerically estimating the uncertainty in nested sampling results due to the stochasticity of the nested sampling algorithm are now available for both evidence calculations (see Skilling 2006;Keeton 2011) and parameter estimation (see Higson et al 2018). However, all of these techniques assume that the nested sampling algorithm was executed perfectly -which requires sampling randomly from the prior within a hard likelihood constraint.…”

Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learning

“…Here the first term is the ratio of the estimated variance of the results of repeated calculations using the vanilla method and the alternative method; the second term is the ratio of the mean number of samples from the nested sampling runs using each method. Numerical results for the efficiency gains from the different methods are show in Table C4 in Appendix C. These use estimates of the variance of results calculated using the bootstrap resampling method described in Higson et al (2018b), which avoids the need to compute large numbers of nested sampling runs but also does not include additional errors from the sampler failing to explore the parameter space fully (see Higson et al 2018a, for a detailed discussion of such errors). As described in Appendix C, we find that the sampler is not able to explore the parameter space perfectly with the settings used, meaning the true variance of results is higher than the bootstrap estimates.…”

“…The adaptive method calculates posterior odds ratios indirectly by sampling the integer parameters T and N, and as a result its sampling errors have different properties to those of the vanilla method (which uses evidence calculations). For a detailed discussion of sampling errors in nested sampling parameter estimation, see Higson et al (2018b). Nested sampling calculations can be made significantly more computationally efficient (or alternatively more accurate for the same amount of computation) using dynamic nested sampling (Higson et al 2017).…”