An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants

Liang, Faming; Jin, Ick Hoon; Song, Qifan; Liu, Jun S.

doi:10.1080/01621459.2015.1009072

Cited by 44 publications

(75 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, we found that a short run of the double Metropolis-Hastings (DMH) (Liang, 2010) was useful in providing particles. This was an approach also used in Liang et al (2016). The DMH Algorithm may be described as follows:…”

Section: Pre-mcmc Details For the Function Emulation Algorithmsmentioning

confidence: 99%

“…Although the DMH algorithm is asymptotically inexact, by increasing the number of iterations for the birth-death MCMC, the algorithm can provide more accurate results (Liang et al, 2016).…”

Section: An Attraction-repulsion Point Process Modelmentioning

confidence: 99%

“…Asymptotically exact algorithms are those where the Markov chain's stationary distribution is equal to the desired posterior distribution. Examples of elegant asymptotically exact methods for this challenging problem include Møller et al (2006), Murray et al (2006), Atchade et al (2008), Lyne et al (2015), Liang et al (2016). While some require the ability to draw independent samples from the probability model, others are complicated to construct and require users to tune the algorithm carefully.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Function Emulation Approach for Doubly Intractable Distributions

Park

Haran

2019

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

Doubly intractable distributions arise in many settings, for example in Markov models for point processes and exponential random graph models for networks. Bayesian inference for these models is challenging because they involve intractable normalising "constants" that are actually functions of the parameters of interest. Although several computational methods have been developed for these models, each can be computationally burdensome or even infeasible for many problems. We propose a novel algorithm that provides computational gains over existing methods by replacing Monte Carlo approximations to the normalising function with a Gaussian process-based approximation. We provide theoretical justification for this method. We also develop a closely related algorithm that is applicable more broadly to any likelihood function that is expensive to evaluate. We illustrate the application of our methods to challenging simulated and real data examples, including an exponential random graph model, a Markov point process, and a model for infectious disease dynamics. The algorithm shows significant gains in computational efficiency over existing methods, and has the potential for greater gains for more challenging problems. For a random graph model example, we show how this gain in efficiency allows us to carry out accurate Bayesian inference when other algorithms are computationally impractical.Gaussian processes have been widely used for interpolation in spatial statistics (Krige, 1951, Cressie, 2015, as well as in "computer model emulation", to approximate the relationship between input parameters and the output of a complex computer model (cf. Sacks et al., 1989, Kennedy andO'Hagan, 2001). We show how Gaussian processes are very effective in our twostage approximation, and how our method may be useful in addressing inferential challenges for doubly intractable distributions. We also describe a second algorithm that is applicable in principle to a much wider class of problems -Bayesian inference when the likelihood function (not just its normalising function) is difficult to evaluate.The outline of the remainder of this paper is as follows. In Section 2 we describe existing Bayesian algorithms for intractable normalising functions and discuss their computational chal-

show abstract

Section: Pre-mcmc Details For the Function Emulation Algorithmsmentioning

confidence: 99%

“…Although the DMH algorithm is asymptotically inexact, by increasing the number of iterations for the birth-death MCMC, the algorithm can provide more accurate results (Liang et al, 2016).…”

Section: An Attraction-repulsion Point Process Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Function Emulation Approach for Doubly Intractable Distributions

Park

Haran

2019

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

show abstract

“…As an exact evaluation of Z ( θ ) needs to sum over the entire space of x , which consists of 3 N different elements with N denoting the total number of spins, Z ( θ ) is intractable even for a small size model, say N = 100. How to estimate the parameters for such a model has been studied in the recent literature 9,12,21–23…”

Section: Potts Modelmentioning

confidence: 99%

Lung Cancer Pathological Image Analysis Using a Hidden Potts Model

Wang

et al. 2017

Cancer Inform

Self Cite

View full text Add to dashboard Cite

Nowadays, many biological data are acquired via images. In this article, we study the pathological images scanned from 205 patients with lung cancer with the goal to find out the relationship between the survival time and the spatial distribution of different types of cells, including lymphocyte, stroma, and tumor cells. Toward this goal, we model the spatial distribution of different types of cells using a modified Potts model for which the parameters represent interactions between different types of cells and estimate the parameters of the Potts model using the double Metropolis-Hastings algorithm. The double Metropolis-Hastings algorithm allows us to simulate samples approximately from a distribution with an intractable normalizing constant. Our numerical results indicate that the spatial interaction between the lymphocyte and tumor cells is significantly associated with the patient’s survival time, and it can be used together with the cell count information to predict the survival of the patients.

show abstract

“…For most models efficient sampling routines are readily available in standard statistical software such as R [19], or can be constructed using general procedures [23, 24]. Extensions of the SVE algorithm where data are sampled using a Markov chain have also been considered [25, 26], and although not investigated here, we anticipate that our approach also extends in this direction. The more general notion is this: if the problem of simulating data is solved, the SVE algorithm turns data simulation into parameter estimation by producing draws from the posterior distribution.…”

Section: Introductionmentioning

confidence: 99%

Turning Simulation into Estimation: Generalized Exchange Algorithms for Exponential Family Models

et al. 2017

View full text Add to dashboard Cite

The Single Variable Exchange algorithm is based on a simple idea; any model that can be simulated can be estimated by producing draws from the posterior distribution. We build on this simple idea by framing the Exchange algorithm as a mixture of Metropolis transition kernels and propose strategies that automatically select the more efficient transition kernels. In this manner we achieve significant improvements in convergence rate and autocorrelation of the Markov chain without relying on more than being able to simulate from the model. Our focus will be on statistical models in the Exponential Family and use two simple models from educational measurement to illustrate the contribution.

show abstract

An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants

Cited by 44 publications

References 42 publications

A Function Emulation Approach for Doubly Intractable Distributions

A Function Emulation Approach for Doubly Intractable Distributions

Lung Cancer Pathological Image Analysis Using a Hidden Potts Model

Turning Simulation into Estimation: Generalized Exchange Algorithms for Exponential Family Models

Contact Info

Product

Resources

About