Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables

Damien, Paul; Wakefield, Jon; Walker, Stephen G.

doi:10.1111/1467-9868.00179

Cited by 311 publications

(227 citation statements)

References 28 publications

Supporting

Mentioning

224

Contrasting

Unclassified

Order By: Relevance

“…We suspect that the Bactrian kernel can be improved further, because it is based on the simple intuition to reduce the autocorrelation. We also note that alternatives to MH algorithms have been suggested, such as the slice sampler (15) and Langevin diffusion, which requires derivatives of the posterior density (16). The efficiency of good MH algorithms relative to the MH alternatives will be interesting topics for future research.…”

Section: Resultsmentioning

confidence: 98%

Searching for efficient Markov chain Monte Carlo proposal kernels

Yang

Rodríguez

2013

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Markov chain Monte Carlo (MCMC) or the Metropolis-Hastings algorithm is a simulation algorithm that has made modern Bayesian statistical inference possible. Nevertheless, the efficiency of different Metropolis-Hastings proposal kernels has rarely been studied except for the Gaussian proposal. Here we propose a unique class of Bactrian kernels, which avoid proposing values that are very close to the current value, and compare their efficiency with a number of proposals for simulating different target distributions, with efficiency measured by the asymptotic variance of a parameter estimate. The uniform kernel is found to be more efficient than the Gaussian kernel, whereas the Bactrian kernel is even better. When optimal scales are used for both, the Bactrian kernel is at least 50% more efficient than the Gaussian. Implementation in a Bayesian program for molecular clock dating confirms the general applicability of our results to generic MCMC algorithms. Our results refute a previous claim that all proposals had nearly identical performance and will prompt further research into efficient MCMC proposals.Bayesian inference | mixing | convergence rate M arkov chain Monte Carlo (MCMC) algorithms can be used to simulate a probability distribution π(x) that is known only up to a factor, that is, with only πðyÞ πðxÞ known; they are especially important in Bayesian inference where π is the posterior distribution. In a Metropolis-Hastings (MH) algorithm (1, 2), a proposal density q(yjx), with x, y ∈ χ, is used to generate a new state y given the current state x. The proposal is accepted with probability α(x, y). If the proposal is accepted, the new state becomes y; otherwise it stays at x. The algorithm generates a discrete-time Markov chain with state space χ and transition law P having transition probability density pðx; yÞ = qð yjxÞ · αðx; yÞ; y ≠ x; 1 − Z χ qðyjxÞ · αðx; yÞdy; y = x:The acceptance probability α is chosen so that the detailed balance condition is satisfied: π(x)p(x, y) = π(y)p(y, x), for all x, y ∈ χ. The MH choice of α is αðx; yÞ = min & 1; πðyÞ πðxÞ × qðxjyÞ qðyjxÞThe proposal kernel q(yjx) can be very general; as long as it specifies an irreducible aperiodic Markov chain, the algorithm will generate a reversible Markov chain with stationary distribution π. Here we assume that χ is a subset of R k and both π(y) and q(yjx) are densities on χ.Given a sample x 1 , x 2 , . . ., x n simulated from P, the expectation of any function f(x) over πcan be approximated by the time average over the sampleĨThis is an unbiased estimate of I, and converges to I according to the central limit theorem, independent of the initial state x 0 (ref. 3, p. 99). The asymptotic variance ofĨ iswhere V f = var π {f(X)} is the variance of f(X) over π, ρ k = corr{f(x i ), f(x i + k )} is the lag-k autocorrelation (ref. 4, pp. 87-92), and the variance ratio E = V f /ν is the efficiency: an MCMC sample of size N is as informative about I as an independent sample of size NE. Thus, NE is known as the effective sample size. Given π an...

show abstract

Section: Resultsmentioning

confidence: 98%

Searching for efficient Markov chain Monte Carlo proposal kernels

Yang

Rodríguez

2013

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…It would seem that infinite values should be sampled at each step of the MCMC algorithm. However, this will not be necessary following the ideas proposed in Walker (2007), which are based on slice sampling schemes (Damien, Wakefield and Walker, 1999).…”

Section: Bayesian Inference For the Garch-dpm Modelmentioning

confidence: 99%

A semiparametric Bayesian approach to the analysis of financial time series with applications to value at risk estimation

Ausín

Galeano

Ghosh

2014

European Journal of Operational Research

View full text Add to dashboard Cite

Financial time series analysis deals with the understanding of data collected on financial markets. Several parametric distribution models have been entertained for describing, estimating and predicting the dynamics of financial time series. Alternatively, this article considers a Bayesian semiparametric approach. In particular, the usual parametric distributional assumptions of the GARCH-type models are relaxed by entertaining the class of location-scale mixtures of Gaussian distributions with a Dirichlet process prior on the mixing distribution, leading to a Dirichlet process mixture model. The proposed specification allows for a greater exibility in capturing both the skewness and kurtosis frequently observed in financial returns. The Bayesian model provides statistical inference with finite sample validity. Furthermore, it is also possible to obtain predictive distributions for the Value at Risk (VaR), which has become the most widely used measure of market risk for practitioners. Through a simulation study, we demonstrate the performance of the proposed semiparametric method and compare results with the ones from a normal distribution assumption. We also demonstrate the superiority of our proposed semiparametric method using real data from the Bombay Stock Exchange Index (BSE-30) and the Hang Seng Index (HSI). In particular, the usual parametric distributional assumptions of the GARCH-type models are relaxed by entertaining the class of location-scale mixtures of Gaussian distributions with a Dirichlet process prior on the mixing distribution, leading to a Dirichlet process mixture model. The proposed specification allows for a greater flexibility in capturing both the skewness and kurtosis frequently observed in financial returns. The Bayesian model provide statistical inference with finite sample validity. Furthermore, it is also possible to obtain predictive distributions for the Value at Risk (VaR), which has become the most widely used measure of market risk for practitioners. Through a simulation study, we demonstrate the performance of the proposed semiparametric method and compare results with the ones from a normal distribution assumption. We also demonstrate the superiority of our proposed semiparametric method using real data from the Bombay Stock Exchange Index (BSE-30) and the Hang Seng Index (HSI).

show abstract

“…As fully described in the appendix, we fit this model using Gibbs sampling coupled with the Metropolis-Hastings algorithm and auxiliary variable Gibbs [e.g., Damien et al (1999)]. …”

Section: Priors and The Joint Posteriormentioning

confidence: 99%

“…Since these quantities are not easily calculated analytically, given the rather messy form of (11), it seems desirable to pursue a numerical alterative by obtaining draws from (11) and using these draws to calculate any statistic of interest. However, (11) is not of an immediately recognizable functional form, thus calling into question the feasibility of this numerical scheme.…”

Section: Willingness To Paymentioning

confidence: 99%

What are the consequences of consequentiality?

Herriges

Kling

Liu

et al. 2010

Journal of Environmental Economics and Management

150

101

View full text Add to dashboard Cite

We offer an empirical test of a theoretical result in the contingent valuation literature. Specifically, it has been argued from a theoretical point of view that survey participants who perceive a survey to be "consequential" will respond to questions truthfully regardless of the degree of perceived consequentiality. Using survey data from the Iowa Lakes Project, we test this supposition. Specifically, we employ a Bayesian treatment effect model in which the degree of perceived consequentiality, measured as an ordinal response, is permitted to have a structural impact on willingness to pay (WTP) for a hypothetical environmental improvement. We test our theory by determining if the WTP distributions are the same for each value of the ordinal response.In our survey data, a subsample of individuals were randomly assigned supporting information suggesting that their responses to the questionnaires were important and will have an impact on policy decisions. In conjunction with a Bayesian posterior simulator, we use this source of exogenous variation to identify the structural impacts of consequentiality perceptions on willingness to pay, while controlling for the potential of confounding on unobservables. We find evidence consistent with the "knife-edge" theoretical results, namely that the willingness to pay distributions are equal among those believing the survey to be at least minimally consequential, and different for those believing that the survey is irrelevant for policy purposes.

show abstract

Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables

Cited by 311 publications

References 28 publications

Searching for efficient Markov chain Monte Carlo proposal kernels

Searching for efficient Markov chain Monte Carlo proposal kernels

A semiparametric Bayesian approach to the analysis of financial time series with applications to value at risk estimation

What are the consequences of consequentiality?

Contact Info

Product

Resources

About