Multivariate Bayesian Variable Selection and Prediction

Abstract

The multivariate regression model is considered with p regressors. A latent vector with p binary entries serves to identify one of two types of regression coef®cients: those close to 0 and those not. Specializing our general distributional setting to the linear model with Gaussian errors and using natural conjugate prior distributions, we derive the marginal posterior distribution of the binary latent vector. Fast algorithms aid its direct computation, and in high dimensions these are supplemented by a Markov chain Monte Carlo approach to sampling from the known posterior distribution. Problems with hundreds of regressor variables become quite feasible. We give a simple method of assigning the hyperparameters of the prior distribution. The posterior predictive distribution is derived and the approach illustrated on compositional analysis of data involving three sugars with 160 near infra-red absorbances as regressors.

Bayesian variable selection for Gaussian process regression: Application to chemometric calibration of spectrometers

“…The central component of the M-H algorithm is to propose a random move from the current sample, and the proposal is accepted with a certain probability such that the algorithm converges to the target distribution. Following [27,26], the random move, c * , is proposed using the "birth" and "death" steps from the current model, c:…”

“…, K}, each associated with a vector of unknown parameters, θ k . The task of model selection is to identify which model is more probable given the data D. Within a Bayesian framework, this is equivalent to assessing the following probability for each of the K models [26,27,36,37]:…”

“…One example is in linear regression where p(θ k |M k ) is a conjugate prior to the likelihood function [26]. In general this equation must be approximated.…”

“…Due to the large number of candidate models, the algorithms were run for 10000 iterations to enable adequate exploration of the model space. Although this number of iterations is only capable of assessing a small proportion of all the 2 404 ≈ 4.13 × 10 121 candidate models, the MCMC methods tend to concentrate on regions of high posterior probability, and thus satisfactory results are expected [26]. Fig.…”

“…Initially the Gibbs sampling algorithm was proposed to sample the posterior of the indicators [24,25]. This was followed by its extension to multiple responses [26], and its implementation using the Metropolis-Hastings sampling algorithm [27]. [28] reviewed and compared several MCMC methods for variable selection.…”