A fully Bayesian approach to sparse reduced-rank multivariate regression

Abstract

In the context of high-dimensional multivariate linear regression, sparse reduced-rank regression (SRRR) provides a way to handle both variable selection and low-rank estimation problems. Although there has been extensive research on SRRR, statistical inference procedures that deal with the uncertainty due to variable selection and rank reduction are still limited. To fill this research gap, we develop a fully Bayesian approach to SRRR. A major difficulty that occurs in a fully Bayesian framework is that the dimension of parameter space varies with the selected variables and the reduced-rank. Due to the varying-dimensional problems, traditional Markov chain Monte Carlo (MCMC) methods such as Gibbs sampler and Metropolis-Hastings algorithm are inapplicable in our Bayesian framework. To address this issue, we propose a new posterior computation procedure based on the Laplace approximation within the collapsed Gibbs sampler. A key feature of our fully Bayesian method is that the model uncertainty is automatically integrated out by the proposed MCMC computation. The proposed method is examined via simulation study and real data analysis.

Advanced Bayesian regression and dynamic network modelling

“…In particular for large sample sizes T one could thus resort to the M 1,0 model, which does not require the error DAG to be learned. In the same vein, to substantially reduce the configuration space of possible models, one could adopt the idea of [71] and impose the restriction that all responses share the same covariate set rather than allowing for response-specific covariate sets.…”