An efficient computational approach for prior sensitivity analysis and cross‐validation

Abstract

Prior sensitivity analysis and cross-validation are important tools in Bayesian statistics. However, due to the computational expense of implementing existing methods, these techniques are rarely used. In this paper, the authors show how it is possible to use sequential Monte Carlo methods to create an efficient and automated algorithm to perform these tasks. They apply the algorithm to the computation of regularization path plots and to assess the sensitivity of the tuning parameter in g-prior model selection. They then demonstrate the algorithm in a cross-validation context and use it to select the shrinkage parameter in Bayesian regression. The Canadian Journal of Statistics © 2010 Statistical Society of Canada Résumé: La sensibilitéà la loi a priori et la validation croisée sont des outils importants des statistiques bayésiennes. Toutefois, ces techniques sont rarement utilisées en pratique car les méthodes disponibles pour les implémenter sont numériquement très coûteuses. Dans ce papier, les auteurs montrent comment il est possible d'utiliser les méthodes de Monte Carlo séquentielles pour obtenir un algorithme efficace et automatique pour implémenter ces techniques. Ils appliquent cet algorithme au calcul des chemins de régularisation pour un problème de régression età la sensibilité du paramètre de la loi a priori de Zellner pour un problème de sélection de variables. Ils appliquent ensuite cet algorithme pour la validation croisée et l'utilisent afin de sélectionner le paramètre de régularisation dans un problème de régression bayésienne. La revue canadienne de statistique

Inferring metabolic networks using the Bayesian adaptive graphical lasso with informative priors

“…Park and Casella [39] explore this setting, demonstrating that as the shrinkage parameter λ is increased, the Bayesian lasso coefficient estimates tend to zero more slowly than under the original version of the lasso, but that for appropriately chosen penalty parameters the posterior median estimates are very close to those from the original lasso. The Bayesian version of the elastic net uses a combination of double exponential and normal priors to capture the L 1 and L 2 penalty terms of the original elastic net [5, 30]. The Bayesian adaptive lasso avoids over-penalization of large effects through a prior formulation which allows the scale parameter to vary across coefficients [19].…”

Bayesian clustering in decomposable graphs

“…Because of its relation to product partition models (described later), we term this prior the product graphical model prior. To clearly demonstrate the control the product graphical model prior (5) gives relative to the binomial prior, we set b = 1/1000, highly penalizing the number of separators and hence resulting in highly separated cliques. In addition, we look at two different values for a; a = 0.1, resulting in fewer and larger cliques, and a = 10, resulting in more (but smaller) cliques.…”

“…Figure 3 shows the 4 graphs with Figure 3. In contrast, the separation of cliques from the product graphical model prior (5) would allow these crops to be planted together. Such decisions could be made from the highest posterior graph, or by conducting Bayesian model averaging to obtain the expected utility of a given decision.…”

On consistency and optimality of Bayesian variable selection based on $$g$$ g -prior in normal linear regression models

“…It may be noted that the prior covariance matrix is related to the Fisher information matrix in the linear model. This prior and its variants have been widely used in the literature in linear models; see, for example, Zellner (1986), Chaturvedi et al (1997), Fernández et al (2001), Consonni and Veronese (2008), Krishna et al (2009) and Bornn et al (2010).…”