This paper proposes a new approach to sparsity called the horseshoe estimator. The horseshoe is a close cousin of other widely used Bayes rules arising from, for example, double-exponential and Cauchy priors, in that it is a member of the same family of multivariate scale mixtures of normals. But the horseshoe enjoys a number of advantages over existing approaches, including its robustness, its adaptivity to different sparsity patterns, and its analytical tractability. We prove two theorems that formally characterize both the horseshoe's adeptness at large outlying signals, and its super-efficient rate of convergence to the correct estimate of the sampling density in sparse situations. Finally, using a combination of real and simulated data, we show that the horseshoe estimator corresponds quite closely to the answers one would get by pursuing a full Bayesian model-averaging approach using a discrete mixture prior to model signals and noise.

We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Pólya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negative binomial regression, nonlinear mixed-effects models, and spatial models for count data. In each case, our data-augmentation strategy leads to simple, effective methods for posterior inference that: (1) circumvent the need for analytic approximations, numerical integration, or Metropolis-Hastings; and (2) outperform other known data-augmentation strategies, both in ease of use and in computational efficiency. All methods, including an efficient sampler for the Pólya-Gamma distribution, are implemented in the R package BayesLogit.In the technical supplement appended to the end of the paper, we provide further details regarding the generation of Pólya-Gamma random variables; the empirical benchmarks reported in the main manuscript; and the extension of the basic dataaugmentation framework to contingency tables and multinomial outcomes.

This paper studies the multiplicity-correction effect of standard Bayesian variable-selection priors in linear regression. Our first goal is to clarify when, and how, multiplicity correction happens automatically in Bayesian analysis, and to distinguish this correction from the Bayesian Ockham's-razor effect. Our second goal is to contrast empirical-Bayes and fully Bayesian approaches to variable selection through examples, theoretical results and simulations. Considerable differences between the two approaches are found. In particular, we prove a theorem that characterizes a surprising aymptotic discrepancy between fully Bayes and empirical Bayes. This discrepancy arises from a different source than the failure to account for hyperparameter uncertainty in the empirical-Bayes estimate. Indeed, even at the extreme, when the empirical-Bayes estimate converges asymptotically to the true variable-inclusion probability, the potential for a serious difference remains.to generate insights about complex, high-dimensional systems. Still, the results of such studies are often used to buttress scientific conclusions or guide policy decisions-conclusions or decisions that may be quite wrong if the implicit multiple-testing problem is ignored.Our first objective is to clarify how multiplicity correction enters Bayesian variable selection: by allowing the choice of prior model probabilities to depend upon the data in an appropriate way. Some useful references on this idea include Waller and Duncan (. We also clarify the difference between multiplicity correction and the Bayesian Ockham's-razor effect [see Jefferys and Berger (1992)], which induces a very different type of penalty on model complexity. This discussion will highlight the fact that not all Bayesian analyses automatically adjust for multiplicity.Our second objective is to describe and investigate a peculiar discrepancy between fully Bayes and empirical-Bayes variable selection. This discrepancy seems to arise from a different source than the failure to account for uncertainty in the empirical-Bayes estimate-the usual issue in such problems. Indeed, even when the empirical-Bayes estimate converges asymptotically to the true hyperparameter value, the potential for a serious difference remains.The existence of such a discrepancy between fully Bayesian answers and empirical-Bayes answers-especially one that persists even in the limitis of immediate interest to Bayesians, who often use empirical Bayes as a computational simplification. But the discrepancy is also of interest to non-Bayesians for at least two reasons.First, frequentist complete-class theorems suggest that if an empirical-Bayes analysis does not approximate some fully Bayesian analysis, then it may be suboptimal and needs alternative justification. Such justifications can be found for a variety .Second, theoretical and numerical investigations of the discrepancy revealed some unsettling properties of the standard empirical-Bayes analysis in variable selection. Of most concern is that empirical Bayes has t...

We study the classic problem of choosing a prior distribution for a location parameter β = (β 1 ,. .. , βp) as p grows large. First, we study the standard "global-local shrinkage" approach, based on scale mixtures of normals. Two theorems are presented which characterize certain desirable properties of shrinkage priors for sparse problems. Next, we review some recent results showing how Lévy processes can be used to generate infinite-dimensional versions of standard normal scale-mixture priors, along with new priors that have yet to be seriously studied in the literature. This approach provides an intuitive framework both for generating new regularization penalties and shrinkage rules, and for performing asymptotic analysis on existing models.

This paper argues that the half-Cauchy distribution should replace the inverse-Gamma distribution as a default prior for a top-level scale parameter in Bayesian hierarchical models, at least for cases where a proper prior is necessary. Our arguments involve a blend of Bayesian and frequentist reasoning, and are intended to complement the original case made by Gelman (2006) in support of the folded-t family of priors. First, we generalize the half-Cauchy prior to the wider class of hypergeometric inverted-beta priors. We derive expressions for posterior moments and marginal densities when these priors are used for a top-level normal variance in a Bayesian hierarchical model. We go on to prove a proposition that, together with the results for moments and marginals, allows us to characterize the frequentist risk of the Bayes estimators under all global-shrinkage priors in the class. These theoretical results, in turn, allow us to study the frequentist properties of the half-Cauchy prior versus a wide class of alternatives. The half-Cauchy occupies a sensible "middle ground" within this class: it performs very well near the origin, but does not lead to drastic compromises in other parts of the parameter space. This provides an alternative, classical justification for the repeated, routine use of this prior. We also consider situations where the underlying mean vector is sparse, where we argue that the usual conjugate choice of an inverse-gamma prior is particularly inappropriate, and can lead to highly distorted posterior inferences. Finally, we briefly summarize some open issues in the specification of default priors for scale terms in hierarchical models.