Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models

Abstract

Structured additive regression provides a general framework for complexGaussian and non-Gaussian regression models, with predictors comprising arbitrary combinations of nonlinear functions and surfaces, spatial effects, varying coefficients, random effects and further regression terms. The large flexibility of structured additive regression makes function selection a challenging and important task, aiming at (1) selecting the relevant covariates, (2) choosing an appropriate and parsimonious representation of the impact of covariates on the predictor and (3) determining the required interactions. We propose a spike-and-slab prior structure for function selection that allows to include or exclude single coefficients as well as blocks of coefficients representing specific model terms. A novel multiplicative parameter expansion is required to obtain good mixing and convergence properties in a Markov chain Monte Carlo simulation approach and is shown to induce desirable shrinkage properties. In simulation studies and with (real) benchmark classification data, we investigate sensitivity to hyperparameter settings and compare performance to competitors. The flexibility and applicability of our approach are demonstrated in an additive piecewise exponential model with time-varying effects for right-censored survival times of intensive care patients with sepsis.Geoadditive and additive mixed logit model applications are discussed in an extensive appendix.

Bayesian Generalized Additive Models for Location, Scale, and Shape for Zero-Inflated and Overdispersed Count Data

“…For the future, it would therefore be desirable to develop automatic model choice and variable selection strategies in the spirit of Belitz and Lang [2008] in a frequentist setting or Scheipl et al [2012] in a Bayesian approach via spike and slab priors.…”

The spatial distribution of Schistosoma mansoni infection in four regions of western Côte d’Ivoire

“…One component (exclusion component) is a narrow spike around zero, while the other component (inclusion component) is a wide slab away from zero. For categorical variables, we applied a peNMIG prior that allows to simultaneously including or excluding all coefficients related with the categories of the same variable, by improving shrinkage properties (Scheipl et al, 2012). We included the variables with the inclusion component predominant (i.e.…”

“…Briefly, normal mixture of inverse Gammas with parameter expansion (peNMIG) spike-and-slab priors was applied on the model (Scheipl et al, 2012). We used mixed inverse Gamma distributions for the priors of the coefficients.…”

Objective Bayesian Model Selection in Generalized Additive Models With Penalized Splines

“…We compared the results with those from pure variable selection including only linear functions ("hyper-g linear" and "hyper-g/n linear"), Bayesian fractional polynomials ("Bayesian FPs") (Sabanés Bové and Held, 2011a), spike-and-slab function selection ("Spike-and-slab", Scheipl et al, 2012) and splines knot selection ("Knot selection", Denison, Mallick, and Smith, 1998, using code from chapters 3 and 4 in Denison, Holmes, Mallick, and Smith, 2002).…”

“…For the same model class, Belitz and Lang (2008) propose to use information-criteria or cross-validation, while Fahrmeir, Kneib, and Konrath (2010) and Scheipl, Fahrmeir, and Kneib (2012) use spike-and-slab priors for variable and function selection (see also Scheipl, Kneib, and Fahrmeir (2013) for simulation studies comparing their approach to the one presented in this paper). Brezger and Lang (2008) adopt the concept of Bayesian contour probabilities (Held, 2004) to decide on the inclusion and form of covariate effects.…”