Spatially Adaptive Bayesian Penalized Splines With Heteroscedastic Errors

Abstract

Penalized splines have become an increasingly popular tool for nonparametric smoothing because of their use of low-rank spline bases, which makes computations tractable while maintaining accuracy as good as smoothing splines. This article extends penalized spline methodology by both modeling the variance function nonparametrically and using a spatially adaptive smoothing parameter. This combination is needed for satisfactory inference and can be implemented effectively by Bayesian MCMC. The variance process controlling the spatially adaptive shrinkage of the mean and the variance of the heteroscedastic error process are modeled as log-penalized splines. We discuss the choice of priors and extensions of the methodology, in particular, to multivariate smoothing. A fully Bayesian approach provides the joint posterior distribution of all parameters, in particular, of the error standard deviation and penalty functions. MATLAB, C, and FORTRAN programs implementing our methodology are publicly available.

Simultaneous Confidence Bands for Penalized Spline Estimators

“…An obvious way to construct a simultaneous credible region forf is outlined in Crainiceanu et al (2007). Suppose thatf is the posterior mean estimator and that the posterior standard deviation for each point contained inf has been computed.…”

“…The pointwise credible intervals provide us with a measure of where information on the estimated curve is sparse corresponding to wider intervals or dense corresponding to narrower intervals. In the approach by Crainiceanu et al (2007) this information is obtained from posterior standard deviations. This, however, has the drawback that overand underestimation of the penalized spline are treated in a symmetric fashion whereas the quantile-based approach allows for different uncertainty for over-and underestimation of the curve.…”

“…While such differences may be generally small in Gaussian smoothing situations, they will typically become more relevant in non-Gaussian observation models. A further advantage over the credible band by Crainiceanu et al (2007) is that our proposal does not depend on a specific point estimator, since our credible band makes full use of the posterior sample information, considering a 1 − α sample of the curves to determine the required scaling factor.…”

“…Besag et al (1995) propose to combine appropriate order statistics of the univariate samples. Crainiceanu et al (2007) consider simultaneous confidence bands when posterior normality for the parameter vector can be assumed. Held (2004) constructs simultaneous posterior probability statements about vector parameters based on a Rao-Blackwellized estimate of the posterior density.…”

Fast Adaptive Penalized Splines

“…This is being formulated as hierarchical mixed model, with spline coefficients following a normal distribution, which by itself has a smooth structure over the variances. The modelling exercise is in line with Baladandayuthapani, Mallick & Carroll (2005) or Crainiceanu, Ruppert & Carroll (2006). But in contrast to these papers Laplace's method is used for estimation based on the marginal likelihood.…”