A Bayesian Approach to Non-Parametric Monotone Function Estimation

Abstract

The paper proposes two Bayesian approaches to non-parametric monotone function estimation. The first approach uses a hierarchical Bayes framework and a characterization of smooth monotone functions given by Ramsay that allows unconstrained estimation. The second approach uses a Bayesian regression spline model of Smith and Kohn with a mixture distribution of constrained normal distributions as the prior for the regression coefficients to ensure the monotonicity of the resulting function estimate. The small sample properties of the two function estimators across a range of functions are provided via simulation and compared with existing methods. Asymptotic results are also given that show that Bayesian methods provide consistent function estimators for a large class of smooth functions. An example is provided involving economic demand functions that illustrates the application of the constrained regression spline estimator in the context of a multiple-regression model where two functions are constrained to be monotone. Copyright (c) 2009 Royal Statistical Society.

Nonparametric function estimation subject to monotonicity, convexity and other shape constraints

“…Early work in the estimation of monotone functions includes Wright and Wegman (1980) and Friedman and Tibshirani (1984). More recently, Neelon and Dunson (2004) and Shively et al (2009) developed monotone estimation methods in the context of Gaussian models, Manski and Tamer (2002) and Banerjee et al (2009) discussed the problem in binary models, and Dunson (2005) and Schipper et al (2007) considered the problem in Poisson and generalized mixed models, respectively.…”

“…In a Bayesian context the constraints are imposed through the prior distributions on the coefficients. Shively et al (2009) use this idea to develop a Bayesian method for monotone function estimation using fixed-knot splines in Gaussian regression models. The current paper departs from Shively et al (2009) in the following key respects: (1) shape constraints via prior distributions on the spline coefficients are developed for free-knot spline models; (2) the methodology development applies to the family of models that have logconcave likelihood functions, not just Gaussian regressions; and (3) additional shape constraints that include convexity and functions restricted to a single minimum are developed for both free-knot and fixed-knot models.…”

A smoothed monotonic regression via L2 regularization

“…[20,23,30,31,37,39,42]). In general, these methods involve Markov chain Monte Carlo sampling or other type of stochastic optimization, which makes them computationally heavy compared to the aforementioned frequentist alternatives.…”

“…However, in order to compare the predictive power of these models and our approach, we include in our simulation studies the approach described in Neelon and Dunson [30,31]. The choice of the method was motivated by the public availability of the code [32], and also by the results of the simulation in Shively et al [39] demonstrating that there is no obvious winner among the methods in Holmes and Heard [20], Neelon and Dunson [31] and Shively et al [39]. We have also decided not to include the methods based on Gaussian process optimization (such as [23,37,42]) into our simulations because they require at least O(n 3 ) numerical operations per iteration, which makes these algorithms too expensive for large data.…”

A Bayesian semi-parametric model to estimate relationships between crash counts and roadway characteristics

“…Fahrmeir and Osuna (2007) used regression splines for a negative binomial model of count data in the context of automobile insurance claims data, while Xie and Zhang (2008) use regression splines in a frequentist context with fixed knot locations to model accident counts at traffic intersections. Neelon and Dunson (2004), Dunson (2005), Schipper et al (2007) and Shively, Sager and Walker (2009) Unfortunately, transportation data sets are often imperfect, with some design variables out of date (e.g., the road was improved but the road file was not updated) and many crashes going unreported. Also, important explanatory variables are often unavailable (e.g., number of snowy days at a location, roadside clear zone slope and width, and average speeds of travel).…”