A computational approach to nonparametric regression: bootstrapping CMARS method

Abstract

Bootstrapping is a computer-intensive statistical method which treats the data set as a population and draws samples from it with replacement. This resampling method has wide application areas especially in mathematically intractable problems. In this study, it is used to obtain the empirical distributions of the parameters to determine whether they are statistically significant or not in a special case of nonparametric regression, conic multivariate adaptive regression splines (CMARS), a statistical machine learning algorithm. CMARS is the modified version of the well-known nonparametric regression model, multivariate adaptive regression splines (MARS), which uses conic quadratic optimization. CMARS is at least as complex as MARS even though it performs better with respect to several criteria. To achieve a better performance of CMARS with a less complex model, three different bootstrapping regression methods, namely, random-X, fixed-X and wild bootstrap are applied on four data sets with different size and scale. Then, the performances of the models are compared using various criteria including accuracy, precision, complexity, stability, robustness and computational efficiency. The results imply that bootstrap methods give more precise parameter estimates although they are computationally inefficient and that among all, random-X resampling produces better models, particularly for medium size and scale data sets.

Random Ensemble MARS: Model Selection in Multivariate Adaptive Regression Splines Using Random Forest Approach

“…[17] applied information measure of complexity (ICOMP) as a powerful model selection criterion for the MARS modelling. [18] used Bootstrapping to obtain the empirical distributions of the parameters and to determine whether they were statistically significant or not in a special case of nonparametric regression.…”

Bootstrap-based model selection in subset polynomial regression

“…The estimator of the multivariate distribution function is used as the basis for the implementation of the Bootstrap method. Yazici et al [8] used the Bootstrap method to obtain the empirical distribution of the parameters in the nonparametric regression of Conic Multivariate Adaptive Regression Splines (CMARS). The results showed that the bootstrap method provides an…”