Convergence of the backfitting algorithm for additive models

Abstract

The backfitting algorithm is an iterative procedure for fitting additive models in which, at each step, one component is estimated keeping the other components fixed, the algorithm proceeding component by component and iterating until convergence. Convergence of the algorithm has been studied by Buja, Hastie, and Tibshirani (1989). We give a simple, but more general, geometric proof of the convergence of the backfitting algorithm when the additive components are estimated by penalized least squares. Our treatment covers spline smoothers and structural time series models, and we give a full discussion of the degenerate case. Our proof is based on Halperin's (1962) generalization of von Neumann's alternating projection theorem.1991 Mathematics subject classification (Amer. Math. Soc): 62G05, 62J02, 65DO7, 65D10.

Nonparametric Inferences for Additive Models

“…Among these methods, the backfitting algorithm is considered a useful fitting tool and has received much attention for its easy of implementation. Hardle and Hall (1993) and Ansley and Kohn (1994) explored the convergence of the algorithm based on projection smoothers. Opsomer and Ruppert (1997) Jianqing Fan is Professor, Department of Operation Research and Financial Engineering, Princeton University, Princeton, NJ 08544, and Professor of Statistics, Chinese University of Hong Kong (E-mail: jqfan@princeton.edu).…”

Asymptotic Properties of Backfitting Estimators

“…The study of the bivariate model in Opsomer and Ruppert [15] provides most of the methodological`t ools'' needed for the current article and will be frequently referred to. It should be noted that since the results on the statistical properties of the backfitting estimators in Opsomer and Ruppert [15] as well as those in the present article apply to a non-projection smoothing method, the results are only valid when the true underlying mean function is assumed to be of the form (1).…”

“…They introduce the concept of concurvity to describe nonlinear dependencies between the covariates that lead to degenerate (non-unique) solutions to the backfitting algorithm. Ha rdle and Hall [7] and Ansley and Kohn [1] further explore the convergence of the algorithm in the context of projection and spline smoothers, respectively. Opsomer and Ruppert [15] derive sufficient conditions for the existence and uniqueness of the estimators for the bivariate additive model for local polynomial regression, a widely used non-projection smoother.…”

Nonlinear Time Series

“…The idea of extending additive models includes the projection pursuit model due to Friedman and Stuetzle (1981), the transformed additive model by Breiman and Friedman (1985), and the generalized additive model by Hastie and Tibshirani (1986). The convergence of the backfitting algorithm has been studied by a number of authors, including Breiman and Friedman (1985), Buja, Hastie, and Tibshirani (1989), , Ansley and Kohn (1994), and Opsomer and Ruppert (1997). The asymptotic bias and variance of the backfitting estimator using the local polynomial fitting were investigated by Opsomer and Ruppert (1997) and Opsomer (2000).…”