Learning Rates of Least-Square Regularized Regression

Abstract

This paper considers the regularized learning algorithm associated with the leastsquare loss and reproducing kernel Hilbert spaces. The target is the error analysis for the regression problem in learning theory. A novel regularization approach is presented, which yields satisfactory learning rates. The rates depend on the approximation property and the capacity of the reproducing kernel Hilbert space measured by covering numbers. When the kernel is C ∞ and the regression function lies in the corresponding reproducing kernel Hilbert space, the rate is m −ζ with ζ arbitrarily close to 1, regardless of the variance of the bounded probability distribution.Short Title: Least-square Regularized Regression

Kernel methods in system identification, machine learning and function estimation: A survey

“…Its asymptotic behavior, as N goes to ∞, has been studied in many recent works, see e.g. Smale and Zhou (2007) and Wu, Ying, and Zhou (2006), also in the context of NARX identification (De Nicolao & Trecate, 1999).…”

Learning Theory Estimates via Integral Operators and Their Approximations

“…They may be improved when some extra information about the kernel such as its regularity is available. See [14].…”

“…The rates for this approximation in L 2 ρ X have been considered in [3,4,16,11,14], while the approximation in the space H K (hence in L ∞ ρ X by (1.2) and in C s by [17]) has been shown in [11]. (An early version of Theorem 1 below appeared in a late version of [11], and was subsequently removed.)…”

Convergence Rate of Coefficient Regularized Kernel-based Learning Algorithms

“…It is known that one of the purposes of learning is to obtain f z through samples z and provide the consistency analysis of f z and f ρ . Kernel-based method is a popular way for this purpose, see [1,2,3,4,5,6,7].…”