Regularized learning in Banach spaces as an optimization problem: representer theorems

Zhang, Haizhang; Zhang, Jun

doi:10.1007/s10898-010-9575-z

Cited by 41 publications

(69 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since unlike Hilbert spaces of the same dimension the L p (I) spaces are not isomorphic to each other they exhibit a richer geometric variety which is potentially useful for the development of new learning algorithms. Note that above example is one dimensional for notational simplicity and similar constructions yield RKBS isomorphic to L p µ (R d ) where µ is a finite positive Borel measure on R d as shown in Zhang and Zhang [21]. The corresponding RKBS B consists of functions of the form…”

Section: Theorem 53mentioning

confidence: 95%

“…In this section we give several examples of Banach spaces to which the results in this paper apply. These examples are taken from the work of Zhang, Xu and Zhang [20], and Zhang and Zhang [21]. In these papers the theory of reproducing kernel Banach spaces (RKBS) is developed.…”

Section: Examplesmentioning

confidence: 99%

“…Furthermore kernels can be defined on input sets which a priori do not have a mathematical structure by embedding the set into a Hilbert space. The importance and wide success of kernel methods and the representer theorem have led to extensions of the theory to reproducing kernel Banach spaces (RKBS) by Zhang, Xu and Zhang [20] and representer theorems for learning in RKBS by Zhang and Zhang [21]. This motivates the study of the more general regularisation problem…”

Section: Introductionmentioning

confidence: 99%

“…Finally in section 5 we give some examples of spaces to which our results apply. This section is based on the work of Zhang, Xu and Zhang [20] and Zhang and Zhang [21] on reproducing kernel Banach spaces so we will first be presenting the relevant definitions and results on the construction of RKBS from [20] in this section. We then give a few examples which have been presented in these papers.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

When is there a representer theorem?

Schlegel

2019

J Glob Optim

View full text Add to dashboard Cite

We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is at the core of kernel methods in machine learning as it makes the problem computationally tractable. Most literature deals only with sufficient conditions for representer theorems in Hilbert spaces. We prove necessary and sufficient conditions for the existence of representer theorems in reflexive Banach spaces and illustrate why in a sense reflexivity is the minimal requirement on the function space. We further show that if the learning relies on the linear representer theorem, then the solution is independent of the regulariser and in fact determined by the function space alone. This in particular shows the value of generalising Hilbert space learning theory to Banach spaces.

show abstract

Section: Theorem 53mentioning

confidence: 95%

Section: Examplesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

When is there a representer theorem?

Schlegel

2019

J Glob Optim

View full text Add to dashboard Cite

show abstract

“…In the classical setting, a representer theorem states that a minimizer of a Tikhonov regularized empirical risk function defined over a reproducing kernel Hilbert space can be represented as a finite linear combination of the feature map values on the training points [58]. The investigation in Banach spaces was initiated in [50] and continued in [79,80]. In this section representer theorems are established in the general context of Banach spaces, totally convex regularizers, vector-valued functions, and approximate minimization.…”

Section: Representer and Sensitivity Theorems In Banach Spacesmentioning

confidence: 99%

Regularized learning schemes in feature Banach spaces

2017

View full text Add to dashboard Cite

This paper proposes a unified framework for the investigation of constrained learning theory in reflexive Banach spaces of features via regularized empirical risk minimization. The focus is placed on Tikhonov-like regularization with totally convex functions. This broad class of regularizers provides a flexible model for various priors on the features, including in particular hard constraints and powers of Banach norms. In such context, the main results establish a new general form of the representer theorem and the consistency of the corresponding learning schemes under general conditions on the loss function, the geometry of the feature space, and the modulus of total convexity of the regularizer. In addition, the proposed analysis gives new insight into basic tools such as reproducing Banach spaces, feature maps, and universality. Even when specialized to Hilbert spaces, this framework yields new results that extend the state of the art.

show abstract

Representer Theorem

Wahba¹,

Wang²

2019

Wiley StatsRef: Statistics Reference Online

View full text Add to dashboard Cite

The representer theorem plays an outsized role in a large class of learning problems. It provides a means to reduce infinite dimensional optimization problems to tractable finite dimensional ones. This article reviews the representer theorem for various learning problems under the reproducing kernel Hilbert spaces framework. We present solutions to the penalized least squares and penalized likelihood for nonparametric regression and support vector machines for classification as a solution to the penalized hinge loss. We discuss extensions of the representer theorem for regression with functional data.

show abstract

Regularized learning in Banach spaces as an optimization problem: representer theorems

Cited by 41 publications

References 24 publications

When is there a representer theorem?

When is there a representer theorem?

Regularized learning schemes in feature Banach spaces

Representer Theorem

Contact Info

Product

Resources

About