Regularized learning schemes in feature Banach spaces

Abstract

This paper proposes a unified framework for the investigation of constrained learning theory in reflexive Banach spaces of features via regularized empirical risk minimization. The focus is placed on Tikhonov-like regularization with totally convex functions. This broad class of regularizers provides a flexible model for various priors on the features, including in particular hard constraints and powers of Banach norms. In such context, the main results establish a new general form of the representer theorem and the consistency of the corresponding learning schemes under general conditions on the loss function, the geometry of the feature space, and the modulus of total convexity of the regularizer. In addition, the proposed analysis gives new insight into basic tools such as reproducing Banach spaces, feature maps, and universality. Even when specialized to Hilbert spaces, this framework yields new results that extend the state of the art.

Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods

“…Concerning statistical properties (such as consistency, learning rates and generalization upper bounds for the risk) of Banach valued learning algorithms, these were also investigated in [27], [51], [49] [16], albeit only in the case of independent training data.…”

“…Following [16], [57], as a direction for future work, one can investigate the geometrical properties of the underlying Banach space norm so as to ensure the possibility of learning in the normed space on the one hand, and to satisfy the smoothness assumption A1 on the other. In such a situation the concentration results presented in this paper will apply and have the potential to provide a pivotal tool in the analysis of such schemes for weakly dependent data.…”

Bias Reduction in Variational Regularization

“…Variational regularization methods with nonquadratic functionals such as total variation or 1 -norms have evolved to a standard tool in inverse problems [10,34], image processing [13], compressed sensing [12], and recently related fields such as learning theory [15]. The popularity of such approaches stems from superior structural properties compared to other regularization approaches.…”

Generalized support vector regression: Duality and tensor-kernel representation

“…This type of kernels describe Banach spaces of analytic functions and include generalizations of the exponential and polynomial kernels as well as, in the complex case, generalizations of the Szegö and Bergman kernels.solve the dual problem as well as to compute the solution of the primal (infinite dimensional) problem. This is what it is known as the kernel trick and makes support vector regression effective and so popular in applications [21].Learning in Banach spaces of functions is an emerging area of research which in principle permits to consider learning problems with more general types of norms than Hilbert norms [5,10,27]. The main motivation for this generalization comes from the need of finding more effective sparse representations of data or for feature selection.…”

“…Learning in Banach spaces of functions is an emerging area of research which in principle permits to consider learning problems with more general types of norms than Hilbert norms [5,10,27]. The main motivation for this generalization comes from the need of finding more effective sparse representations of data or for feature selection.…”