Concentration estimates for learning with unbounded sampling

Abstract

The least-square regression problem is considered by regularization schemes in reproducing kernel Hilbert spaces. The learning algorithm is implemented with samples drawn from unbounded sampling processes. The purpose of this paper is to present concentration estimates for the error based on 2 -empirical covering numbers, which improves learning rates in the literature.

Constructive analysis for coefficient regularization regression algorithms

“…It is widely used in [19,23,18,3] and etc., more detailed analysis can be found in [25,26]. More recent references [5,8,7,16] use 2 -empirical covering number to obtain a sharper upper bound for the excess generalization error E(f z,λ ) − E(f ρ ).…”

“…Here we will follow the work of [5]. Since the functions f s and f z,λ vary while the sample size m is different, we need a concentration inequality for a set of functions like in [20].…”

“…Let u, μ > 0 satisfying (8), and define g s and f s by (5) and (6) respectively. Then for any 0 < δ < 1, there exists a subset W 1 of Z m with measure at least 1 − δ 5 , when z ∈ W 1…”

Half supervised coefficient regularization for regression learning with unbounded sampling

“…One usually assumes that the output data are uniformly bounded [4,17,18,26,27]. In recent years, learning with unbounded output data has started gaining attention [3,8,12,23,24]. In [12], capacity-dependent error bounds and learning rates have been derived for least square regularized regression learning with unbounded sampling by means of an integral operator approach introduced in [17].…”

“…In [23], a sample error bound has been deduced for the ERM learning algorithm with unbounded sampling by a covering number argument. In [8], concentration estimates for least square regression algorithms have been presented by l 2 -empirical covering numbers. *Corresponding author.…”