Approximation Analysis of Learning Algorithms for Support Vector Regression and Quantile Regression

Abstract

We study learning algorithms generated by regularization schemes in reproducing kernel Hilbert spaces associated with anϵ-insensitive pinball loss. This loss function is motivated by theϵ-insensitive loss for support vector regression and the pinball loss for quantile regression. Approximation analysis is conducted for these algorithms by means of a variance-expectation bound when a noise condition is satisfied for the underlying probability measure. The rates are explicitly derived under a priori conditions on approximation and capacity of the reproducing kernel Hilbert space. As an application, we get approximation orders for the support vector regression and the quantile regularized regression.

Learning rates of regression with q-norm loss and threshold

“…One need to point out that the proof of Theorem 2 is only applicable to the case q > 1. However, when q = 1, it is a special case of quantile regression and the same learning rates as those of Theorem 2 can be found in [17,18].…”

“…When fixing ǫ > 0, error analysis was conducted in [12]. Xiang, Hu and Zhou [17,18] showed how to accelerate learning rates and preserve sparsity by adapting ǫ. In [5], they discussed the convergence ability with flexible ǫ in an online algorithm.…”

“…x ∈ X, f ∈ L 2 ρ X and suppose that the minimizer f q is in the range of L ν K with 0 < ν ≤ 1 2 . When q = 1, the approximation error D(λ) can be O(λ ν 1−ν ) for quantile regression [18]. When q = 2, D(λ) = O(λ 2ν ) for the least square.…”

Perturbation of convex risk minimization and its application in differential private learning algorithms

“…, Then Still we choose stepping-stone function , which leads to and . Reference [31] shows that , so we can follow the choice for ϵ and λ in Theorem 4 with to get the learning rate as where C̃ is a constant independent of m .…”

Quantile Regression Learning with Coefficient Dependent lq-Regularizer

“…is studied in [19]. Now, we restrict our attention to coefficient-based regularization schemes in a data dependent hypothesis …”