Learning rates for the risk of kernel-based quantile regression estimators in additive models

Christmann, Andreas; Zhou, Ding‐Xuan

doi:10.1142/s0219530515500050

Cited by 27 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results in this paper only requires thatd → ∞ as n → ∞, and there is no similar constraint on the number of ambient dimension d. In other words, our results cover the fixed dimensional case, which has been studied extensively; see [14,18,28] and among others. Interestingly, the result of the excess risk via the proposed method shows that the Lasso-type method is more favorable in term of prediction performance than sparsity recovery, since the latter requires strong incoherent conditions to guarantee sparsity recovery consistency; see the related details in [54].…”

Section: 1mentioning

confidence: 90%

“…This is the optimal rate in the classical literature of statistical learning for finitely many predictors (see Theorem 3 of [42]). Under the framework of RKHS, Christmann and Zhou [14] gave the same learning rates of regularized kernel based methods for additive but fixed dimensional models. Nonetheless, their rates require a variance-expectation bound, different from our sparse 1 -regularized method.…”

Section: A Fast Oracle Inequalitymentioning

confidence: 98%

See 1 more Smart Citation

Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

Lv¹,

Lin²,

Lian³

et al. 2018

Ann. Statist.

View full text Add to dashboard Cite

This paper considers the estimation of the sparse additive quantile regression (SAQR) in high-dimensional settings. Given the nonsmooth nature of the quantile loss function and the nonparametric complexities of the component function estimation, it is challenging to analyze the theoretical properties of ultrahigh-dimensional SAQR. We propose a regularized learning approach with a two-fold Lasso-type regularization in a reproducing kernel Hilbert space (RKHS) for SAQR. We establish nonasymptotic oracle inequalities for the excess risk of the proposed estimator without any coherent conditions. If additional assumptions including an extension of the restricted eigenvalue condition are satisfied, the proposed method enjoys sharp oracle rates without the light tail requirement. In particular, the proposed estimator achieves the minimax lower bounds established for sparse additive mean regression. As a by-product, we also establish the concentration inequality for estimating the population mean when the general Lipschitz loss is involved. The practical effectiveness of the new method is demonstrated by competitive numerical results.

show abstract

Section: 1mentioning

confidence: 90%

Section: A Fast Oracle Inequalitymentioning

confidence: 98%

Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

Lv¹,

Lin²,

Lian³

et al. 2018

Ann. Statist.

View full text Add to dashboard Cite

show abstract

“…Suppose that

and

for each

in ( 4 ) with

. Here,

is an unknown univariate function in a reproducing kernel Hilbert space (RKHS)

associated with kernel

and norm

[ 30 , 31 ], and

is an intrinsic subset with cardinality

. This means each observation

is generated according to:

where

and

satisfies the condition ( 2 ).…”

Section: Methodsmentioning

confidence: 99%

“…The mode-induced regression metric is robust to the non-Gaussian noise according to the theoretical and empirical evaluations [ 14 , 15 , 17 ]. The regularized penalty addresses the sparsity and smoothness of the estimator, which has shown promising performance for mean regression [ 2 , 29 , 30 , 31 ]. Therefore, different from mean-based kernel regression and additive models, the mode-based approach enjoys robustness and interpretability simultaneously due to its metric criterion and trade-off penalty.…”

Section: Introductionmentioning

confidence: 99%

Error Bound of Mode-Based Additive Models

Deng

Chen

Song

et al. 2021

Entropy

View full text Add to dashboard Cite

Due to their flexibility and interpretability, additive models are powerful tools for high-dimensional mean regression and variable selection. However, the least-squares loss-based mean regression models suffer from sensitivity to non-Gaussian noises, and there is also a need to improve the model’s robustness. This paper considers the estimation and variable selection via modal regression in reproducing kernel Hilbert spaces (RKHSs). Based on the mode-induced metric and two-fold Lasso-type regularizer, we proposed a sparse modal regression algorithm and gave the excess generalization error. The experimental results demonstrated the effectiveness of the proposed model.

show abstract

“…If the predicted value y i is less than a distance ε away from the actual value t i , if |t i − y i | < ε. In figure 3, represents the region bound by y i ± ε for all i is called ε-insensitive region [9][10][11]. By modifying to the penalty function is that output variables which are lies outside the region are given one of two slack variable penalties depending on whether they lay above ξ + or below ξ − the region where ξ + > 0 and ξ − < 0 for all i…”

Section: Regression With ε-Insensitive Regionmentioning

confidence: 99%

SVM approach for non-parametric method in classification and regression learning process on feature selection with $epsilon$-insensitive region

Premalatha¹,

Vijayalakshmi²

2019

Malaya J. Mat

View full text Add to dashboard Cite

Machine Learning is considered as a subfield of Artificial Intelligence. Machine Learning is concerned with the development of techniques and design the methods which enable the computer to learn. The field of machine learning is concerned with constructing computer program that automatically improve its performance with experience. In today's machine learning applications, support vector machines are considered (SVM) one of the most robust and accurate methods among all well-known algorithms and also being developed at a fast space. The aim of SVM is to find the best classification function, in a two-class learning task, and to distinguish between members of the two classes in the training data. Hence, the goal of machine learning is to find the output hypothesis that performed the correct classification of the training data, but the other earlier algorithms to find the hypothesis that accurate fit to the data. SVM requires that each data instance is represented as a vector of real numbers. Hence, if there are categorical attributes, convert them into numerical data, then we using m numbers to represent an m-category attribute. Only one of the m numbers is one, and others are zero. Machine learning has been applied in various field such as medical diagnosis, bioinformatics, detecting credit card fraud, classifying DNA sequences, speech and handwriting recognition, object recognition in computer vision, and robot locomotion. The objective is algorithmic approach for non parametric methods to tractable for high dimensional massive datasets.

show abstract

Learning rates for the risk of kernel-based quantile regression estimators in additive models

Cited by 27 publications

References 35 publications

Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

Error Bound of Mode-Based Additive Models

SVM approach for non-parametric method in classification and regression learning process on feature selection with $epsilon$-insensitive region

Contact Info

Product

Resources

About