Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space

Abstract

This paper considers the estimation of the sparse additive quantile regression (SAQR) in high-dimensional settings. Given the nonsmooth nature of the quantile loss function and the nonparametric complexities of the component function estimation, it is challenging to analyze the theoretical properties of ultrahigh-dimensional SAQR. We propose a regularized learning approach with a two-fold Lasso-type regularization in a reproducing kernel Hilbert space (RKHS) for SAQR. We establish nonasymptotic oracle inequalities for the excess risk of the proposed estimator without any coherent conditions. If additional assumptions including an extension of the restricted eigenvalue condition are satisfied, the proposed method enjoys sharp oracle rates without the light tail requirement. In particular, the proposed estimator achieves the minimax lower bounds established for sparse additive mean regression. As a by-product, we also establish the concentration inequality for estimating the population mean when the general Lipschitz loss is involved. The practical effectiveness of the new method is demonstrated by competitive numerical results.

Estimating high‐dimensional additive Cox model with time‐dependent covariate processes

“…Under the sparse and additive settings, similar results have been published for various stationary models such as additive regression (Huang et al, ; Meier et al, ; Ravikumar et al, ) and additive quantile regressions Lv et al (). The existing additive regression corresponds to the least square loss or quantile loss, which are sum of iid random variables.…”

A unified penalized method for sparse additive quantile models: an RKHS approach

“…This has been verified under some mild conditions on the underlying distribution (Steinwart and Christmann 2011;Lv et al 2016).…”

“…While the first combined penalty leads to a group lasso formulation, it appears to be lack of theoretical justification. The second term has been proved to enjoy some theoretical properties (Lv et al 2016;Raskutti et al 2012); however, it still requires SOCP. To enjoy both theoretical properties and computational efficiency, we propose to equip the additive QR model with the new combination of the sparsity and smoothness penalty I n ( f ) in (2).…”

Superposition, reduction of multivariable problems, and approximation

“…We get H W = L 2 (J m ) iff W ⊂ P (R m ) contains all the rational points. For additional details regarding reproducing kernel Hilbert spaces, see e.g., [28,29,30,31].…”