Direction Estimation in Single-Index Regressions via Hilbert-Schmidt Independence Criterion

Zhang, Nan; Yin, Xiangrong

doi:10.5705/ss.2013.113

Cited by 7 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this article, following the work of Zhang and Yin (2015) and Tan et al (2018b), we develop a new approach using Hilbert-Schmidt Independence Criterion (HSIC) for singleindex models. The proposed method can handle the scenario p > n and require the weakest conditions among the existing high-dimensional sparse SDR methods.…”

Section: Introductionmentioning

confidence: 99%

“…To sum up, the main contributions of our work are as follows. First, our method extends the HSIC-based single-index regression (Zhang and Yin, 2015) to a sufficient variable selection method. Since it does not involve the inversion of the sample covariance matrix, it can naturally handle a large p small n situation.…”

Section: Introductionmentioning

confidence: 99%

“…According to Gretton et al (2005b), HSIC equals 0 if and only if two random variables are independent, which makes it possible for its application in the field of SDR. Indeed, under a mild condition, Zhang and Yin (2015) showed that solving (2.2) with respect to a p × 1 vector β would yield a basis of S Y |X , or in other words, the single-index direction:…”

Section: Introductionmentioning

confidence: 99%

“…For details about the regularity conditions and the specific form of V 11 , please refer to Zhang and Yin (2015) and its online supplementary material.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

High-Dimensional Sparse Single-Index Regression Via Hilbert-Schmidt Independence Criterion

Wu¹,

Deng²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

Hilbert-Schmidt Independence Criterion (HSIC) has recently been used in the field of single-index models to estimate the directions. Compared with some other well-established methods, it requires relatively weaker conditions. However, its performance has not yet been studied in the high-dimensional scenario, where the number of covariates is much larger than the sample size. In this article, we propose a new efficient sparse estimate in HSIC based single-index model. This new method estimates the subspace spanned by the linear combinations of the covariates directly and performs variable selection simultaneously. Due to the non-convexity of the objective function, we use a majorize-minimize approach together with the linearized alternating direction method of multipliers algorithm to solve the optimization problem. The algorithm does not involve the inverse of the covariance matrix and therefore can handle the large p small n scenario naturally. Through extensive simulation studies and a real data analysis, we show our proposal is efficient and effective in the high-dimensional setting. The Matlab codes for this method are available online.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…For details about the regularity conditions and the specific form of V 11 , please refer to Zhang and Yin (2015) and its online supplementary material.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High-Dimensional Sparse Single-Index Regression Via Hilbert-Schmidt Independence Criterion

Wu¹,

Deng²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Let z i = ( z i 1 , ⋯, z ip z ) T be a p z × 1 vector of such key features. Finally, model (1) may be approximated by yi=ffalse(zifalse)+σyεiy=ffalse(zi1,⋯,zitalicipzfalse)+σyεiy.When z i = Γ z x i , in which Γ z is a p z × p x matrix, model (3) reduces to the well-known semi parametric index model in the dimensional reduction literature (Li, 1991; Cook and Ni, 2005; Zhang and Yin, 2014; Yang, 2016). Most existing dimension reduction methods focus on the scenario when p x is smaller than n .…”

Section: Introductionmentioning

confidence: 99%

MILFM: Multiple Index Latent Factor Model Based on High-Dimensional Features

2018

View full text Add to dashboard Cite

The aim of this article is to develop a multiple-index latent factor modeling (MILFM) framework to build an accurate prediction model for clinical outcomes based on a massive number of features. We develop a three-stage estimation procedure to build the prediction model. MILFM uses an independent screening method to select a set of informative features, which may have a complex nonlinear relationship with the outcome variables. Moreover, we develop a latent factor model to project all informative predictors onto a small number of local subspaces, which lead to a few key features that capture reliable and informative covariate information. Finally, we fit the regularized empirical estimate to those key features in order to accurately predict clinical outcomes. We systematically investigate the theoretical properties of MILFM, such as risk bounds and selection consistency. Our simulation results and real data analysis show that MILFM outperforms many state-of-the-art methods in terms of prediction accuracy.

show abstract

High-dimensional sparse single–index regression via Hilbert–Schmidt independence criterion

Chen,

Deng,

et al. 2024

Stat Comput

View full text Add to dashboard Cite

Hilbert-Schmidt Independence Criterion (HSIC) has recently been introduced to the field of single-index models to estimate the directions. Compared with other well-established methods, the HSIC based method requires relatively weak conditions. However, its performance has not yet been studied in the prevalent highdimensional scenarios, where the number of covariates can be much larger than the sample size. In this article, based on HSIC, we propose to estimate the possibly sparse directions in the high-dimensional single-index models through a parameter reformulation. Our approach estimates the subspace of the direction directly and performs variable selection simultaneously. Due to the non-convexity of the objective function and the complexity of the constraints, a majorize-minimize algorithm together with the linearized alternating direction method of multipliers is developed to solve the optimization problem. Since it does not involve the inverse of the covariance matrix, the algorithm can naturally handle large p small n scenarios. Through extensive simulation studies and a real data analysis, we show that our proposal is efficient and effective in the high-dimensional settings. The Matlab codes for this method are available online.

show abstract

Direction Estimation in Single-Index Regressions via Hilbert-Schmidt Independence Criterion

Cited by 7 publications

References 26 publications

High-Dimensional Sparse Single-Index Regression Via Hilbert-Schmidt Independence Criterion

High-Dimensional Sparse Single-Index Regression Via Hilbert-Schmidt Independence Criterion

MILFM: Multiple Index Latent Factor Model Based on High-Dimensional Features

High-dimensional sparse single–index regression via Hilbert–Schmidt independence criterion

Contact Info

Product

Resources

About