A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates

Abstract

In this paper, we formulate the partially linear single-index models as bi-index dimension reduction models for the purpose of identifying significant covariates in both the linear part and the single-index part through only one combined index in a dimension reduction approach. This is different from all existing dimension reduction methods in the literature, which in general identify two basis directions in the subspace spanned by the parameter vectors of interest, rather than the two parameter vectors themselves. This approach makes the identification and the subsequent estimation and variable selection easier than existing methods for multi-index models. When the number of parameters diverges with the sample size, we then adopt coordinate-independent sparse estimation procedure to select significant covariates and estimate the corresponding parameters. The resulting sparse dimension reduction estimators are shown to be consistent and asymptotically normal with the oracle property. Simulations are conducted to evaluate the performance of the proposed method, and a real data set is analysed for an illustration.

Using single-index ODEs to study dynamic gene regulatory network

“…[ 26 ] proposed semiparametrically efficient profile least-squares estimators for parameter estimation, and employed the SCAD approach to simultaneously select variables and estimate regression coefficients. [ 27 ] studied estimation and variable selection coupling with dimension reduction procedures.…”

Statistical inference on restricted partial linear regression models with partial distortion measurement errors

“…In many applications, semi‐parametric regression models have been proposed to relax assumptions on traditional parametric models, which can retain the ease of interpretation of parameters in linear regression models as well as the flexibility of non‐parametric models. Examples included but are not limited to partial linear models (PLMs) (H äRDLE , L IANG and G AO , ; H E , T ANG and Z UO , ; H UANG and D AVIDSON , ; M üLLER and V AN D E G EER , ; W ANG , B ROWN and C AI , ; W ANG and J ING , ; Y ANG , L I and L IAN , ; Z HU and N G , ; L IANG et al , ; L IU , W ANG and L IANG , ), partial linear single‐index models (C ARROLL et al , ; L IANG et al , ; W ANG et al , ; X IA and H äRDLE , ; Y U and R UPPERT , ; Z HANG et al , ), partial linear varying‐coefficient models (B RAVO , ; L I et al , ; W ANG , Z HU and Z HOU , ; Z HOU and L IANG , ), and so on. We focus on the PLMs): where Y is the response variable, W is a d ‐dimensional covariate vector, θ 0 is an unknown coefficient parameter, V is a univariate covariate, r (·) is an unknown smooth function, ε is an error term with finite variance and E ( ε ) = 0, and the superscript τ denotes the transpose operator on a vector or a matrix throughout this paper.…”