Elastic-net Regularized High-dimensional Negative Binomial Regression: Consistency and Weak Signal Detection

Abstract

We study sparse negative binomial regression (NBR) for count data by showing non-asymptotic merits of the Elastic-net estimator. Two types of oracle inequalities are derived for the Elastic-net estimates of NBR by utilizing Compatibility Factor or Stabil Condition. The second-type oracle inequality is for random design which can be extended to many 1 + 2 regularized M-estimation with the corresponding empirical process having stochastic Lipschitz properties. To show some high probability events, we derive concentration inequality for suprema empirical processes for the weighted sum of negative binomial variables. For applications, we show the sign consistency provided that the non-zero components in sparse true vector are larger than a proper choice of the weakest signal detection threshold; and the second application is that we show the grouping effect inequality with high probability; thirdly, under some assumptions of design matrix, we can recover the true variable set with high probability if the weakest signal detection threshold is large than the turning parameter up to a known constant; at last, we briefly discuss the de-biased Elastic-net estimator and numerical studies are given to support the proposal.

Concentration Inequalities for Statistical Inference

“…where the last inequality is from β −β * 1 ≤ 4e 5LB k sλ. For general losses beyond linear models, the crucial techniques in the nonasymptotical analysis of increasing-dimensional and high-dimensional regressions, which are Bahadur representation's for the M-estimator [49,64] and concentration for Lipschitz loss functions [18,98], respectively. In large-dimensional regressions with p n → c, the theory of random matrix [93], leave-one-out analysis [27,53] and approximate message passing [23,26,27] play important roles for obtaining asymptotical results.…”

Sparse Density Estimation with Measurement Errors

“…More adaptively, we prefer to use the weighted restriction , where the weights ’s are data-dependent and will be specified later. From [ 23 ], we add Elastic-net penality with tuning parameter c , which is in regards to the measurement errors (see [ 24 , 25 ]) for a similar purpose. We would have in the situation without measurement errors.…”

“… ; (H.3): . (H.2) is an assumption in sparse estimation, and the assumption (H.3) is a classical compact parameter space assumption in sparse high-dimensional regressions (see [ 9 , 25 ]).…”

“…There are few results on support recovery, while most of the results are the consistency of the -error and prediction errors. Here, we borrow the framework of [ 25 , 33 ]. They give many proof techniques to deal with the corrected support identification in linear models by regularization.…”

Weak Signal Identification and Inference in Penalized Likelihood Models for Categorical Responses

“…To achieve model selection consistency or variable screening property for a high-dimensional problem, one common condition is the "beta-min" condition, which requires that the nonzero regression coefficients are sufficiently large (Zhao and Yu, 2006;Huang and Xie, 2007;Van de Geer et al, 2011;Tibshirani, 2011;Zhang and Jia, 2017). Therefore, classical methods for variable selection often focus on strong signals that satisfy such condition.…”