The structured elastic net for quantile regression and support vector classification

Abstract

In view of its ongoing importance for a variety of practical applications, feature selection via 1 -regularization methods like the lasso has been subject to extensive theoretical as well empirical investigations. Despite its popularity, mere 1 -regularization has been criticized for being inadequate or ineffective, notably in situations in which additional structural knowledge about the predictors should be taken into account. This has stimulated the development of either systematically different regularization methods or double regularization approaches which combine 1 -regularization with a second kind of regularization designed to capture additional problem-specific structure. One instance thereof is the 'structured elastic net', a generalization of the proposal in Zou and Hastie (J. R. Stat. Soc. Ser. B 67:301-320, 2005), studied in Slawski et al. (Ann. Appl. Stat. 4(2):1056-1080, 2010) for the class of generalized linear models.In this paper, we elaborate on the structured elastic net regularizer in conjunction with two important loss functions, the check loss of quantile regression and the hinge loss of support vector classification. Solution paths algorithms are developed which compute the whole range of solutions as one regularization parameter varies and the second one is kept fixed.The methodology and practical performance of our approach is illustrated by means of case studies from image classification and climate science.

An Intelligent System Approach for Asthma Prediction in Symptomatic Preschool Children

“…The SVM classifiers solve the following quadratic programming problem: $\begin{array}{c}\min \frac{1}{2}{\omega }^T\omega +C{\sum }_{i=1}^N{\xi }_i\end{array}$ subject to y i ( ω T ϕ ( x i ) + b ) = 1 − ξ i , ξ i ≥ 0, i = 1,…, N , ξ i represents the degree of misclassification of the data x i and C is the penalty parameter of the error term [28]. …”

Volatile Compounds Selection via Quantile Correlation and Composite Quantile Correlation: A Whiting Case Study

“…For future investigation on whiting data, it will be very interesting to explore the following issues: 1) Simultaneous model selection in multiple quantile regression [11] 2) Selection of groups of highly correlated compounds [8] 3) Quantile regression models and inference processes based on [15] and [16].…”

Variable selection using P‐splines

“…The elastic net method was introduced by Zou and Hastie . See Zou and Zhang, Wu, Slawski, and Zhou for some recent contributions.…”

“…Among the important properties that distinguish between the various (·) functions are: the smoothness (mainly differentiability) of the function at zero; and the convexity or nonconvexity of the function. Indeed, the convexity of (and hence of J) determines whether the second term in the minimization problem (12) is convex or not. The differentiability of also has an impact on how easy or difficult it is to find a solution of the optimization problem.…”