Hyperparameter selection for group-sparse regression: A probabilistic approach

Abstract

This work analyzes the effects on support recovery for different choices of the hyper-or regularization parameter in LASSO-like sparse and group-sparse regression problems. The hyperparameter implicitly selects the model order of the solution, and is typically set using cross-validation (CV). This may be computationally prohibitive for large-scale problems, and also often overestimates the model order, as CV optimizes for prediction error rather than support recovery. In this work, we propose a probabilistic approach to select the hyperparameter, by quantifying the type I error (false positive rate) using extreme value analysis. From Monte Carlo simulations, one may draw inference on the upper tail of the distribution of the spurious parameter estimates, and the regularization level may be selected for a specified false positive rate. By solving the scaled group-LASSO problem, the choice of hyperparameter becomes independent of the noise variance. Furthermore, the effects on the false positive rate caused by collinearity in the dictionary is discussed, including ways of circumventing them. The proposed method is compared to other hyperparameter-selection methods in terms of support recovery, false positive rate, false negative rate, and computational complexity. Simulated data illustrate how the proposed method outperforms CV in both computational complexity and support recovery.

Multi-Source Direction-of-Arrival Estimation using Group-Sparse Fitting of Steered Response Power Maps

“…Since the regularization parameter λ present in ( 22) is here heuristically set as explained in Sec. 4 and depends on setup conditions, such as the noise level and the number of target sources [28], its chosen value may be suboptimal in the simulations considered. Further tuning could be employed, for example, to investigate the proposed method's performance under higher levels of reverberation.…”

Sparse group selection and analysis offunction‐relatedresidue forprotein‐staterecognition

“…To select variables inside a group (instead of choosing all in a group as in group lasso), SGL 55,58 has an additional L 1 ‐norm penalty (See Equation for details). The resulting model of using both L 1 ‐norm and group norm is better known as sparse group learning, 53,58–60 with SGL using the least square loss (see Equation for details).…”

Generalized Time-Updating Sparse Covariance-Based Spectral Estimation

“…It is often a non-trivial task to select such hyperparameters properly, although there is some work indicating that one may formulate selection algorithms for this (see e.g. [7], and the references therein).…”