SLOPE—Adaptive variable selection via convex optimization

Abstract

We introduce a new estimator for the vector of coefficients β in the linear model y = Xβ + z, where X has dimensions n × p with p possibly larger than n. SLOPE, short for Sorted L-One Penalized Estimation, is the solution to minb∈ℝp12‖y−Xb‖ℓ22+λ1false|bfalse|false(1false)+λ2false|bfalse|false(2false)+⋯+λpfalse|bfalse|false(pfalse),where λ1 ≥ λ2 ≥ … ≥ λp ≥ 0 and false|bfalse|false(1false)≥false|bfalse|false(2false)≥⋯≥false|bfalse|false(pfalse) are the decreasing absolute values of the entries of b. This is a convex program and we demonstrate a solution algorithm whose computational complexity is roughly comparable to that of classical ℓ1 procedures such as the Lasso. Here, the regularizer is a sorted ℓ1 norm, which penalizes the regression coefficients according to their rank: the higher the rank—that is, stronger the signal—the larger the penalty. This is similar to the Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] procedure (BH) which compares more significant p-values with more stringent thresholds. One notable choice of the sequence {λi} is given by the BH critical values λBHfalse(ifalse)=zfalse(1−i⋅q/2pfalse), where q ∈ (0, 1) and z(α) is the quantile of a standard normal distribution. SLOPE aims to provide finite sample guarantees on the selected model; of special interest is the false discovery rate (FDR), defined as the expected proportion of irrelevant regressors among all selected predictors. Under orthogonal designs, SLOPE with λBH provably controls FDR at level q. Moreover, it also appears to have appreciable inferential properties under more general designs X while having substantial power, as demonstrated in a series of experiments running on both simulated and real data.

Reconstruction algorithm for fluorescence molecular tomography using sorted L-one penalized estimation

“…In this study, we adopted the FastProxSL1 algorithm proposed in [21] to compute the prox and update the regularization parameters. For the experiments in Section 3, the step sizes t k 1∕0.9 ∧ k, the iteration ends when k 1000 or jX k1 − X k j < d, and the tolerance d 1e − 8, which satisfy the convergence condition of the proximal gradient method [27].…”

“…However, the choice of regularization parameter is nontrivial. To achieve adaptivity and accuracy, we formulate the FMT reconstruction to a sorted L 1_norm minimization problem and introduce a sorted L-one penalized estimation (SLOPE) [21] method to solve it. Moreover, an iterative-shrinking permissible region (ISPR) strategy is also combined with SLOPE to improve the performance.…”

Knowledge Discovery from Complex High Dimensional Data

“…In high dimensional settings, statistical understanding of these algorithms is crucial not only to obtain quality solutions but also to invent new types of algorithms, as witnessed in recent literature [2,8,49,57]. Efficient and distributed algorithm implementations also become critical due to high computational demands.…”

A flexible shrinkage operator for fussy grouped variable selection

“…In this paper, we propose a family of shrinkage variable selection operators by controlling the kth largest norm (KAN), a special case of SLOPE studied in Bogdan et al (2015) for the investigation of the false discovery rate. Different from the SLOPE, the proposed KAN method is designed to encourage some grouped variable selection without any group information.…”