Gather and Conquer: Region-Based Strategies to Accelerate Safe Screening Tests

Abstract

In this paper, we propose new methodologies to decrease the computational cost of safe screening tests for LASSO. We first introduce a new screening strategy, dubbed "joint screening test", which allows the rejection of a set of atoms by performing one single test. Our approach enables to find good compromises between complexity of implementation and effectiveness of screening. Second, we propose two new methods to decrease the computational cost inherent to the construction of the (so-called) "safe region". Our numerical experiments show that the proposed procedures lead to significant computational gains as compared to standard methodologies.

Region-Based Relaxations To Accelerate Greedy Approaches

“…A proof of this result can be found in our companion paper [15,Theorem 2]. We see that by combining Theorem 2 with (8), we obtain an easy computable upper bound on max a∈A\Anot a, r .…”

“…In this paper, we propose an efficient methodology to perform (2) with a prescribed accuracy. Our procedure is inspired from our work [15]: it consists in performing simple tests allowing to identify groups of atoms not attaining the maximum value of (2). Although motivated in the context of continuous dictionaries, our methodology applies to both the continuous and discrete settings.…”

“…The choice of the regions {R l } L l=1 is a degree of freedom of the procedure, different choices leading to different performance. The complexity associated to the evaluation of the left-hand side of ( 4) is a direct function of the geometry of R. The following result, proved in our companion paper [15,Theorem 1], shows that for "dome" and "sphere" regions, test (4) can be carried out at the cost of one inner product in H:…”

Short and Squeezed: Accelerating the Computation of Antisparse Representations with Safe Squeezing

“…In other words, the left-hand-side of ( 9) ensures that the i-th entry of x -the solution of (1) -belongs to I as soon the i-th column of A is not orthogonal to some vector u defined in (7). We emphasize that when some i passes the test, the sign of the saturation is also an output of the test.…”

“…Over the past few years, safe screening has sparked a surge of interest in the LASSO literature e.g., [4][5][6][7] and beyond 1 regularization [8][9][10]. However, up to our knowl-Part of this work has been funded thanks to the Becose ANR project no.…”