On Convex Envelopes and Regularization of Non-convex Functionals Without Moving Global Minima

Abstract

We provide theory for the computation of convex envelopes of non-convex functionals including an 2 -term, and use these to suggest a method for regularizing a more general set of problems. The applications are particularly aimed at compressed sensing and low rank recovery problems but the theory relies on results which potentially could be useful also for other types of non-convex problems. For optimization problems where the 2 -term contains a singular matrix we prove that the regularizations never move the global minima. This result in turn relies on a theorem concerning the structure of convex envelopes which is interesting in its own right. It says that at any point where the convex envelope does not touch the non-convex functional we necessarily have a direction in which the convex envelope is affine.

On phase retrieval via matrix completion and the estimation of low rank PSD matrices

“…Ideally one would like to work with the convex envelope of the functional, but this is not doable in practice. The quadratic envelope yields a non-convex continuous surrogate which has recently been shown to have the desirable property of yielding a regularizer which, just like the convex envelope, does not move the global minima [Car19]. While not completely removing other local minimizers, it reduces the amount and in practice it seems that the corresponding algorithm can find the global minimizer in fairly difficult problems.…”

“…We remark that the maximum negative quadrature of Q γ (F ) is γ, as shown in [Car19], so the condition γ < ρ ensures that the proximal operator is single valued (since the corresponding minimization problem is strongly convex).…”

New Insights on the Optimality Conditions of the $$\ell _2-\ell _0$$ Minimization Problem

“…Although a variety of necessary optimality conditions with different degree of sophistication have been defined, analyzed, and hierarchized (see Section 2 and Figure 1), there is a lack of connection between some of them. In particular, the relation between conditions that are based on the support and those that derive from exact continuous relaxations [11,41,42] have not been studied yet.…”

“…Theorem 2 (Links between (1) and (2)-(3) [41,11]) Let L 0 (L, respectively) be the set of local minimizers of F 0 (F , respectively). Let G 0 ⊆ L 0 (G ⊆L, respectively) be the corresponding subset of global minimizers.…”

“…Numerous penalties have been proposed and analyzed in the literature [7,9,13,18,20,37,38,48,49,51,52]. Moreover, some of them lead to exact relaxations of F 0 in the sense that their minimizers coincide [11,41,42].…”