Envelope Functions: Unifications and Further Properties

Abstract

Forward-backward and Douglas-Rachford splitting are methods for structured nonsmooth optimization. With the aim to use smooth optimization techniques for nonsmooth problems, the forward-backward and Douglas-Rachford envelopes where recently proposed. Under specific problem assumptions, these envelope functions have favorable smoothness and convexity properties and their stationary points coincide with the fixed-points of the underlying algorithm operators. This allows for solving such nonsmooth optimization problems by minimizing the corresponding smooth convex envelope function. In this paper, we present a general envelope function that unifies and generalizes existing ones. We provide properties of the general envelope function that sharpen corresponding known results for the special cases. We also present a new interpretation of the underlying methods as being majorizationminimization algorithms applied to their respective envelope functions.

A new envelope function for nonsmooth DC optimization

“…Lemma 6. In problem (10), suppose that f is L f -smooth and g is proper, convex, and lsc. Then, for every…”

“…Most related to our approach, [4] analyzes a Gauss-Seidel-type FBS in the spirit of the PALM algorithm [7], and [16] exploits the interpretation of FBS as a gradient-type algorithm on the forwardbackward envelope (FBE) [17,22] to develop quasi-Newton methods for the nonsmooth and nonconvex problem (2). The gradient interpretation of splitting schemes originated in [20] with the proximal point algorithm and has recently been extended to several other schemes [10,17,18,23]. In this work we undertake a converse direction: first we design a smooth surrogate of the nonsmooth DC function in (P), and then derive a novel splitting algorithm from its gradient steps.…”

Prediction-correction for Nonsmooth Time-varying Optimization via Forward-backward Envelopes

“…x k`1|k´x ď ζ P x k´xk`1|k (22) x k`1´xk`1 ď ζ C x k`1|k´xk`1 (23) and therefore the last thing to do is to combine them to derive a bound for the error x k`1´xk`1 . Following the same steps of [14, Appendix B] from inequalities (21), (22) and the results above it is possible to compute…”

“…Remark 4 The recent work [22] proved that if ϕ is convex quadratic, then the FBE is strongly convex and smooth, and notice that this is exactly the case of h k in the prediction step. Therefore we can minimize the FBE at the prediction step using a Newton method with BFGS scheme, without the need for the line search that requires a larger number of iterations.…”

“…In Sec. 2, we will report some results about the FBE, and we refer the reader to [20][21][22] for an in-depth treatment of the subject.…”