A proximal difference-of-convex algorithm with extrapolation

Abstract

We consider a class of difference-of-convex (DC) optimization problems whose objective is levelbounded and is the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous concave function. While this kind of problems can be solved by the classical difference-of-convex algorithm (DCA) [26], the difficulty of the subproblems of this algorithm depends heavily on the choice of DC decomposition. Simpler subproblems can be obtained by using a specific DC decomposition described in [27]. This decomposition has been proposed in numerous work such as [18], and we refer to the resulting DCA as the proximal DCA. Although the subproblems are simpler, the proximal DCA is the same as the proximal gradient algorithm when the concave part of the objective is void, and hence is potentially slow in practice. In this paper, motivated by the extrapolation techniques for accelerating the proximal gradient algorithm in the convex settings, we consider a proximal difference-of-convex algorithm with extrapolation to possibly accelerate the proximal DCA. We show that any cluster point of the sequence generated by our algorithm is a stationary point of the DC optimization problem for a fairly general choice of extrapolation parameters: in particular, the parameters can be chosen as in FISTA with fixed restart [15]. In addition, by assuming the Kurdyka-Lojasiewicz property of the objective and the differentiability of the concave part, we establish global convergence of the sequence generated by our algorithm and analyze its convergence rate. Our numerical experiments on two difference-of-convex regularized least squares models show that our algorithm usually outperforms the proximal DCA and the general iterative shrinkage and thresholding algorithm proposed in [17].

A refined convergence analysis of $$\hbox {pDCA}_{e}$$ pDCA e with applications to simultaneous sparse recovery and outlier detection

“…By assuming this new potential function is a KL function and F is level-bounded, we show that the whole sequence generated by pDCA e is convergent. We then study a relationship between the KL assumption used in this paper and the one used in [38]. Specifically, under a suitable smoothness assumption on P 2 , we show that if the potential function used in [38] has a KL exponent of 1 2 , so does our new potential function.…”

“…More importantly, as we shall see later in Section 5, the objectives of models for simultaneous sparse recovery and outlier detection can be written as DC functions whose concave parts are typically nonsmooth. Thus, for these problems, the analysis in [38] cannot be applied to studying global sequential convergence nor local convergence rate of the sequence generated by pDCA e .…”

“…In this paper, we refine the convergence analysis of pDCA e in [38] to cover the case when the P 2 in (1) is possibly nonsmooth. Our analysis is based on a potential function different from the one used in [38]. By assuming this new potential function is a KL function and F is level-bounded, we show that the whole sequence generated by pDCA e is convergent.…”

“…In Section 2, we introduce notation and preliminary results. We also review some convergence properties of the pDCA e from [38]. In Section 3, we present our refined global convergence analysis for pDCA e .…”

Sparse optimization problem with s-difference regularization

“…To address these nonconvex regularization problems, many iterative algorithms are investigated by researchers, such as the DCA [45][46][47][48] (or Convex-ConCave Procedure (CCCP) [49], or the Multi-Stage (MS) convex relaxation [22]), and its accelerate versions: Boosted Difference of Convex function Algorithms (BDCA) [50] and proximal Difference-of-Convex Algorithm with extrapolation (pDCAe) [51], the alternating direction method of multipliers (ADMM) [52], split Bregman iteration (SBI) [53], General Iterative Shrinkage and Thresholding (GIST) [54], nonmonotone accelerated proximal gradient (nmAPG) [55], which is an extension of the APG [56].…”