An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing

Abstract

This paper considers the convex minimization problem with linear constraints and a separable objective function which is the sum of many individual functions without coupled variables. An algorithm is developed by splitting the augmented Lagrangian function in a parallel way. The new algorithm differs substantially from existing splitting methods in alternating style which require solving the decomposed subproblems sequentially, while it remains the main superiority of existing splitting methods in that the resulting subproblems could be simple enough to have closed-form solutions for such an application whose functions in the objective are simple. We show applicability and encouraging efficiency of the new algorithm by some applications in image processing.Received by the editor February 16, 2012 and, in revised form, December 1, 2012. 2010 Mathematics Subject Classification. Primary 90C06, 90C25, 94A08.

A distributed Douglas-Rachford splitting method for multi-block convex minimization problems

“…The second one is Jacobiantype splitting method, which decomposes the quadratic term in a Jacobian way such that the resulting subproblems can be solved simultaneously. Specifically, for given initial points x k 1 , · · · , x k m , λ k , they first generate the predictor via ( 1.3b) then update the next iterate via an additional correction step to guarantee their global convergence, e.g., see [25,28,30,31] for more details. It is clear that both types of splitting methods can make full use of the separable property.…”

“…From the data reported in [35,49], we observe that less number of correction variables may get more promising numerical results. Moreover, the numerical results in [28] show that Jacobian-type splitting methods outperform Gauss-Seidel-type methods for solving large-scale problems in terms of computing time. Thus, we hopefully explore an algorithm that not only possess some easier subproblems in many cases, but also inherits the advantages of Gauss-Seidel-type and Jacobian-type splitting methods such as less correction (prediction) variables and simultaneous implementation on subproblems.…”

“…All the resulting subproblems (3.2) are simple enough to have closed-form solutions as well as the subproblems in [28,49]. First, let us summarize two operators used in [6,7] for presenting the closed-form solutions.…”

“…Since our method (DDRSM) can also be regarded as a parallel splitting method, we only compare 'DDRSM' with two benchmark simultaneous splitting methods in the literature: 1) the parallel splitting augmented Lagrangian method in [29] ('PSALM' for short); 2) the augmented-Lagrangian-based parallel splitting method in [28] (denoted by 'ALPSM').…”

The augmented Lagrangian method with full Jacobian decomposition and logarithmic-quadratic proximal regularization for multiple-block separable convex programming

“…The scheme (1.7), however, is not necessarily convergent even when m = 2, as shown in [22]. In the literature, it was suggested to correct the output of (1.7) by some correction steps to ensure the convergence; some prediction-correction methods based on the Jacobian decomposition of ALM (1.7) were thus presented in the literature, see, e.g., [20,22]. Note that these prediction-correction methods usually converge fast for some applications arising in image processing and other areas.…”