A partially parallel splitting method for multiple-block separable convex programming with applications to robust PCA

Abstract

We consider a multiple-block separable convex programming problem, where the objective function is the sum of m individual convex functions without overlapping variables, and the constraints are linear, aside from side constraints. Based on the combination of the classical Gauss-Seidel and the Jacobian decompositions of the augmented Lagrangian function, we propose a partially parallel splitting method, which differs from existing augmented Lagrangian based splitting methods in the sense that such an approach simplifies the iterative scheme significantly by removing the potentially expensive correction step. Furthermore, a relaxation step, whose computational cost is negligible, can be incorporated into the proposed method to improve its practical performance. Theoretically, we establish global convergence of the new method in the framework of proximal point algorithm and worst-case nonasymptotic O(1/t) convergence rate results in both ergodic and nonergodic senses, where t counts the iteration. The efficiency of the proposed method is further demonstrated through numerical results on robust PCA, i.e., factorizing from incomplete information of an

A Proximal Fully Parallel Splitting Method for Stable Principal Component Pursuit

“…In this section, we run a series of numerical experiments and apply the proposed PFPSM to solve some concrete examples of SPCP (6). We compare it with the splitting method for separable convex programming (denoted by HTY) in [30], the full Jacobian decomposition of the augmented Lagrangian method (denoted by FJDALM) in [13], and the fully parallel ADMM (denoted by PADMM) in [14].…”

“…In this section, we shall present the proximal fully parallel splitting method for solving (6) and discuss its convergence results.…”

“…In this subsection, we are going to show the efficiency of the proposed method by computing the real-world problem: background extraction from surveillance video with missing and noisy data. In the experiment, following [6], we focus on a well-tested video taken in an airport, which consists of 50 grayscale frames with each frame having 144 × 176 pixels. We vectorize all frames of the video and get a matrix ∈ R 25344×50 where each column represents a frame, which is further contaminated with additive Gaussian noise of mean zero and standard deviation = 10 −3 .…”

“…Recently, there has been an intensive study on the iteration methods for solving (6) [5,6,[10][11][12]. Interestingly, these methods usually only exploit the first-order information of the objective function of (6), and most of them are based on the well-known alternating direction method with multipliers (ADMM), such as the sequential ADMM [11], the partially parallel ADMMs [5,6,10,12], and the fully parallel ADMMs [13,14]. As an influential first-order iteration method in optimization, ADMM was independently proposed by Glowinski and Marrocco [15] and Gabay and Mercier [16] about forty years ago, and due to its high efficiency in the numerical simulation, ADMM has been receiving considerable attention in recent years.…”

“…The second one is that a Glowinski relaxation factor ∈ ( √ 3/2, 2/ √ 3) is attached to the updating formula for its Lagrange multiplier, and larger values of often can accelerate the convergence of the proposed method (as to be reported in Section 4). To the best of our knowledge, this is the first fully parallel ADMMtype method with the Glowinski relaxation factor for solving problem (6). The third one is that it dynamically updates the step size .…”