A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization

Abstract

The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables. At each iteration of this method, a single block of variables is optimized, while the remaining variables are held fixed. To ensure the convergence of the BCD method, the subproblem to be optimized in each iteration needs to be solved exactly to its unique optimal solution. Unfortunately, these requirements are often too restrictive for many practical scenarios. In this paper, we study an alternative inexact BCD approach which updates the variable blocks by successively minimizing a sequence of approximations of f which are either locally tight upper bounds of f or strictly convex local approximations of f . We focus on characterizing the convergence properties for a fairly wide class of such methods, especially for the cases where the objective functions are either non-differentiable or nonconvex. Our results unify and extend the existing convergence results for many classical algorithms such as the BCD method, the difference of convex functions (DC) method, the expectation maximization (EM) algorithm, as well as the alternating proximal minimization algorithm.

Dynamic Resource Allocation for Virtualized Wireless Networks in Massive-MIMO-Aided and Fronthaul-Limited C-RAN

“…Therefore, the computational complexity is i 1 × log(ξ2) for Step 2. Since the proposed algorithm follows the block coordinate descent (BCD) method, its convergence is guaranteed [30]. In particular, at each BCD iteration, a single block of variables is optimized, while the remaining variables are held fixed.…”

Cyclic overlapping block coordinate search for optimizing building design

“…The quasi-equivalence of convergence criteria for non-overlapping and overlapping block coordinate searches has been proven (Chen, 2005;Cassioli et al, 2013;Razaviyayn et al, 2013) and the better convergence of random block coordinate search over cyclic block coordinate search for smooth and unsmooth problems has been observed (Nesterov, 2012;Richtárik et al, 2014).…”

“…The convergence of block-coordinate descent is problematic for non-convex problems and discontinuous problems (Razaviyayn et al, 2013). But several issues regarding BCD has been proven circumventable.…”

“…But several issues regarding BCD has been proven circumventable. Its convergence has been proven for both quasi-convex and pseudo-convex objective functions (Tseng, 2001;Razaviyayn et al, 2013). Moreover, the convergence of BCD in some kinds of non-differentiable problems has been proven when they can be reduced to dual problems (Tseng et al, 2009a, b).…”

A globally convergent algorithm for lasso-penalized mixture of linear regression models

“…We note that these algorithms are also known as a block successive lower-bound maximization (BSLM) algorithm in the language of Razaviyayn et al (2013).…”