Synchronous Parallel Block Coordinate Descent Method for Nonsmooth Convex Function Minimization

Dai, Yutong; Weng, Yang

doi:10.1007/s11424-020-8313-y

Cited by 5 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to (15), the variable xk is a stationary point to (1) if g k = 0 [34]. Hence, g k can be regarded as a measure of the stationarity of variable xk .…”

Section: Convergence Rate For Nonconvex Optimizationmentioning

confidence: 99%

“…A popular approach to solving problem (1) is the projected stochastic gradient descent (pSGD) [12]- [14]. In pSGD and its variants, the iteration variable is projected back onto X after taking a step in the direction of negative stochastic gradient [15]- [20]. Such algorithms are efficient when the computational cost of performing the projection is low, e.g., projecting onto a hypercube or simplex.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Momentum-based Frank-Wolfe Algorithm for Stochastic Optimization

Hou¹,

Zeng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper considers distributed stochastic optimization, in which a number of agents cooperate to optimize a global objective function through local computations and information exchanges with neighbors over a network. Stochastic optimization problems are usually tackled by variants of projected stochastic gradient descent. However, projecting a point onto a feasible set is often expensive. The Frank-Wolfe (FW) method has well-documented merits in handling convex constraints, but existing stochastic FW algorithms are basically developed for centralized settings. In this context, the present work puts forth a distributed stochastic Frank-Wolfe solver, by judiciously combining Nesterov's momentum and gradient tracking techniques for stochastic convex and nonconvex optimization over networks. It is shown that the convergence rate of the proposed algorithm is O(k − 1 2 ) for convex optimization, and O(1/log 2 (k)) for nonconvex optimization. The efficacy of the algorithm is demonstrated by numerical simulations against a number of competing alternatives.

show abstract

“…According to (15), the variable xk is a stationary point to (1) if g k = 0 [34]. Hence, g k can be regarded as a measure of the stationarity of variable xk .…”

Section: Convergence Rate For Nonconvex Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Distributed Momentum-based Frank-Wolfe Algorithm for Stochastic Optimization

Hou¹,

Zeng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…They established the global convergence and linear convergence rate for this method under a locally Lipschitz error bound assumption. Subsequently, Yun, Tseng and Toh [48] extended the BCGD algorithm proposed in [42] to the problem from R n to R m×n , which requires an additional convexity assumption on G. For the same type of problem, Dai and Weng [19] brought forward a synchronous parallel block coordinate descent algorithm with a randomized variant. Additionally, when G is not assumed to be convex, Hua and Yamashita [23] discussed a class of block coordinate proximal gradient algorithms based on the Bregman functions, which may be different at each iteration.…”

Section: Introductionmentioning

confidence: 99%

Smoothing Randomized Block-coordinate Proximal Gradient Algorithms for Nonsmooth Nonconvex Composite Optimization

Bian

2023

Preprint

View full text Add to dashboard Cite

In this paper, we propose a smoothing randomized block-coordinate proximal gradient (S-RBCPG) algorithm and a Bregman randomized block-coordinate proximal gradient (B-RBCPG) algorithm for minimizing the sum of two nonconvex nonsmooth functions and one of which is separable. The pivotal tool of our analysis is the connection of the proximal gradient mapping with V-proximal mapping and Bregman proximal mapping. S-RBCPG algorithm overcome the non-smoothness issue of the objective function by utilizing the smoothing technique and we establish its subsequential convergence. Further, B-RBCPG algorithm is designed for cases where the separable function is separately relative smooth (that is, each separation part is relative smooth). Then, we establish the global convergence and R-linear convergence rate of the B-RBCPG algorithm under expectation by assuming the Kurdyka-Łojasiewicz property on the objective function without any convexity assumption. Finally, we use numerical experiments to illustrate the effectiveness and convergence of the proposed algorithms. MSC Classification: 90C30 , 90C26 , 65K05

show abstract

Smoothing randomized block-coordinate proximal gradient algorithms for nonsmooth nonconvex composite optimization

Li,

Bian

2024

Numer Algor

View full text Add to dashboard Cite

Synchronous Parallel Block Coordinate Descent Method for Nonsmooth Convex Function Minimization

Cited by 5 publications

References 17 publications

Distributed Momentum-based Frank-Wolfe Algorithm for Stochastic Optimization

Distributed Momentum-based Frank-Wolfe Algorithm for Stochastic Optimization

Smoothing Randomized Block-coordinate Proximal Gradient Algorithms for Nonsmooth Nonconvex Composite Optimization

Smoothing randomized block-coordinate proximal gradient algorithms for nonsmooth nonconvex composite optimization

Contact Info

Product

Resources

About