Decentralized Dual Proximal Gradient Algorithms for Non-Smooth Constrained Composite Optimization Problems

Li, Huaqing; Hu, Jinhui; Ran, Liang; Wang, Zheng; Lü, Qingguo; Du, Zhenyuan; Huang, Tingwen

doi:10.1109/tpds.2021.3072373

Cited by 14 publications

(2 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…D ISTRIBUTED optimization has found extensive applications in various fields such as machine learning [1], [2], deep learning [3], power systems [4], [5], signal processing [6], resource allocation [7], and distributed model predictive control [8], [9] thanks to its advantages of alleviating the computational burden for the agents, high efficiency for the multi-agent system, and guaranteed privacy for each agent in a peer-to-peer network. However, when facing a category of large-scale optimization problems, distributed batch gradient methods still suffer from a high per-iteration computational complexity result from the local batch gradient computation at each iteration.…”

Section: Introductionmentioning

confidence: 99%

Push-LSVRG-UP: Distributed Stochastic Optimization Over Unbalanced Directed Networks With Uncoordinated Triggered Probabilities

Chen

et al. 2023

IEEE Trans. Netw. Sci. Eng.

Self Cite

View full text Add to dashboard Cite

Distributed stochastic optimization, arising in the crossing and integration of traditional stochastic optimization, distributed computing and storage, and network science, has advantages of high efficiency and a low per-iteration computational complexity in resolving large-scale optimization problems. This paper concentrates on resolving a large-scale convex finite-sum optimization problem in a multi-agent system over unbalanced directed networks. To tackle this problem in an efficient way, a distributed consensus optimization algorithm, adopting the push-sum technique and a distributed loopless stochastic variance-reduced gradient (LSVRG) method with uncoordinated triggered probabilities, is developed and named Push-LSVRG-UP. Each agent under this algorithmic framework performs only local computation and communicates only with its neighbors without leaking their private information. The convergence analysis of Push-LSVRG-UP is relied on analyzing the contraction relationships between four error terms associated with the multi-agent system. Theoretical results provide an explicit feasible range of the constant step-size, a linear convergence rate, and an iteration complexity of Push-LSVRG-UP when achieving the globally optimal solution. It is shown that Push-LSVRG-UP achieves the superior characteristics of accelerated linear convergence, fewer storage costs, and a lower per-iteration computational complexity than most existing works. Meanwhile, the introduction of an uncoordinated probabilistic triggered mechanism allows Push-LSVRG-UP to facilitate the independence and flexibility of agents in computing local batch gradients. In simulations, the practicability and improved performance of Push-LSVRG-UP are manifested via resolving two distributed learning problems based on real-world datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Push-LSVRG-UP: Distributed Stochastic Optimization Over Unbalanced Directed Networks With Uncoordinated Triggered Probabilities

Chen

et al. 2023

IEEE Trans. Netw. Sci. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, as the size of data continues to grow, this kind of centralized strategy is limited by the computing power of the hardware. In contrast to this, the distributed methods distribute computing tasks to agents over decentralized networks [ 1 , 2 ]. Each agent keeps an arithmetic unit and a memory unit.…”

Section: Introductionmentioning

confidence: 99%

Decentralized Primal-Dual Proximal Operator Algorithm for Constrained Nonsmooth Composite Optimization Problems over Networks

Feng

Ran

Meng

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

In this paper, we focus on the nonsmooth composite optimization problems over networks, which consist of a smooth term and a nonsmooth term. Both equality constraints and box constraints for the decision variables are also considered. Based on the multi-agent networks, the objective problems are split into a series of agents on which the problems can be solved in a decentralized manner. By establishing the Lagrange function of the problems, the first-order optimal condition is obtained in the primal-dual domain. Then, we propose a decentralized algorithm with the proximal operators. The proposed algorithm has uncoordinated stepsizes with respect to agents or edges, where no global parameters are involved. By constructing the compact form of the algorithm with operators, we complete the convergence analysis with the fixed-point theory. With the constrained quadratic programming problem, simulations verify the effectiveness of the proposed algorithm.

show abstract

Distributed Composite Optimization for Multi-agent Systems with Asynchrony

Ran

et al. 2022

Complex Systems: Spanning Control and Computational Cybernetics: Foundations

View full text Add to dashboard Cite

Decentralized Dual Proximal Gradient Algorithms for Non-Smooth Constrained Composite Optimization Problems

Cited by 14 publications

References 35 publications

Push-LSVRG-UP: Distributed Stochastic Optimization Over Unbalanced Directed Networks With Uncoordinated Triggered Probabilities

Push-LSVRG-UP: Distributed Stochastic Optimization Over Unbalanced Directed Networks With Uncoordinated Triggered Probabilities

Decentralized Primal-Dual Proximal Operator Algorithm for Constrained Nonsmooth Composite Optimization Problems over Networks

Distributed Composite Optimization for Multi-agent Systems with Asynchrony

Contact Info

Product

Resources

About