Distributed Nesterov Gradient Methods Over Arbitrary Graphs

Ran, Xin; Jakovetić, Dušan; Khan, Usman A.

doi:10.1109/lsp.2019.2925537

Cited by 52 publications

(41 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where y k ∈ ℝ n is a variable and is corrected in the next update. Refer to [31], this method removes the gaps between the lower oracle complexity bounds of the function class. Furthermore, the Nesterov method corrects the gradient in every iteration and makes the update of the gradient more flexible, in some way, which can accelerate the convergence.…”

Section: The Nesterov Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Convergence of an accelerated distributed optimisation algorithm over time‐varying directed networks

et al. 2020

IET Control Theory & Appl

View full text Add to dashboard Cite

In this article, studying distributed optimisation over time-varying directed networks where a group of agents aims at cooperatively minimising a sum of local objective functions is focused on. Each agent uses only local computation and communication in the overall process without leaking their private information. Via incorporating both a distributed heavyball method and a distributed Nesterov method, a double accelerated distributed algorithm leveraging a gradient-tracking technique and using uncoordinated step-sizes, is developed. By employing both row-and column-stochastic weight matrices, the proposed algorithm can bypass the implementation of doubly stochastic weight matrices and avoid eigenvector estimation existing in some algorithms using only row-or column-stochastic weight matrices. Under the assumptions that the agents' local objective functions are smooth and strongly convex, and the aggregated directed networks of every finite consecutive directed network are strongly connected, the proposed algorithm is proved to converge linearly to the global optimal solution when the largest step-size is positive and sufficiently small, and the largest momentum parameter is non-negative. The proposed algorithm is also applied to fixed directed networks which are considered as a special case of time-varying directed networks. Simulation results further verify the effectiveness of the proposed algorithm and correctness of the theoretical findings.

show abstract

Section: The Nesterov Methodsmentioning

confidence: 99%

“…, referring to [26][27][28][29][30][31]39]. When dealing with the distributed optimisation problem over time-varying directed networks, TV- algorithm converges linearly.…”

Section: The Tv- Algorithmmentioning

confidence: 99%

Convergence of an accelerated distributed optimisation algorithm over time‐varying directed networks

et al. 2020

IET Control Theory & Appl

View full text Add to dashboard Cite

show abstract

“…Nesterov-type acceleration was first proposed in [29] by Nesterov which was originally aimed at accelerating gradient descent-type (first-order) methods. Recently, this technique has been extended to wider applications, such as ADMM-based algorithms [30], gradient tracking-based algorithms [31]. Inspired by these works, the acceleration version of RDDGT with Nesterov momentum named RDDGT-N can be derived where β is the momentum parameter and the x-update is changed to a two-step process:…”

Section: Algorithm Accelerationmentioning

confidence: 99%

Distributed Dual Gradient Tracking for Economic Dispatch in Power Systems with Noisy Information

Wu¹,

Liu²,

Zhu³

2021

Preprint

View full text Add to dashboard Cite

Distributed algorithms can be efficiently used for solving economic dispatch problem (EDP) in power systems. To implement a distributed algorithm, a communication network is required, making the algorithm vulnerable to noise which may cause detrimental decisions or even instability. In this paper, we propose an agent-based method which enables a fully distributed solution of the EDP in power systems with noisy information exchange. Through the novel design of the gradient tracking update and introducing suppression parameters, the proposed algorithm can effectively alleviate the impact of noise and it is shown to be more robust than the existing distributed algorithms. The convergence of the algorithm is also established under standard assumptions. Moreover, a strategy are presented to accelerate our proposed algorithm. Finally, the algorithm is tested on several IEEE bus systems to demonstrate its effectiveness and scalability.

show abstract

“…Saadatniaki, Xin and Khan [28] propose a variant of AB/push-pull method with time-varying weight matrices and show that the proposed method converges linearly to the optimal solution when each local objective is smooth and the global objective is strongly-convex. Accelerate techniques have also been incorporated into AB/push-pull method, such as, Xin and Khan [29] employ the heavyball method to accelerate the AB/push-pull method where the R-linear rate for strongly convex smooth objective has been obtained; Xin, Jakoveti and Khan [30] combine nesterov gradient method with AB/push-pull method and show that the new method achieves robust numerical performance for both convex and strongly-convex objectives. Moreover, the extended versions of AB/push-pull method have been used to solve practical problem, such as, resource allocation [31] and the distributed multi-cluster game [32].…”

Section: Introductionmentioning

confidence: 99%

Asymptotic Properties of $\mathcal{S}$-$\mathcal{AB}$ Method with Diminishing Stepsize

Zhao¹,

Liu²

2021

Preprint

View full text Add to dashboard Cite

The popular AB/push-pull method for distributed optimization problem may unify much of the existing decentralized first-order methods based on gradient tracking technique. More recently, the stochastic gradient variant of AB/Push-Pull method (S-AB) has been proposed, which achieves the linear rate of converging to a neighborhood of the global minimizer when the step-size is constant. This paper is devoted to the asymptotic properties of S-AB with diminishing step-size. Specifically, under the condition that each local objective is smooth and the global objective is strongly-convex, we first present the boundedness of the iterates of S-AB and then show that the iterates converge to the global minimizer with the rate O 1/ √ k . Furthermore, the asymptotic normality of Polyak-Ruppert averaged S-AB is obtained and applications on statistical inference are discussed. Finally, numerical tests are conducted to demonstrate the theoretic results.

show abstract

Distributed Nesterov Gradient Methods Over Arbitrary Graphs

Cited by 52 publications

References 47 publications

Convergence of an accelerated distributed optimisation algorithm over time‐varying directed networks

Convergence of an accelerated distributed optimisation algorithm over time‐varying directed networks

Distributed Dual Gradient Tracking for Economic Dispatch in Power Systems with Noisy Information

Asymptotic Properties of $\mathcal{S}$-$\mathcal{AB}$ Method with Diminishing Stepsize

Contact Info

Product

Resources

About