Convergence and Applications of a Gossip-Based Gauss-Newton Algorithm

Xiao, Li; Scaglione, Anna

doi:10.1109/tsp.2013.2276440

Cited by 32 publications

(18 citation statements)

References 55 publications

(159 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some recent developments include the EXTRA algorithm proposed in [7], which guarantees convergence of the consensus-based gradient descent algorithm using fixed step size. Combined algorithms that mix consensus technique with other optimization methods have also been developed, e.g., the alternating direction method of multipliers (ADMM) [13][14][15], the primal-dual method [16] and the Gauss-Newton method [17]. Most of this prior art focuses on convex optimization problems.…”

Section: Relation To Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

Wai

Chang

Scaglione

2015

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In handling massive-scale signal processing problems arising from 'big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensusbased methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including 'sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.

show abstract

Section: Relation To Prior Workmentioning

confidence: 99%

“…Most of this prior art focuses on convex optimization problems. Only a few related works on consensus-based decentralized algorithms exist for nonconvex optimization, e.g., [14,[17][18][19]. In particular, a stochastic decentralized algorithm have been proposed in [18] for tackling a class of non-convex problems.…”

Section: Relation To Prior Workmentioning

confidence: 99%

A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

Wai

Chang

Scaglione

2015

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In light of this, various authors have proposed to tackle (1) through decentralized algorithms that are built on the average consensus protocol [9,10]. For example, [11][12][13][14][15][16][17][18] studied the decentralized gradient methods; [19,20] considered the Newton-type methods; [21][22][23][24][25] considered the decentralized primal-dual algorithms. [26,27] are built on the successive convex approximation framework.…”

Section: Introductionmentioning

confidence: 99%

“…[26,27] are built on the successive convex approximation framework. The convergence properties of these algorithms were investigated extensively, especially for convex objectives [4,[12][13][14][15][16][17][18][19][20][21][22][23][24][25]; for non-convex objectives, a few recent results can be found in [11,[25][26][27][28][29]. However, most prior work are projection-based such that each iteration of the above algorithms necessitates a projection step onto the constraint set C, or solving a sub-problem with similar complexity.…”

Section: Introductionmentioning

confidence: 99%

Decentralized Frank–Wolfe Algorithm for Convex and Nonconvex Problems

Wai

Lafond

Scaglione

et al. 2017

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

Decentralized optimization algorithms have received much attention due to the recent advances in network information processing. However, conventional decentralized algorithms based on projected gradient descent are incapable of handling high dimensional constrained problems, as the projection step becomes computationally prohibitive to compute. To address this problem, this paper adopts a projection-free optimization approach, a.k.a. the Frank-Wolfe (FW) or conditional gradient algorithm. We first develop a decentralized FW (DeFW) algorithm from the classical FW algorithm. The convergence of the proposed algorithm is studied by viewing the decentralized algorithm as an inexact FW algorithm. Using a diminishing step size rule and letting t be the iteration number, we show that the DeFW algorithm's convergence rate is O(1/t) for convex objectives; is O(1/t 2 ) for strongly convex objectives with the optimal solution in the interior of the constraint set; and is O(1/ √ t) towards a stationary point for smooth but non-convex objectives. We then show that a consensus-based DeFW algorithm meets the above guarantees with two communication rounds per iteration. Furthermore, we demonstrate the advantages of the proposed DeFW algorithm on low-complexity robust matrix completion and communication efficient sparse learning. Numerical results on synthetic and real data are presented to support our findings.

show abstract

“…The algorithm employs network gossiping [16], [17] based on the uncoordinated random exchange protocol [18]. The convergence of this method is analytically proven in [19].…”

Section: Introductionmentioning

confidence: 99%

Decentralized power system state estimation with reduced inter-area communication

Kashyap

Werner

Huang

2015

2015 IEEE International Conference on Digital Signal Processing (DSP)

View full text Add to dashboard Cite

In this paper, we present a new algorithm for decentralized state estimation in multi-area power systems. The proposed method features significantly reduced communication between areas when compared to the existing gossip-based Gauss-Newton approach. We achieve this by confining gossip iterations associated with a specific state variable to only those areas which actually observe that state variable. This increases the speed of convergence of the gossip iterations, while reducing the amount of communication required for the gossip iterations to converge. We demonstrate the efficacy of our solution with simulations on the IEEE 118-bus system with different communication models.

show abstract

Convergence and Applications of a Gossip-Based Gauss-Newton Algorithm

Cited by 32 publications

References 55 publications

A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

Decentralized Frank–Wolfe Algorithm for Convex and Nonconvex Problems

Decentralized power system state estimation with reduced inter-area communication

Contact Info

Product

Resources

About