Decentralized exact coupled optimization

Alghunaim, Sulaiman A.; Yuan, Kun; Sayed, Ali H.

doi:10.1109/allerton.2017.8262757

Cited by 7 publications

(18 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For sparse networks with a large number of parameters to estimate, it is much more efficient to devise distributed techniques that solve (3) directly rather than transform (3) into the form in (1) via vector extension. It is shown in the simulations in [32] and [35] that this extension technique not only increases complexity but it often degrades convergence performance as well, which we show analytically in this work.…”

Section: B Contribution and Related Workmentioning

confidence: 56%

“…Similar clustering was used in [35] for the deterministic problem where the ADMM method was employed with identical penalty factors across all clusters. In our previous work [32], we studied the same deterministic case but developed instead a first-order method for solving (3) without constraints by relying on the exact diffusion strategy from [41], [42], which unlike ADMM does not require inner minimization steps.…”

Section: B Contribution and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Coupled Multiagent Stochastic Optimization

Alghunaim

Sayed

2020

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

This work develops effective distributed strategies for the solution of constrained multi-agent stochastic optimization problems with coupled parameters across the agents. In this formulation, each agent is influenced by only a subset of the entries of a global parameter vector or model, and is subject to convex constraints that are only known locally. Problems of this type arise in several applications, most notably in disease propagation models, minimum-cost flow problems, distributed control formulations, and distributed power system monitoring. This work focuses on stochastic settings, where a stochastic risk function is associated with each agent and the objective is to seek the minimizer of the aggregate sum of all risks subject to a set of constraints. Agents are not aware of the statistical distribution of the data and, therefore, can only rely on stochastic approximations in their learning strategies. We derive an effective distributed learning strategy that is able to track drifts in the underlying parameter model. A detailed performance and stability analysis is carried out showing that the resulting coupled diffusion strategy converges at a linear rate to an O(µ)−neighborhood of the true penalized optimizer.

show abstract

Section: B Contribution and Related Workmentioning

confidence: 56%

Section: B Contribution and Related Workmentioning

confidence: 99%

Distributed Coupled Multiagent Stochastic Optimization

Alghunaim

Sayed

2020

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, each constraint involves the neighborhood of an agent, so that for these types of problems we can select C e = N s (for s = e) since neighborhoods are naturally connected. Now, more generally, even if some chosen subnetwork happens to be disconnected, we can always embed it into a larger connected sub-network as long as the entire network is connected -an explanation of this embedding procedure can be found in [28], [30]. We now provide one example.…”

Section: Problem Formulationmentioning

confidence: 99%

A Proximal Diffusion Strategy for Multiagent Optimization With Sparse Affine Constraints

Alghunaim

Yuan

Sayed

2020

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

This work develops a proximal primal-dual distributed strategy for multi-agent optimization problems that involve multiple coupled affine constraints, and where each constraint may involve only a subset of the agents. The constraints are generally sparse, meaning that only a small subset of the agents are involved in them. This scenario arises in many applications including distributed control formulations, resource allocation problems, and smart grids. Traditional distributed solutions tend to ignore the structure of the constraints and lead to degraded performance. We instead develop a distributed solution that exploits the sparsity structure. Under constant step-size learning, we establish the asymptotic convergence of the distributed algorithm in the presence of non-smooth terms, and further show that convergence occurs at a linear rate in the smooth case. We also examine how the performance of the algorithm is influenced by the sparsity of the constraints. Simulations illustrate the superior performance of the proposed strategy.

show abstract

“…There have been many works in the literature studying distributed multitask adaptive strategies and their convergence behavior. Nevertheless, with few exceptions [20], most of these works focus on mean-square-error costs.…”

Section: Introductionmentioning

confidence: 99%

“…Based on the type of prior information that may be available about how the tasks are related to each other, multitask learning algorithms can be derived by translating the prior information into constraints on the parameter vectors to be inferred [12]- [22]. For example, in [18]- [20], distributed strategies are developed under the assumption that the parameter vectors across the agents overlap partially. A more general scenario is considered in [21] where it is assumed that the tasks across the agents are locally coupled through linear equality constraints.…”

Section: Introductionmentioning

confidence: 99%

Learning Over Multitask Graphs—Part I: Stability Analysis

Nassif

Vlaski

Richard

et al. 2020

IEEE Open J. Signal Process.

Self Cite

View full text Add to dashboard Cite

This paper formulates a multitask optimization problem where agents in the network have individual objectives to meet, or individual parameter vectors to estimate, subject to a smoothness condition over the graph. The smoothness condition softens the transition in the tasks among adjacent nodes and allows incorporating information about the graph structure into the solution of the inference problem. A diffusion strategy is devised that responds to streaming data and employs stochastic approximations in place of actual gradient vectors, which are generally unavailable.The approach relies on minimizing a global cost consisting of the aggregate sum of individual costs regularized by a term that promotes smoothness. We show in this Part I of the work, under conditions on the step-size parameter, that the adaptive strategy induces a contraction mapping and leads to small estimation errors on the order of the small step-size. The results in the accompanying Part II will reveal explicitly the influence of the network topology and the regularization strength on the network performance and will provide insights into the design of effective multitask strategies for distributed inference over networks.

show abstract

Decentralized exact coupled optimization

Cited by 7 publications

References 37 publications

Distributed Coupled Multiagent Stochastic Optimization

Distributed Coupled Multiagent Stochastic Optimization

A Proximal Diffusion Strategy for Multiagent Optimization With Sparse Affine Constraints

Learning Over Multitask Graphs—Part I: Stability Analysis

Contact Info

Product

Resources

About