Decentralized partitioning over adaptive networks

Khawatmi, Sahar; Zoubir, Abdelhak M.

doi:10.1109/mlsp.2016.7738880

Cited by 2 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conditions (13) and (14) state that the P columns of U are right and left eigenvectors of A associated with the eigenvalue 1. Together with these two conditions, condition (12) means that A has P eigenvalues at one, and that all other eigenvalues are strictly less than one in magnitude.…”

Section: Distributed Inference Under Subspace Constraintsmentioning

confidence: 99%

“…Observe that this strategy can be written in the form of (9) with A k = a k I L and A = A ⊗ I L . It can be verified that, when A satisfies (15) over a strongly connected network, the matrix A will satisfy (8), (13), (14), and (12). with overlapping parameter vectors [22]- [24] can also be recast in the form (2).…”

Section: Distributed Inference Under Subspace Constraintsmentioning

confidence: 99%

“…The next statement exploits the eigen-structure of A e that will be useful for establishing the mean-square stability. (13), (14), and (12) admits a Jordan canonical decomposition of the form:…”

Section: A Modeling Conditionsmentioning

confidence: 99%

“…We start by noting that the M × M matrix A satisfying conditions (13), (14), and (12) admits a Jordan canonical decomposition of the form:…”

Section: Appendix C Proof Of Lemmamentioning

confidence: 99%

See 3 more Smart Citations

Adaptation and Learning Over Networks Under Subspace Constraints—Part I: Stability Analysis

Nassif

Vlaski

Sayed

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in low-dimensional subspaces. This constrained formulation includes consensus optimization as a special case, and allows for more general task relatedness models such as smoothness. While such formulations can be solved via projected gradient descent, the resulting algorithm is not distributed. Starting from the centralized solution, we propose an iterative and distributed implementation of the projection step, which runs in parallel with the stochastic gradient descent update. We establish in this Part I of the work that, for small step-sizes µ, the proposed distributed adaptive strategy leads to small estimation errors on the order of µ. We examine in the accompanying Part II [2] the steady-state performance. The results will reveal explicitly the influence of the gradient noise, data characteristics, and subspace constraints, on the network performance. The results will also show that in the small step-size regime, the iterates generated by the distributed algorithm achieve the centralized steady-state performance.where J k (·) is the cost function at agent k, N is the number of agents in the network, and w ∈ C L is the global parameter vector, which all agents need to agree upon-see Fig. 1 (middle). Each agent seeks to estimate w o through local computations and communications among neighboring agents without the need to know any of the costs besides their own. Among many useful strategies that have been proposed in the literature [3]-[10], diffusion strategies [3]-[5] are particularly attractive since they are scalable, robust, and enable continuous learning and adaptation in response to drifts in the location of the minimizer. However, there exist many network applications that require more complex models and flexible algorithms than consensus implementations since their agents may involve the need to estimate and track multiple distinct, though related, objectives. For instance, in distributed power system state estimation, the local state vectors to be estimated at neighboring control centers may overlap partially since the areas in a power system are interconnected [11], [12]. Likewise, in monitoring applications, agents need to track the movement of multiple correlated targets and to exploit the correlation profile in the data for enhanced accuracy [13], [14]. Problems of this kind, where nodes need to infer multiple, though related, parameter vectors, are referred to as multitask problems. Existing strategies to address multitask problems generally exploit prior knowledge on how the tasks across the network relate to each other [11]-[29]. For example, one way to model relationships among tasks is to formulate convex optimization problems with appropriate co-regularizers between neighboring agents [13], [16]-[19]. Graph spectral regularization can also be used in order t...

show abstract

Section: Distributed Inference Under Subspace Constraintsmentioning

confidence: 99%

Section: Distributed Inference Under Subspace Constraintsmentioning

confidence: 99%

Section: A Modeling Conditionsmentioning

confidence: 99%

“…We start by noting that the M × M matrix A satisfying conditions (13), (14), and (12) admits a Jordan canonical decomposition of the form:…”

Section: Appendix C Proof Of Lemmamentioning

confidence: 99%

See 2 more Smart Citations

Adaptation and Learning Over Networks Under Subspace Constraints—Part I: Stability Analysis

Nassif

Vlaski

Sayed

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…An analogy would be a school of fish tracking a food source: all elements in the fish school sense distance and direction to the same food source and are interested in approaching it. On the other hand, multi-task networks [9,10,11,12,13,14,15,16,17] involve agents sensing data arising from different models and different clusters of agents may be interested in identifying separate models. A second analogy is a school of fish sensing information about multiple food sources.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Decentralized Decision-Making Over Multi-Task Networks

2019

Self Cite

View full text Add to dashboard Cite

In important applications involving multi-task networks with multiple objectives, agents in the network need to decide between these multiple objectives and reach an agreement about which single objective to follow for the network.In this work we propose a distributed decision-making algorithm. The agents are assumed to observe data that may be generated by different models. Through localized interactions, the agents reach agreement about which model to track and interact with each other in order to enhance the network performance. We investigate the approach for both static and mobile networks. The simulations illustrate the performance of the proposed strategies.

show abstract

Decentralized partitioning over adaptive networks

Cited by 2 publications

References 17 publications

Adaptation and Learning Over Networks Under Subspace Constraints—Part I: Stability Analysis

Adaptation and Learning Over Networks Under Subspace Constraints—Part I: Stability Analysis

Decentralized Decision-Making Over Multi-Task Networks

Contact Info

Product

Resources

About