A Sparsity Promoting Adaptive Algorithm for Distributed Learning

Chouvardas, Symeon; Slavakis, Konstantinos; Kopsinis, Yannis; Theodoridis, Sergios

doi:10.1109/tsp.2012.2204987

Cited by 92 publications

(80 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Under condition (20), the modified diffusion strategy (15)- (16) converges in the mean if the matrices A 1 and A 2 are constructed according to (18)- (19) and the step-sizes {µ k } satisfy condition (13) for those nodes whose observed model is the same as the desired model w • q for the network.…”

Section: Theoremmentioning

confidence: 99%

“…Using the arguments in Section VI, we assume in the following that the nodes have achieved agreement on the desired model, say, w • q as in (20). We know from the proof of Theorem 1 (see Appendix A) that a modified diffusion network is equivalent to a network with a mixture of informed and uninformed nodes, as studied in [58].…”

Section: B Convergence Rate Of Diffusion Adaptationmentioning

confidence: 99%

“…References [13], [14] provide overviews of diffusion techniques and their application to distributed adaptation, learning, and optimization over networks. Examples of further applications and studies appear, e.g., in [15]- [20].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Distributed Decision-Making Over Adaptive Networks

Sayed

2014

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

In distributed processing, agents generally collect data generated by the same underlying unknown model (represented by a vector of parameters) and then solve an estimation or inference task cooperatively.In this paper, we consider the situation in which the data observed by the agents may have risen from two different models. Agents do not know beforehand which model accounts for their data and the data of their neighbors. The objective for the network is for all agents to reach agreement on which model to track and to estimate this model cooperatively. In these situations, where agents are subject to data from unknown different sources, conventional distributed estimation strategies would lead to biased estimates relative to any of the underlying models. We first show how to modify existing strategies to guarantee unbiasedness. We then develop a classification scheme for the agents to identify the models that generated the data, and propose a procedure by which the entire network can be made to converge towards the same model through a collaborative decision-making process. The resulting algorithm is applied to model fish foraging behavior in the presence of two food sources.

show abstract

Section: Theoremmentioning

confidence: 99%

Section: B Convergence Rate Of Diffusion Adaptationmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Decision-Making Over Adaptive Networks

Sayed

2014

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Sparse diffusion schemes are presented in [17] and [18] that provide adaptive algorithms for distributed learning in networks. In [17], projection methods over hyperslabs and weighted l 1 -balls are presented and analyzed for distributed learning.…”

Section: Introductionmentioning

confidence: 99%

“…In [17], projection methods over hyperslabs and weighted l 1 -balls are presented and analyzed for distributed learning. Penalized cost functions are used in [18] to enforce the sparsity of the solution.…”

Section: Introductionmentioning

confidence: 99%

Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis

Taheri

Vorobyov

2014

Signal Processing

View full text Add to dashboard Cite

A new reweighted l 1 -norm penalized least mean square (LMS) algorithm for sparse channel estimation is proposed and studied in this paper. Since standard LMS algorithm does not take into account the sparsity information about the channel impulse response (CIR), sparsity-aware modifications of the LMS algorithm aim at outperforming the standard LMS by introducing a penalty term to the standard LMS cost function which forces the solution to be sparse. Our reweighted l 1 -norm penalized LMS algorithm introduces in addition a reweighting of the CIR coefficient estimates to promote a sparse solution even more and approximate l 0 -pseudo-norm closer. We provide in depth quantitative analysis of the reweighted l 1 -norm penalized LMS algorithm. An expression for the excess mean square error (MSE) of the algorithm is also derived which suggests that under the right conditions, the reweighted l 1 -norm penalized LMS algorithm outperforms the standard LMS, which is expected. However, our quantitative analysis also answers the question of what is the maximum sparsity level in the channel for which the reweighted l 1 -norm penalized LMS algorithm is better than the standard LMS. Simulation results showing the better performance of the reweighted l 1 -norm penalized LMS algorithm compared to other existing LMS-type algorithms are given.

show abstract

Sparsity-Aware Learning

Theodoridis

2015

Machine Learning

View full text Add to dashboard Cite

A Sparsity Promoting Adaptive Algorithm for Distributed Learning

Cited by 92 publications

References 42 publications

Distributed Decision-Making Over Adaptive Networks

Distributed Decision-Making Over Adaptive Networks

Reweighted l1-norm penalized LMS for sparse channel estimation and its analysis

Sparsity-Aware Learning

Contact Info

Product

Resources

About