Estimate exchange over network is good for distributed hard thresholding pursuit

Zaki, Ahmed; Mitra, Partha P.; Rasmussen, Lars K.; Chatterjee, Saikat

doi:10.1016/j.sigpro.2018.10.010

Cited by 10 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pruned BPDN (pNBPDN) is more close to distributed greedy algorithms that work with the same system setup. Examples of the greedy algorithms are network greedy pursuit (NGP) [22] and distributed hard thresholding pursuit (DHTP) [31]. We mention that NGP and DHTP have RIP conditions δ 3s (A l ) < 0.362 and δ 3s (A l ) < 0.333, respectively.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Convex Optimization Based Sparse Learning Over Networks

Zaki

Chatterjee

2019

2019 27th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum solution for that node. The nodes exchange their signal estimates over the network in order to refine their local estimates. At a node, the optimization algorithm is based on an ℓ1-norm minimization with appropriate modifications to promote sparsity as well as to include influence of estimates from neighboring nodes. Our expectation is that local estimates in each node improve fast and converge, resulting in a limited demand for communication of estimates between nodes and reducing the processing time. We provide restricted-isometry-property (RIP)-based theoretical analysis on estimation quality. In the scenario of clean observation, it is shown that the local estimates converge to the exact sparse signal under certain technical conditions. Simulation results show that the proposed algorithms show competitive performance compared to a globally optimum distributed LASSO algorithm in the sense of convergence speed and estimation error.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Our algorithms are locally optimum at each node of the network. This endeavor is an extension of past work in designing distributed greedy algorithms [31], [22] to the regime of convex optimization.…”

Section: B Literature Reviewmentioning

confidence: 99%

Convex Optimization Based Sparse Learning Over Networks

Zaki

Chatterjee

2019

2019 27th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Essentially, the model ( 1) is to find the best k-term approximation of the target signal, which best fits the acquired measurements. Similar models also arise in numerous areas such as statistical learning [5]- [7], wireless communication [8], [9], low-rank matrix recovery [10]- [14], linear inverse and optimization problems [15]- [18]. While the SCO and related problems can be solved via convex optimization, nonconvex optimization and orthogonal matching pursuit (OMP) (see, e.g., [1]- [4]), the thresholding methods with low computational complexity are particularly suited for solving the SCO model.…”

Section: Introductionmentioning

confidence: 95%

“…Proof. (i) Under ( 17), Theorem 3.4 claims that the sequence {x (p) : p ≥ 1}, generated by NT and NT q with λ p ≡ 1 and concave g α (•), satisfies (18). Note that A T ν ′ 2 ≤ σ max (A) ν ′ 2 , where σ max (A) is the largest singular value of A.…”

Section: Guaranteed Performance and Stabilitymentioning

confidence: 99%

Natural Thresholding Algorithms for Signal Recovery With Sparsity

Zhao

Luo

2022

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

The algorithms based on the technique of optimal k-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal k-thresholding model, and thus the computational cost for this family of algorithms is significantly lower than that of the OT-based algorithms. The guaranteed performance of NT-type algorithms for signal recovery from noisy measurements is shown under the restricted isometry property and concavity of the objective function of regularized optimal kthresholding model. Empirical results indicate that the NT-type algorithms are robust and very comparable to several mainstream algorithms for sparse signal recovery.

show abstract

“…The hard thresholding methods have widely been studies in the area of compressed sensing or sparse approximation [5,6,7,29,30,4]. The latest development and applications of these methods can be found in [8,9,34,46,52,54,48]. Although the sparse optimization problems like (1) arising from compressed sensing are usually NP-hard [42], it does not prohibit the fast development of various computational methods for such problems.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Optimal Thresholding Algorithms for Compressed Sensing

Zhao

Luo

2019

Preprint

View full text Add to dashboard Cite

The optimal thresholding is a new technique that has recently been developed for compressed sensing and signal approximation. The optimal k-thresholding (OT) and optimal k-thresholding pursuit (OTP) provide basic algorithmic frameworks that are fundamental to the development of practical and efficient optimal-thresholding-type algorithms for compressed sensing. In this paper, we first prove an improved bound for the guaranteed performance of these basic frameworks in terms of the kth order restricted isometry property of the sensing matrices. More importantly, we analyze the newly developed algorithms called relaxed optimal thresholding (ROTω) and relaxed optimal thresholding pursuit (ROTPω) which are derived from the tightest convex relaxation of the OT and OTP. Numerical results in [54] have demonstrated that such approaches are truly efficient to recover a wide range of signals and can remarkably outperform the existing hard-thresholding-type methods as well as the classic ℓ 1 -minimization in numerous situations. However, the guaranteed performance/convergence of these algorithms with ω ≥ 2 has not yet established. The main purpose of this paper is to establish the first guaranteed performance results for the ROTω and ROTPω.

show abstract

Estimate exchange over network is good for distributed hard thresholding pursuit

Cited by 10 publications

References 46 publications

Convex Optimization Based Sparse Learning Over Networks

Convex Optimization Based Sparse Learning Over Networks

Natural Thresholding Algorithms for Signal Recovery With Sparsity

Analysis of Optimal Thresholding Algorithms for Compressed Sensing

Contact Info

Product

Resources

About