Convolutional Dictionary Learning: A Comparative Review and New Algorithms

Garcia-Cardona, Cristina; Wohlberg, Brendt

doi:10.1109/tci.2018.2840334

Cited by 157 publications

(122 citation statements)

References 38 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…To solve the problem (6), we utilize the Alternating Direction Method of Multipliers (ADMM) [50]- [53] in this paper. We refer the readers to [54] for other algorithms to solve this problem. By introducing dual variable U, penalty parameter ρ and auxiliary variable Y, the corresponding scaled augmented Lagrangian function is defined as [50]:…”

Section: A Sparse Coefficients Updatementioning

confidence: 99%

Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration

Kang

Hong

Liu

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version please go to IEEE Transactions on Neural Networks and Learning Systems on IEEE Xplore. Interferometric phase restoration has been investigated for decades and most of the state-of-the-art methods have achieved promising performances for InSAR phase restoration. These methods generally follow the nonlocal filtering processing chain aiming at circumventing the staircase effect and preserving the details of phase variations. In this paper, we propose an alternative approach for InSAR phase restoration, i.e. Complex Convolutional Sparse Coding (Com-CSC) and its gradient regularized version. To our best knowledge, this is the first time that we solve the InSAR phase restoration problem in a deconvolutional fashion. The proposed methods can not only suppress interferometric phase noise, but also avoid the staircase effect and preserve the details. Furthermore, they provide an insight of the elementary phase components for the interferometric phases. The experimental results on synthetic and realistic high-and medium-resolution datasets from TerraSAR-X StripMap and Sentinel-1 interferometric wide swath mode, respectively, show that our method outperforms those previous state-of-the-art methods based on nonlocal InSAR filters, particularly the state-of-the-art method: InSAR-BM3D. The source code of this paper will be made publicly available for reproducible research inside the community.Index Terms-Convolutional dictionary learning, sparse coding, SAR interferometry (InSAR), nonlocal filtering

show abstract

Section: A Sparse Coefficients Updatementioning

confidence: 99%

Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration

Kang

Hong

Liu

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…One class of approaches, which is the one we follow in this work, uses greedy methods to solve the original problem with the 0 quasi-norm. Another class of approaches relaxes the 0 quasi-norm to the 1 norm, which converts the CSC objective into a convex optimization problem [12], [13], [14]. The advantage of greedy approaches is that they are more efficient computationally [7].…”

Section: Optimization Objectivementioning

confidence: 99%

“…The majority of existing CDU frameworks estimate the templates {h c } C c=1 by minimizing the error of reconstructing y using its linear approximation Hx. The key differences between existing approaches are the constraints imposed on the templates and the optimization methods used, as detailed in a recent survey [12]. To the best of our knowledge, existing CDU approaches do not address the problem of learning the templates in the presence of time-quantization errors.…”

Section: Cdu Frameworkmentioning

confidence: 99%

Convolutional Dictionary Learning With Grid Refinement

Song

Flores

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Given a continuous-time signal that can be modeled as the superposition of localized, time-shifted events from multiple sources, the goal of Convolutional Dictionary Learning (CDL) is to identify the location of the events-by Convolutional Sparse Coding (CSC)-and learn the template for each source-by Convolutional Dictionary Update (CDU). In practice, because we observe samples of the continuous-time signal on a uniformlysampled grid in discrete time, classical CSC methods can only produce estimates of the times when the events occur on this grid, which degrades the performance of the CDU. We introduce a CDL framework that significantly reduces the errors arising from performing the estimation in discrete time. Specifically, we construct an expanded dictionary that comprises, not only discrete-time shifts of the templates, but also interpolated variants, obtained by bandlimited interpolation, that account for continuous-time shifts. For CSC, we develop a novel computationally efficient CSC algorithm, termed Convolutional Orthogonal Matching Pursuit with interpolated dictionary (COMP-INTERP). We benchmarked COMP-INTERP to Contiunuous Basis Pursuit (CBP), the state-of-the-art CSC algorithm for estimating offthe-grid events, and demonstrate, on simulated data, that 1) COMP-INTERP achieves a similar level of accuracy, and 2) is two orders of magnitude faster. For CDU, we derive a novel procedure to update the templates given sparse codes that can occur both on and off the discrete-time grid. We also show that 3) dictionary update with the overcomplete dictionary yields more accurate templates. Finally, we apply the algorithms to the spike sorting problem on electrophysiology recording and show their competitive performance.

show abstract

“…For instance, the blind gain and phase calibration problem [LLB16, LS18, LLB18] is closely related to the MCS-BD problem, as mentioned by [LB18]. It is also of great interest to extend our approach for solving the so-called convolutional dictionary learning problem [CF17,GCW18], in which each measurement consists of a superposition of multiple circulant convolutions:…”

Section: Future Directionsmentioning

confidence: 99%

Exact and Efficient Multi-Channel Sparse Blind Deconvolution — A Nonconvex Approach

Xiao

Zhu

2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

View full text Add to dashboard Cite

We study the multi-channel sparse blind deconvolution (MCS-BD) problem, whose task is to simultaneously recover a kernel a and multiple sparse inputs txiu p i"1 from their circulant convolution yi " a f xi (i " 1,¨¨¨, p). We formulate the task as a nonconvex optimization problem over the sphere. Under mild statistical assumptions of the data, we prove that the vanilla Riemannian gradient descent (RGD) method, with random initializations, provably recovers both the kernel a and the signals txiu p i"1 up to a signed shift ambiguity. In comparison with state-of-the-art results, our work shows significant improvements in terms of sample complexity and computational efficiency. Our theoretical results are corroborated by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods on both synthetic and real datasets. 1 Here, (i) BGpθq and BRpθq denote Bernoulli-Gaussian and Bernoulli-Rademacher distribution, respectively; (ii) θ P r0, 1s is the Bernoulli parameter controlling the sparsity level of x i ; (iii) ε denotes the recovery precision of global solution a‹, i.e., min ℓ }a´s ℓ ra‹s} ď ε; (iv) r O and r Ω hides logpnq, θ and other factors. 2 Recently, similar loss has been considered for short and sparse deconvolution [ZKW18] and complete dictionary learning [ZYL`19]. 3 As the tail of BGpθq distribution is heavier than that of BRpθq, their sample complexity would be even worse if BGpθq model was considered. 4 We say x obeys a Bernoulli-Rademacher distribution when x " b d r where d denotes point-wise product, b follows i.i.d. Bernoulli distribution and r follows i.i.d. Rademacher distribution.Contributions of this paper. In this work, we introduce an efficient optimization method for solving MCS-BD. We consider a natural nonconvex formulation based on a smooth relaxation of ℓ 1 -loss. Under mild assumptions of the data, we prove the following result.With random initializations, a vanilla RGD efficiently finds an approximate solution, which can be refined by a subgradient method that converges exactly to the target solution in a linear rate.We summarize our main result in Table 1. By comparison 5 with [LB18], our approach demonstrates substantial improvements for solving MCS-BD in terms of both sample and time complexity. Moreover, our experimental results imply that our analysis is still far from tight -the phase transitions suggest that p ě Ωppoly logpnqq samples might be sufficient for exact recovery, which is favorable for applications (as real data in form of images can have millions of pixels, resulting in huge dimension n). Our analysis is inspired by recent results on orthogonal dictionary learning [GBW18, BJS18], but much of our theoretical analysis is tailored for MCS-BD with a few extra new ingredients. Our work is the first result provably showing that vanilla gradient descent type methods with random initialization solve MCS-BD efficiently. Moreover, our ideas could potentially lead to new algorithmic guarantees for other nonconvex problems such a...

show abstract

Convolutional Dictionary Learning: A Comparative Review and New Algorithms

Cited by 157 publications

References 38 publications

Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration

Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration

Convolutional Dictionary Learning With Grid Refinement

Exact and Efficient Multi-Channel Sparse Blind Deconvolution — A Nonconvex Approach

Contact Info

Product

Resources

About