Geometry and Symmetry in Short-and-Sparse Deconvolution

Kuo, Han-Wen; Zhang, Yuqian; Lau, Yenson; Wright, John

doi:10.1137/19m1237569

Cited by 25 publications

(46 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, we believe the probability tools of decoupling and measure concentration we developed here can form a solid foundation for studying other nonconvex problems under the random convolutional model. Those problems include blind calibration [80][81][82], sparse blind deconvolution [76,[83][84][85][86][87][88][89][90][91][92][93], and convolutional dictionary learning [16,[94][95][96][97]].…”

Section: Geometric Analysis and Global Resultmentioning

confidence: 99%

Convolutional Phase Retrieval via Gradient Descent

Zhang

Eldar

et al. 2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

We study the convolutional phase retrieval problem, of recovering an unknown signal x ∈ C n from m measurements consisting of the magnitude of its cyclic convolution with a given kernel a ∈ C m . This model is motivated by applications such as channel estimation, optics, and underwater acoustic communication, where the signal of interest is acted on by a given channel/filter, and phase information is difficult or impossible to acquire. We show that when a is random and the number of observations m is sufficiently large, with high probability x can be efficiently recovered up to a global phase shift using a combination of spectral initialization and generalized gradient descent. The main challenge is coping with dependencies in the measurement operator. We overcome this challenge by using ideas from decoupling theory, suprema of chaos processes and the restricted isometry property of random circulant matrices, and recent analysis of alternating minimization methods. ). Most of the work has been conducted when QQ and YZ were with Department of Electrical Engineering and Data Science Institute at Columbia University. An extended abstract of the current work has appeared in NeurIPS'17 [1]. 1 The convolution can be written in matrix-vector form as a x = Caιn→mx, where Ca ∈ R m×m denotes the circulant matrix generated by a and ιn→m is a zero padding operator. In other words, a x = Ax with A ∈ C m×n being a matrix formed by the first n columns of Ca. arXiv:1712.00716v3 [stat.CO] 6 Oct 2019Structured random measurements. The study of structured random measurements in signal processing has quite a long history [44]. For compressed sensing [45], the work [46-48] studied random Fourier measurements, and later [13,14] proved similar results for partial random convolution measurements. However, the study of structured random measurements for phase retrieval is still quite limited. In particular, [49] and [50] studied t-designs and coded diffraction patterns (i.e., random masked Fourier measurements) using semidefinite programming. Recent work studied nonconvex optimization using coded diffraction patterns [34] and STFT measurements [51], both of which minimize a nonconvex objective similar to (1.5). These measurement models are motivated by different applications. For instance, coded diffraction is designed for imaging applications such as X-ray diffraction imaging, STFT can be applied to frequency resolved optical gating [52] and some speech processing tasks [53]. Both of the results show iterative contraction in a region that is at most O(1/ √ n)-close to the optimum. Unfortunately, for both results either the radius of the contraction region is not large enough for initialization to reach, or they require extra artificial technique such as resampling the data. In comparison, the contraction region we show for the random convolutional model is larger O(1/polylog(n)), which is achievable in the initialization stage via the spectral method. For a more detailed review of this subject, we refer the readers to Section 4 of [44]....

show abstract

Section: Geometric Analysis and Global Resultmentioning

confidence: 99%

Convolutional Phase Retrieval via Gradient Descent

Zhang

Eldar

et al. 2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other algorithms for multi-channel blind deconvolution include sparse spectral methods [21] and nonconvex regularization [22]. A different model called "sparse-and-short" deconvolution is studied in [6,23]. The work of Li and Bresler [4] is the most related to ours, which considered the same problem by running perturbed manifold gradient descent with a random initialization, over a spherically constrained loss function based on ℓ 4 norm maximization.…”

Section: Related Workmentioning

confidence: 99%

“…When the filter is unknown, it leads to the so-called blind deconvolution problem. This problem is ill-posed without extra assumptions on the filter, since the number of unknowns is much larger than the number of observations [4,5,6,7,8]. Luckily, in many situations, one can make multiple observations sharing the same filter, but with diverse sparse inputs, either spatially or temporally, thanks to the advances of sensing technologies.…”

Section: Introductionmentioning

confidence: 99%

Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently

Shi

Chi

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Multi-channel sparse blind deconvolution, or convolutional sparse coding, refers to the problem of learning an unknown filter by observing its circulant convolutions with multiple input signals that are sparse. This problem finds numerous applications in signal processing, computer vision, and inverse problems. However, it is challenging to learn the filter efficiently due to the bilinear structure of the observations with the respect to the unknown filter and inputs, leading to global ambiguities of identification. In this paper, we propose a novel approach based on nonconvex optimization over the sphere manifold by minimizing a smooth surrogate of the sparsity-promoting loss function. It is demonstrated that the manifold gradient descent with random initializations will provably recover the filter, up to scaling and shift ambiguity, as soon as the number of observations is sufficiently large under an appropriate random data model. Numerical experiments are provided to illustrate the performance of the proposed method with comparisons to existing methods.

show abstract

“…They assume that there is a single measurement of the form y " a f x, that x is sparse and the length of the kernel a is much shorter than y and x. In particular, Zhang et al [ZKW18] formulated the problem as an ℓ 4 -maximization problem over the sphere similar to [LB18], proving on a local region that every local minimizer is near a truncated signed shift of a. Kuo et al [KLZW19] studied a dropped quadratic simplification of bilinear Lasso objective [LQK`19], which provably obtains exact recovery for an incoherent kernel a and sparse x. However, as the kernel and measurements are not the same length in SaS, the SaS deconvolution is much harder than MCS-BD: the problem has spurious local minimizers such as shift-truncations, so that most of the results there can only show benign local geometry structure regardless of the choice of objectives.…”

Section: Short-and-sparse Deconvolutionmentioning

confidence: 99%

Exact and Efficient Multi-Channel Sparse Blind Deconvolution — A Nonconvex Approach

Xiao

Zhu

2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

View full text Add to dashboard Cite

We study the multi-channel sparse blind deconvolution (MCS-BD) problem, whose task is to simultaneously recover a kernel a and multiple sparse inputs txiu p i"1 from their circulant convolution yi " a f xi (i " 1,¨¨¨, p). We formulate the task as a nonconvex optimization problem over the sphere. Under mild statistical assumptions of the data, we prove that the vanilla Riemannian gradient descent (RGD) method, with random initializations, provably recovers both the kernel a and the signals txiu p i"1 up to a signed shift ambiguity. In comparison with state-of-the-art results, our work shows significant improvements in terms of sample complexity and computational efficiency. Our theoretical results are corroborated by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods on both synthetic and real datasets. 1 Here, (i) BGpθq and BRpθq denote Bernoulli-Gaussian and Bernoulli-Rademacher distribution, respectively; (ii) θ P r0, 1s is the Bernoulli parameter controlling the sparsity level of x i ; (iii) ε denotes the recovery precision of global solution a‹, i.e., min ℓ }a´s ℓ ra‹s} ď ε; (iv) r O and r Ω hides logpnq, θ and other factors. 2 Recently, similar loss has been considered for short and sparse deconvolution [ZKW18] and complete dictionary learning [ZYL`19]. 3 As the tail of BGpθq distribution is heavier than that of BRpθq, their sample complexity would be even worse if BGpθq model was considered. 4 We say x obeys a Bernoulli-Rademacher distribution when x " b d r where d denotes point-wise product, b follows i.i.d. Bernoulli distribution and r follows i.i.d. Rademacher distribution.Contributions of this paper. In this work, we introduce an efficient optimization method for solving MCS-BD. We consider a natural nonconvex formulation based on a smooth relaxation of ℓ 1 -loss. Under mild assumptions of the data, we prove the following result.With random initializations, a vanilla RGD efficiently finds an approximate solution, which can be refined by a subgradient method that converges exactly to the target solution in a linear rate.We summarize our main result in Table 1. By comparison 5 with [LB18], our approach demonstrates substantial improvements for solving MCS-BD in terms of both sample and time complexity. Moreover, our experimental results imply that our analysis is still far from tight -the phase transitions suggest that p ě Ωppoly logpnqq samples might be sufficient for exact recovery, which is favorable for applications (as real data in form of images can have millions of pixels, resulting in huge dimension n). Our analysis is inspired by recent results on orthogonal dictionary learning [GBW18, BJS18], but much of our theoretical analysis is tailored for MCS-BD with a few extra new ingredients. Our work is the first result provably showing that vanilla gradient descent type methods with random initialization solve MCS-BD efficiently. Moreover, our ideas could potentially lead to new algorithmic guarantees for other nonconvex problems such a...

show abstract

Geometry and Symmetry in Short-and-Sparse Deconvolution

Cited by 25 publications

References 46 publications

Convolutional Phase Retrieval via Gradient Descent

Convolutional Phase Retrieval via Gradient Descent

Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently

Exact and Efficient Multi-Channel Sparse Blind Deconvolution — A Nonconvex Approach

Contact Info

Product

Resources

About