Short-and-Sparse Deconvolution -- A Geometric Approach

Lau, Yenson; Qu, Qing; Kuo, Han-Wen; Zhou, Pengcheng; Zhang, Yuqian; Wright, John

doi:10.48550/arxiv.1908.10959

Cited by 6 publications

(12 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, these landscapes do not exhibit spurious local minimizers or flat saddles and can be easily optimized via gradient-based methods [8]. Examples include phase retrieval [41], low-rank matrix recovery [9,8], dictionary learning [40,35,21] and blind deconvolution [20].…”

Section: Related Workmentioning

confidence: 99%

An Unconstrained Layer-Peeled Perspective on Neural Collapse

Ji¹,

Lu²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural collapse is a highly symmetric geometric pattern of neural networks that emerges during the terminal phase of training, with profound implications on the generalization performance and robustness of the trained networks. To understand how the last-layer features and classifiers exhibit this recently discovered implicit bias, in this paper, we introduce a surrogate model called the unconstrained layer-peeled model (ULPM). We prove that gradient flow on this model converges to critical points of a minimum-norm separation problem exhibiting neural collapse in its global minimizer. Moreover, we show that the ULPM with the cross-entropy loss has a benign global landscape for its loss function, which allows us to prove that all the critical points are strict saddle points except the global minimizers that exhibit the neural collapse phenomenon. Empirically, we show that our results also hold during the training of neural networks in real-world tasks when explicit regularization or weight decay is not used.

show abstract

Section: Related Workmentioning

confidence: 99%

An Unconstrained Layer-Peeled Perspective on Neural Collapse

Ji¹,

Lu²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, short-and-sparse deconvolution (SaSD) receives much attention [19,20,21,22]. It assumes that the convolution is from a short kernel and a sparse signal.…”

Section: Introductionmentioning

confidence: 99%

“…It assumes that the convolution is from a short kernel and a sparse signal. In stead of matrix lifting, a popular approach [19,20,21,22] is to cast blind deconvolution as a bilinear Lasso problem and solve it by alternating minimization, where the two unknown sequences are updated alternatively by fixing the other. The bilinear Lasso formulation is non-convex.…”

Section: Introductionmentioning

confidence: 99%

“…It is well known that the solutions to blind deconvolution subject to scaling and shifting ambiguities. That is, if y This paper focuses on the problem of short-and-sparse blind deconvolution [20,21]. Consider a convolution of two finite-length sequences: without loss of generality assume that x[n] is sparse and h[n] is short.…”

Section: Introductionmentioning

confidence: 99%

“…To partially address the scaling ambiguity, an extra constraint h 2 = 1 can be added. These yield the well-known bilinear Lasso form [21] min…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Short-and-Sparse Deconvolution Via Rank-One Constrained Optimization (ROCO)

Cheng¹,

Dai²

2021

Preprint

View full text Add to dashboard Cite

Short-and-sparse deconvolution (SaSD) aims to recover a short kernel and a long and sparse signal from their convolution. In the literature, formulations of blind deconvolution is either a convex programming via a matrix lifting of convolution, or a bilinear Lasso. Optimization solvers are typically based on bilinear factorizations. In this paper, we formulate SaSD as a non-convex optimization with a rank-one matrix constraint, hence referred to as Rank-One Constrained Optimization (ROCO). The solver is based on alternating direction method of multipliers (ADMM). It operates on the full rank-one matrix rather than bilinear factorizations. Closed form updates are derived for the efficiency of ADMM. Simulations include both synthetic data and real images. Results show substantial improvements in recovery accuracy (at least 19dB in PSNR for real images) and comparable runtime compared with benchmark algorithms based on bilinear factorization.

show abstract

Nonuniform electron distributions in a solenoidal ioniser

Bergin

Martens

Dastoor

2023

J. Phys. D: Appl. Phys.

View full text Add to dashboard Cite

Solenoidal ionisers are a new class of highly efficient helium detectors that are increasingly important for high resolution atom scattering, molecular scattering and scanning helium microscopy. They operate via electron ionisation, where the electrons are trapped by the magnetic field of a solenoid and additional electrostatic potentials. Their ionisation efficiency scales with the electron population they contain, motivating large devices with high emission currents. However, these detectors typically become unstable at high electron densities, constraining their performance improvement. Through imaging the electronpopulation at the exit of the ioniser, we demonstrate that these instabilities arise from nonuniformities in the electron distribution. Considering the ioniser as a non-neutral plasma leads to the proposal of the formation of a virtual cathode and a plasma instability as the origins of the non-uniformity.

show abstract

Short-and-Sparse Deconvolution -- A Geometric Approach

Cited by 6 publications

References 35 publications

An Unconstrained Layer-Peeled Perspective on Neural Collapse

An Unconstrained Layer-Peeled Perspective on Neural Collapse

Short-and-Sparse Deconvolution Via Rank-One Constrained Optimization (ROCO)

Nonuniform electron distributions in a solenoidal ioniser

Contact Info

Product

Resources

About