A Greedy Approach to $\ell_{0,\infty}$-Based Convolutional Sparse Coding

Abstract

Sparse coding techniques for image processing traditionally rely on a processing of small overlapping patches separately followed by averaging. This has the disadvantage that the reconstructed image no longer obeys the sparsity prior used in the processing. For this purpose convolutional sparse coding has been introduced, where a shift-invariant dictionary is used and the sparsity of the recovered image is maintained. Most such strategies target the 0 "norm" or the 1 norm of the whole image, which may create an imbalanced sparsity across various regions in the image. In order to face this challenge, the 0,∞ "norm" has been proposed as an alternative that "operates locally while thinking globally". The approaches taken for tackling the non-convexity of these optimization problems have been either using a convex relaxation or local pursuit algorithms. In this paper, we present an efficient greedy method for sparse coding and dictionary learning, which is specifically tailored to 0,∞, and is based on matching pursuit. We demonstrate the usage of our approach in salt-and-pepper noise removal and image inpainting.1. Introduction. Sparse coding can be described as solving the following minimization problem, known as the P 0 problem [9]:where α ∈ R p is a sparse representation of a signal x ∈ R N in the dictionary D ∈ R N ×p . The columns of D, which are referred to as atoms, are a full and overcomplete set, and we will assume without loss of generality that they are normalized to unit 2 norm. The 0 "norm" 1 returns the number of nonzero elements in a vector, also called the sparsity.When modeling natural images, we allow some deviation from the model rather than seeking a perfect reconstruction:An alternative form, in which the sparsity k is known, is:As the P 0 and P k 0 problems are NP-hard, several approximation techniques have been proposed. Matching Pursuit (MP) [28] is a greedy algorithm that in each iteration updates * Accepted for publication in SIAM Journal on Imaging Sciences (SIIMS).

Rethinking the CSC Model for Natural Images

“…In order to answer these questions let us refer to an unsuccessful use of the CSC: image denoising. Applying the CSC model directly on the noisy image leads to disappointing results, falling far behind PA [25,35,36]. We emphasize that the same is true for other applications where noise cannot be neglected, such as deblurring and other inverse problems.…”

“…Instead of operating on patches, it suggests a global dictionary constrained by a specific structure -a concatenation of banded circulant matrices 2 , limiting the degrees of freedom introduced by the general sparsity-based model. Various algorithms have been suggested to efficiently handle the global pursuit [23][24][25][26][27]. These methods have been augmented by efficient dictionary learning algorithms [26,[28][29][30].…”

Improved Image Generation via Sparse Modeling

“…Thus, forcing Γ 0,∞ ≤ λ restricts the number of local atoms to λ. While this constraint is theoretically justified in the context of CSC, projection onto it is known to be challenging [32]. To approximate it in a computationally plausible manner, we use a "needle"based sparsity measure [30,47].…”

Convolutional Sparse Coding based Channel Estimation for OTFS-SCMA in Uplink

“…to extract a u × 1 patch from y. The CSC applications discussed in [33] and [34] establish that the sparse signal recovery algorithms based on convex relaxation (e.g. gradient descent) and greedy approach (e.g.…”

“…If the perfect CE is possible, then the system's SE is equal to that of an uncoded OTFS-SCMA system. Based on (34), for an overloading factor of 150% and |A| = 4, we have SE eff ≈ 3 bits/s/Hz. Observe from Fig.…”