Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization

Elad, Michael; Matalon, Boaz; Zibulevsky, Michael

doi:10.1016/j.acha.2007.02.002

Cited by 165 publications

(151 citation statements)

References 23 publications

Supporting

Mentioning

151

Contrasting

Order By: Relevance

“…Assuming that the multiplication by the dictionary (and its adjoint) has a fast O(n log n) algorithm, the overall process is very fast and effective. We should also mention that fast IST-like algorithms were recently proposed by several authors [5], [22], [57], [3].…”

Section: Inputmentioning

confidence: 99%

On the Role of Sparse and Redundant Representations in Image Processing

2010

Self Cite

View full text Add to dashboard Cite

Abstract-Much of the progress made in image processing in the past decades can be attributed to better modeling of image content, and a wise deployment of these models in relevant applications. This path of models spans from the simple ℓ2-norm smoothness, through robust, thus edge preserving, measures of smoothness (e.g. total variation), and till the very recent models that employ sparse and redundant representations. In this paper, we review the role of this recent model in image processing, its rationale, and models related to it. As it turns out, the field of image processing is one of the main beneficiaries from the recent progress made in the theory and practice of sparse and redundant representations. We discuss ways to employ these tools for various image processing tasks, and present several applications in which state-of-the-art results are obtained.

show abstract

Section: Inputmentioning

confidence: 99%

On the Role of Sparse and Redundant Representations in Image Processing

2010

Self Cite

View full text Add to dashboard Cite

show abstract

“…This strategy can be generalized with the sequential subspace optimization (SESOP) [32], [33] for further accelerations, e.g. PCD-SESOP [11], [33] and PCD-SESOP-MM [34]. Variable splitting technique [35], [36] (also known as separable surrogate functionals (SSF) [33]) provides yet another powerful tool to minimize functions that consist of summation of two terms which are of different nature, e.g.…”

Section: B Approaches To Solve the Problemmentioning

confidence: 99%

“…The strategy of these algorithms is to use smartly-chosen descent directions, that are combinations of the previous two iterates' results. This strategy can be generalized with the sequential subspace optimization (SESOP) [32], [33] for further accelerations, e.g. PCD-SESOP [11], [33] and PCD-SESOP-MM [34].…”

Section: B Approaches To Solve the Problemmentioning

confidence: 99%

An Iterative Linear Expansion of Thresholds for $\ell_{1}$-Based Image Restoration

Pan

Blu

2013

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Abstract-This paper proposes a novel algorithmic framework to solve image restoration problems under sparsity assumptions. As usual, the reconstructed image is the minimum of an objective functional that consists of a data fidelity term and an 1 regularization. However, instead of estimating the reconstructed image that minimizes the objective functional directly, we focus on the restoration process that maps the degraded measurements to the reconstruction. Our idea amounts to parameterizing the process as a linear combination of few elementary thresholding functions (LET) and solve for the linear weighting coefficients by minimizing the objective functional. It is then possible to update the thresholding functions and to iterate this process (i-LET). The key advantage of such a linear parametrization is that the problem size reduces dramatically-each time we only need to solve an optimization problem over the dimension of the linear coefficients (typically less than 10) instead of the whole image dimension. With the elementary thresholding functions satisfying certain constraints, global convergence of the iterated LET algorithm is guaranteed. Experiments on several test images over a wide range of noise levels and different types of convolution kernels clearly indicate that the proposed framework usually outperform state-of-the-art algorithms in terms of both CPU time and number of iterations.Index Terms-Image restoration, sparsity, majorization minimization (MM), iterative reweighted least square (IRLS), thresholding, linear expansion of thresholds (LET).

show abstract

“…However, the classical proximal gradient algorithm has been regarded as a slower until the exciting progress in recent years. The "accelerated" approaches mainly base on an extrapolation step which relies on not only the current point, but also two or more previous computed iterations (e.g., [14,15]). Other dramatic algorithms can refer to Nesterov [16], Beck & Teboulle [6], Tseng [17], O'Donoghue & Candès [18], and references therein.…”

Section: Introductionmentioning

confidence: 99%

Two-Step Proximal Gradient Algorithm for Low-Rank Matrix Completion

Wang

Cao

Zhang

2016

Stat., optim. inf. comput.

View full text Add to dashboard Cite

In this paper, we propose a two-step proximal gradient algorithm to solve nuclear norm regularized least squares for the purpose of recovering low-rank data matrix from sampling of its entries. Each iteration generated by the proposed algorithm is a combination of the latest three points, namely, the previous point, the current iterate, and its proximal gradient point. This algorithm preserves the computational simplicity of classical proximal gradient algorithm where a singular value decomposition in proximal operator is involved. Global convergence is followed directly in the literature. Numerical results are reported to show the efficiency of the algorithm.

show abstract

Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization

Cited by 165 publications

References 23 publications

On the Role of Sparse and Redundant Representations in Image Processing

On the Role of Sparse and Redundant Representations in Image Processing

An Iterative Linear Expansion of Thresholds for $\ell_{1}$-Based Image Restoration

Two-Step Proximal Gradient Algorithm for Low-Rank Matrix Completion

Contact Info

Product

Resources

About