This paper introduces a novel algorithm to approximate the matrix with minimum nuclear norm among all matrices obeying a set of convex constraints. This problem may be understood as the convex relaxation of a rank minimization problem, and arises in many important applications as in the task of recovering a large matrix from a small subset of its entries (the famous Netflix problem). Off-the-shelf algorithms such as interior point methods are not directly amenable to large problems of this kind with over a million unknown entries.This paper develops a simple first-order and easy-to-implement algorithm that is extremely efficient at addressing problems in which the optimal solution has low rank. The algorithm is iterative and produces a sequence of matrices {X k , Y k } and at each step, mainly performs a soft-thresholding operation on the singular values of the matrix Y k . There are two remarkable features making this attractive for low-rank matrix completion problems. The first is that the soft-thresholding operation is applied to a sparse matrix; the second is that the rank of the iterates {X k } is empirically nondecreasing. Both these facts allow the algorithm to make use of very minimal storage space and keep the computational cost of each iteration low. On the theoretical side, we provide a convergence analysis showing that the sequence of iterates converges. On the practical side, we provide numerical examples in which 1, 000×1, 000 matrices are recovered in less than a minute on a modest desktop computer. We also demonstrate that our approach is amenable to very large scale problems by recovering matrices of rank about 10 with nearly a billion unknowns from just about 0.4% of their sampled entries. Our methods are connected with the recent literature on linearized Bregman iterations for ℓ 1 minimization, and we develop a framework in which one can understand these algorithms in terms of well-known Lagrange multiplier algorithms.

No abstract

Split Bregman methods introduced in [47] have been demonstrated to be efficient tools to solve total variation (TV) norm minimization problems, which arise from partial differential equation based image restoration such as image denoising and magnetic resonance imaging (MRI) reconstruction from sparse samples. In this paper, we prove the convergence of the split Bregman iterations, where the number of inner iterations is fixed to be one. Furthermore, we show that these split Bregman iterations can be used to solve minimization problems arising from the analysis based approach for image restoration in the literature. We apply these split Bregman iterations to the analysis based image restoration approach whose analysis operator is derived from tight framelets constructed in [59]. This gives a set of new frame based image restoration algorithms that cover several topics in image restorations, such as image denoising, deblurring, inpainting and cartoontexture image decomposition. Several numerical simulation results are provided.

Discrete affine systems are obtained by applying dilations to a given shiftinvariant system. The complicated structure of the affine system is due, first and foremost, to the fact that it is not invariant under shifts. Affine frames carry the additional difficulty that they are``global'' in nature: it is the entire interaction between the various dilation levels that determines whether the system is a frame, and not the behaviour of the system within one dilation level. We completely unravel the structure of the affine system with the aid of two new notions: the affine product, and a quasi-affine system. This leads to a characterization of affine frames; the induced characterization of tight affine frames is in terms of exact orthogonality relations that the wavelets should satisfy on the Fourier domain. Several results, such as a general oversampling theorem follow from these characterizations. Most importantly, the affine product can be factored during a multiresolution analysis construction, and this leads to a complete characterization of all tight frames that can be constructed by such methods. Moreover, this characterization suggests very simple sufficient conditions for constructing tight frames from multiresolution. Of particular importance are the facts that the underlying scaling function does not need to satisfy any a priori conditions, and that the freedom offered by redundancy can be fully exploited in these constructions.

The variational techniques (e.g. the total variation based method) are well established and effective for image restoration, as well as many other applications, while the wavelet frame based approach is relatively new and came from a different school. This paper is designed to establish a connection between these two major approaches for image restoration. The main result of this paper shows that when spline wavelet frames of are used, a special model of a wavelet frame method, called the analysis based approach, can be viewed as a discrete approximation at a given resolution to variational methods. A convergence analysis as image resolution increases is given in terms of objective functionals and their approximate minimizers. This analysis goes beyond the establishment of the connections between these two approaches, since it leads to new understandings for both approaches. First, it provides geometric interpretations to the wavelet frame based approach as well as its solutions. On the other hand, for any given variational model, wavelet frame based approaches provide various and flexible discretizations which immediately lead to fast numerical algorithms for both wavelet frame based approaches and the corresponding variational model. Furthermore, the built-in multiresolution structure of wavelet frames can be utilized to adaptively choose proper differential operators in different regions of a given image according to the order of the singularity of the underlying solutions. This is important when multiple orders of differential operators are used in various models that generalize the total variation based method. These observations will enable us to design new methods according to the problems at hand, hence, lead to wider applications of both the variational and wavelet frame based approaches. Links of wavelet frame based approaches to some more general variational methods developed recently will also be discussed.