This paper shows that error bounds can be used as effective tools for deriving complexity results for first-order descent methods in convex minimization. In a first stage, this objective led us to revisit the interplay between error bounds and the Kurdyka-Lojasiewicz (KL) inequality. One can show the equivalence between the two concepts for convex functions having a moderately flat profile near the set of minimizers (as those of functions with Hölderian growth). A counterexample shows that the equivalence is no longer true for extremely flat functions. This fact reveals the relevance of an approach based on KL inequality. In a second stage, we show how KL inequalities can in turn be employed to compute new complexity bounds for a wealth of descent methods for convex problems. Our approach is completely original and makes use of a one-dimensional worst-case proximal sequence in the spirit of the famous majorant method of Kantorovich. Our result applies to a very simple abstract scheme that covers a wide class of descent methods. As a byproduct of our study, we also provide new results for the globalization of KL inequalities in the convex framework.Our main results inaugurate a simple methodology: derive an error bound, compute the desingularizing function whenever possible, identify essential constants in the descent method and finally compute the complexity using the one-dimensional worst case proximal sequence. Our method is illustrated through projection methods for feasibility problems, and through the famous iterative shrinkage thresholding algorithm (ISTA), for which we show that the complexity bound is of the form O(q k ) where the constituents of the bound only depend on error bound constants obtained for an arbitrary least squares objective with ℓ 1 regularization.

No abstract

The multilayer perceptron, when trained as a classifier using backpropagation, is shown to approximate the Bayes optimal discriminant function. The result is demonstrated for both the two-class problem and multiple classes. It is shown that the outputs of the multilayer perceptron approximate the a posteriori probability functions of the classes being trained. The proof applies to any number of layers and any type of unit activation function, linear or nonlinear.

Abstract-The pyramid algorithm for computing single wavelet transform coefficients is well known. The pyramid algorithm can be implemented by using tree-structured multirate filter banks. In this paper, we propose a general algorithm to compute multiwavelet transform coeficients by adding proper premultirate filter banks before the vector filter banks that generate multiwavelets. The proposed algorithm can be thought of as a discrete vector-valued wavelet transform for certain discretetime vector-valued signals. The proposed algorithm can be also thought of as a discrete multiwavelet transform for discrete-time signals. We then present some numerical experiments to illustrate the performance of the algorithm, which indicates that the energy compaction for discrete multiwavelet transforms may be better than the one for conventional discrete wavelet transforms.

This paper studies a problem of image restoration that observed images are contaminated by Gaussian and impulse noise. Existing methods for this problem in the literature are based on minimizing an objective functional having the l(1) fidelity term and the Mumford-Shah regularizer. We present an algorithm on this problem by minimizing a new objective functional. The proposed functional has a content-dependent fidelity term which assimilates the strength of fidelity terms measured by the l(1) and l(2) norms. The regularizer in the functional is formed by the l(1) norm of tight framelet coefficients of the underlying image. The selected tight framelet filters are able to extract geometric features of images. We then propose an iterative framelet-based approximation/sparsity deblurring algorithm (IFASDA) for the proposed functional. Parameters in IFASDA are adaptively varying at each iteration and are determined automatically. In this sense, IFASDA is a parameter-free algorithm. This advantage makes the algorithm more attractive and practical. The effectiveness of IFASDA is experimentally illustrated on problems of image deblurring with Gaussian and impulse noise. Improvements in both PSNR and visual quality of IFASDA over a typical existing method are demonstrated. In addition, Fast_IFASDA, an accelerated algorithm of IFASDA, is also developed.