Online Sparsifying Transform Learning—Part I: Algorithms

Abstract

Techniques exploiting the sparsity of signals in a transform domain or dictionary have been popular in signal processing. Adaptive synthesis dictionaries have been shown to be useful in applications such as signal denoising, and medical image reconstruction. More recently, the learning of sparsifying transforms for data has received interest. The sparsifying transform model allows for cheap and exact computations. In this paper, we develop a methodology for online learning of square sparsifying transforms. Such online learning can be particularly useful when dealing with big data, and for signal processing applications such as real-time sparse representation and denoising. The proposed transform learning algorithms are shown to have a much lower computational cost than online synthesis dictionary learning. In practice, the sequential learning of a sparsifying transform typically converges faster than batch mode transform learning. Preliminary experiments show the usefulness of the proposed schemes for sparse representation, and denoising.

Online Sparsifying Transform Learning—Part II: Convergence Analysis

“…Adaptive trans- forms also provide competitive, or useful signal reconstruction quality in applications (cf. [2], [3], and [1] and the references therein). Prior work on transform learning focused on batch learning [2], [4], where the sparsifying transform is learnt using all the training data simultaneously.…”

“…Prior work on transform learning focused on batch learning [2], [4], where the sparsifying transform is learnt using all the training data simultaneously. In Part I [1] of this work as well as here, the focus is instead on the online learning of sparsifying transforms. Various formulations and algorithms for online sparsifying transform learning have been proposed in Part I [1].…”

“…In Part I [1] of this work as well as here, the focus is instead on the online learning of sparsifying transforms. Various formulations and algorithms for online sparsifying transform learning have been proposed in Part I [1]. In this paper, we focus exclusively on the convergence properties of the algorithms in [1].…”

“…T HIS paper, the theoretical counterpart to the work in Part I [1] on data-driven online learning of sparsifying transforms, provides a convergence analysis of the algorithms proposed in [1]. We start with a brief review of the background and motivation for the work.…”

“…We start with a brief review of the background and motivation for the work. More detailed discussions and the relevant references can be found in Part I [1].…”