Sparse stochastic processes are continuous-domain processes that admit a parsimonious representation in some matched wavelet-like basis. Such models are relevant for image compression, compressed sensing, and, more generally, for the derivation of statistical algorithms for solving ill-posed inverse problems. This book introduces an extended family of sparse processes that are specified by a generic (non-Gaussian) innovation model or, equivalently, as solutions of linear stochastic differential equations driven by white Lévy noise. It presents the mathematical tools for their characterization. The two leading threads that underly the exposition are-the statistical property of infinite divisibility, which induces two distinct types of behavior-Gaussian vs. sparse-at the exclusion of any other;-the structural link between linear stochastic processes and spline functions which is exploited to simplify the mathematics. The last chapter is devoted to the use of these models for the derivation of algorithms that recover sparse signals. This leads to a Bayesian reinterpretation of popular sparsitypromoting processing schemes-such as total-variation denoising, LASSO, and wavelet shrinkage-as MAP estimators for specific types of Lévy processes. The book, which is mostly self-contained, is targeted to an audience of graduate students and researchers with an interest in signal/image processing, compressed sensing, approximation theory, machine learning, or statistics.

We introduce a general distributional framework that results in a unifying description and characterization of a rich variety of continuous-time stochastic processes. The cornerstone of our approach is an innovation model that is driven by some generalized white noise process, which may be Gaussian or not (e.g., Laplace, impulsive Poisson or alpha stable). This allows for a conceptual decoupling between the correlation properties of the process, which are imposed by the whitening operator L, and its sparsity pattern which is determined by the type of noise excitation. The latter is fully specified by a Lévy measure. We show that the range of admissible innovation behavior varies between the purely Gaussian and super-sparse extremes. We prove that the corresponding generalized stochastic processes are well-defined mathematically provided that the (adjoint) inverse of the whitening operator satisfies some Lp bound for p ≥ 1. We present a novel operator-based method that yields an explicit characterization of all Lévy-driven processes that are solutions of constant-coefficient stochastic differential equations (SDE). When the underlying system is stable, we recover the family of stationary CARMA processes, including the Gaussian ones. The approach remains valid when the system is unstable and leads to the identification of potentially useful generalizations of the Lévy processes, which are sparse and non-stationary. Finally, we show that these processes admit a sparse representation in some matched wavelet domain and provide a full characterization of their transform-domain statistics.

Abstract-This paper is devoted to the characterization of an extended family of continuous-time autoregressive moving average (CARMA) processes that are solutions of stochastic differential equations driven by white Lévy innovations. These are completely specified by: 1) a set of poles and zeros that fixes their correlation structure and 2) a canonical infinitely divisible probability distribution that controls their degree of sparsity (with the Gaussian model corresponding to the least sparse scenario). The generalized CARMA processes are either stationary or nonstationary, depending on the location of the poles in the complex plane. The most basic nonstationary representatives (with a single pole at the origin) are the Lévy processes, which are the non-Gaussian counterparts of Brownian motion. We focus on the general analog-to-discrete conversion problem and introduce a novel spline-based formalism that greatly simplifies the derivation of the correlation properties and joint probability distributions of the discrete versions of these processes. We also rely on the concept of generalized increment process, which suppresses all long range dependencies, to specify an equivalent discrete-domain innovation model. A crucial ingredient is the existence of a minimally supported function associated with the whitening operator L; this B-spline, which is fundamental to our formulation, appears in most of our formulas, both at the level of the correlation and the characteristic function. We make use of these discrete-domain results to numerically generate illustrative examples of sparse signals that are consistent with the continuousdomain model.

Abstract-We introduce an extended family of continuous-domain stochastic models for sparse, piecewise-smooth signals. These are specified as solutions of stochastic differential equations, or, equivalently, in terms of a suitable innovation model; the latter is analogous conceptually to the classical interpretation of a Gaussian stationary process as filtered white noise. The two specific features of our approach are 1) signal generation is driven by a random stream of Dirac impulses (Poisson noise) instead of Gaussian white noise, and 2) the class of admissible whitening operators is considerably larger than what is allowed in the conventional theory of stationary processes. We provide a complete characterization of these finite-rate-of-innovation signals within Gelfand's framework of generalized stochastic processes. We then focus on the class of scale-invariant whitening operators which correspond to unstable systems. We show that these can be solved by introducing proper boundary conditions, which leads to the specification of random, spline-type signals that are piecewise-smooth. These processes are the Poisson counterpart of fractional Brownian motion; they are nonstationary and have the same -type spectral signature. We prove that the generalized Poisson processes have a sparse representation in a wavelet-like basis subject to some mild matching condition. We also present a limit example of sparse process that yields a MAP signal estimator that is equivalent to the popular TV-denoising algorithm.

Abstract-In this contribution, we study the notion of affine invariance (specifically, invariance to the shifting, scaling, and rotation of the coordinate system) as a starting point for the development of mathematical tools and approaches useful in the characterization and analysis of multivariate fractional Brownian motion (fBm) fields. In particular, using a rigorous and powerful distribution theoretic formulation, we extend previous results of Blu and Unser (2006) to the multivariate case, showing that polyharmonic splines and fBm processes can be seen as the (deterministic vs stochastic) solutions to an identical fractional partial differential equation that involves a fractional Laplacian operator. We then show that wavelets derived from polyharmonic splines have a behavior similar to the fractional Laplacian, which also turns out to be the whitening operator for fBm fields. This fact allows us to study the probabilistic properties of the wavelet transform coefficients of fBm-like processes, leading for instance to ways of estimating the Hurst exponent of a multiparameter process from its wavelet transform coefficients. We provide theoretical and experimental verification of these results. To complement the toolbox available for multiresolution processing of stochastic fractals, we also introduce an extended family of multidimensional multiresolution spaces for a large class of (separable and nonseparable) lattices of arbitrary dimensionality.