Optimality of Operator-Like Wavelets for Representing Sparse AR(1) Processes

Abstract

Abstract-The discrete cosine transform (DCT) is known to be asymptotically equivalent to the Karhunen-Loève transform (KLT) of Gaussian first-order auto-regressive (AR(1)) processes. Since being uncorrelated under the Gaussian hypothesis is synonymous with independence, it also yields an independent-component analysis (ICA) of such signals. In this paper, we present a constructive non-Gaussian generalization of this result: the characterization of the optimal orthogonal transform (ICA) for the family of symmetric--stable AR(1) processes. The degree of sparsity of these processes is controlled by the stability parameter with the only non-sparse member of the family being the classical Gaussian AR(1) process with . Specifically, we prove that, for , a fixed family of operator-like wavelet bases systematically outperforms the DCT in terms of compression and denoising ability. The effect is quantified with the help of two performance criteria (one based on the Kullback-Leibler divergence, and the other on Stein's formula for the minimum estimation error) that can also be viewed as statistical measures of independence. Finally, we observe that, for the sparser kind of processes with , the operator-like wavelet basis, as dictated by linear system theory, is undistinguishable from the ICA solution obtained through numerical optimization. Our framework offers a unified view that encompasses sinusoidal transforms such as the DCT and a family of orthogonal Haar-like wavelets that is linked analytically to the underlying signal model. Index Terms-Operator-like wavelets, independent-component analysis, auto-regressive processes, stable distributions.

MMSE denoising of sparse and non-Gaussian AR(1) processes

“…As it is shown in [12], if we sample the process s(t) ideally and uniformly with period T , the obtained discrete process is also a discrete AR(1) process. Precisely, if we define…”

“…According to [12], the wavelet-like basis that best approximates an independent component analysis (ICA) for AR (1) processes is the operator-like wavelet that matches the operator D + κI. We thus use the operator-like wavelet basis and construct a wavelet frame that is tight and shift-invariant.…”

“…The leading idea of our approach is to replace finite differences by linear-prediction residuals and the Haar wavelets of [8] by an alternative operator-like basis that is matched to the processes [11]. The justification for this latter choice is that these operator-like wavelets have been shown recently to provide an independent component analysis for the sub-class of α-stable AR(1) processes [12]. A remarkable outcome is that the performance of our new wavelet-based denoiser matches that of our generalized message-passing-based algorithm which is guaranteed to be MMSE-optimal for the general AR(1) processes (since the factor graph has no loops).…”

Interpretation of continuous-time autoregressive processes as random exponential splines

“…Among the family of CAR models, the Gaussian ones are by far the most popular and the easiest to specify [1]. The family can also be extended to allow for non-Gaussian sparse models [2], [3], [4].…”

Maximally Localized Radial Profiles for Tight Steerable Wavelet Frames

“…It is observed that more-localized wavelets result in fewer oscillations and are less subject to truncation artifacts. Moreover, it has been theoretically shown that wavelets with better localization are more efficient for decoupling and sparsifying signals [21]. It is worth mentioning that the Simoncelli wavelet performs well in a wide range of applications and is shown to be the most-localized wavelet in a specific sense [22].…”