On the importance of combining wavelet-based nonlinear approximation with coding strategies

Abstract

This paper provides a mathematical analysis of transform compression in its relationship to linear and nonlinear approximation theory. Contrasting linear and nonlinear approximation spaces, we show that there are interesting classes of functions/random processes which are much more compactly represented by wavelet-based nonlinear approximation. These classes include locally smooth signals that have singularities, and provide a model for many signals encountered in practice, in particular for images. However, we also show that nonlinear approximation results do not always translate to efficient compression strategies in a rate-distortion sense. Based on this observation, we construct compression techniques and formulate the family of functions/stochastic processes for which they provide efficient descriptions in a rate-distortion sense. We show that this family invariably leads to Besov spaces, yielding a natural relationship among Besov smoothness, linear/nonlinear approximation order, and compression performance in a rate-distortion sense. The designed compression techniques show similarities to modern high-performance transform codecs, allowing us to establish relevant rate-distortion estimates and identify performance limits.

Sparse orthonormal transforms for image compression

“…Overall distortion for both cases is due to the errors introduced by the approximation and quantization processes. Under mild assumptions LA bit-rate can be approximated to be linear with n. This observation is not true for N A in general but, interestingly, for nontrivial models of multimedia signals and by using associated optimal or near-optimal basis, one can show that N A bit-rate can also be approximated to be linear with n [12], [17], [27], [48]. With these results and under associated conditions, compression distortionrate performance can be approximated to asymptotically track n-term approximation performance for both cases 2 .…”

“…In essence significantly better performance than that of LA with the KLT can be had by switching to N A and using the N A-optimal basis [8], [12], [33], [34], [46][47][48]. The substantial recent interest in N A-optimal designs can be tied to these results.…”

“…Early success of these techniques over images led researchers to analyze and quantify the classes of signals over which wavelet compression is optimal or near-optimal [12], [17]. Surprisingly, one of the ramifications of this analysis has been the clear suboptimality of the wavelet transform over realistic models of images, which are typically formulated as piecewise smooth processes with discontinuities along curves.…”

“…As compression and signal representation has shifted from PCA and linear approximation (LA -approximation using the first n transform coefficients) to nonlinear approximation (N A -approximation using the best n transform coefficients), many researchers have pointed to the optimality of nonlinear approximation using the right basis [12], [17], [27]. Finding the right basis however has proven to be a very difficult task in itself as model-based work is susceptible to model failures and generally-formulated work to important nuances of compression.…”

Critically sampled composite wavelets

“…Further details on this implementation are described in [14] Note that, in this decomposition, j 0 is constant so that an image coding scheme can take advantage of the correlation between levels in the DWT. In particular, one can benefit from tree-based coding schemes to improve this decay rate [15]. This is in contrast to similar schemes which do not fix the number of angular divisions in order to try and maintain a parabolic scaling law of width ∝ height 2 .…”