Extending the Scope of Wavelet Regression Methods by Coefficient-Dependent Thresholding

Abstract

ABSTRACT. Various aspects of the wavelet approach to nonparametric regression are considered, with the overall aim of extending the scope of wavelet techniques, to irregularlyspaced data, to regularly-spaced data sets of arbitrary size, to heteroscedastic and correlated data, and to data some of which may be downweighted or omitted as outliers.At the core of the methodology discussed is the following problem: if a sequence has a given covariance structure, what is the variance and covariance structure of its discrete wavelet transform? For sequences whose length is a power of 2, an algorithm for finding all the variances and within-level covariances in the wavelet table is developed and investigated in detail. In particular, it is shown that if the original sequence has band-limited covariance matrix, then the time required by the algorithm is linear in the length of the sequence.Up to now, most statistical work on wavelet methods presumes that the number of observations is a power of 2 and that the independent variable takes values on a regular grid. The variance-calculation algorithm allows data on any set of independent variable values to be treated, by first interpolating to a fine regular grid of suitable length, and then constructing a wavelet expansion of the gridded data. The gridded data will, in general, have a band-limited covariance matrix, and the algorithm therefore allows the elements of the wavelet transform to be thresholded individually using thresholds proportional to their standard deviation.Various thresholding methods are discussed and investigated. Exact risk formulae for the mean square error of the methodology for given design are derived and used, to avoid, as far as possible, the need for simulation in assessing performance. Both for regular and irregular data, good performance is obtained by noise-proportional thresholding, with thresholds somewhat smaller than the classical universal threshold.The general approach allows outliers in the data to be removed or downweighted, and aspects of such robust techniques are developed and demonstrated in an example. Another natural application is to data that are themselves correlated, where the covariance of the wavelet coefficients is not due to an initial grid transform but is an intrinsic feature of the data. The use of the method in these circumstances is demonstrated by an application to data synthesized in the study of ion channel gating. The basic approach of the paper has many other potential applications, and some of these are discussed briefly.Arne Kovac is wissenschaftlicher Mitarbeiter,

Wavelet kernel penalized estimation for non-equispaced design regression

“…The problem of estimating a regression function in the framework of wavelet thresholding in the irregular design case has been now addressed by many authors: see for instance the interpolation methods of Hall and Turlach (1997), Kovac and Silverman (2000), Nason (2002), the transformation method of Cai and Brown (1998), the isometric method of Sardy et al (1999), or the specific construction of ad hoc wavelets by lifting of Delouille et al (2001). Other ideas based on orthogonal expansions have also been used (see e.g.…”

“…After the transformation, the new vector is multivariate normal with mean R f and covariance that is assumed to have a finite bandwidth so that the computational complexity of their algorithm is of order n. For the KS procedure a term-by-term estimator with soft-thresholding and Stein's unbiased risk estimation policy (Sure Shrink) was considered as it is implemented in the R-package Wavethresh3 (Nason 1998). It is detailed in Kovac and Silverman (2000). Additionally, for both estimators (ours and KS's) the lowest level of detail coefficients (primary resolution) we have used was the value from Kovac and Silverman log 2 (log(n)).…”

“…Kohler (2003)) but are quite heavy (and therefore quite slow) from a computational point of view. Our estimate will be compared with the Kovac and Silverman (2000) wavelet term-by-term thresholding procedure (KS for short) for denoising functions sampled at nonequispaced design points. We believe that the KS procedure which is implemented in R is relatively easy to use and quite efficient that is why we have used it as a reference method for direct comparisons.…”

“…We believe that the KS procedure which is implemented in R is relatively easy to use and quite efficient that is why we have used it as a reference method for direct comparisons. Recall that the Kovac and Silverman (2000) procedure relies upon a linear interpolation transformation R to the observed data vector y that maps it to a new vector of size 2 J (2 J −1 < n ≤ 2 J ), corresponding to a new design with equispaced points. After the transformation, the new vector is multivariate normal with mean R f and covariance that is assumed to have a finite bandwidth so that the computational complexity of their algorithm is of order n. For the KS procedure a term-by-term estimator with soft-thresholding and Stein's unbiased risk estimation policy (Sure Shrink) was considered as it is implemented in the R-package Wavethresh3 (Nason 1998).…”

Mean Integrated Squared Error of Nonlinear Wavelet-based Estimators with Long Memory Data

“…errors or a stationary process with short-range dependence such as classic ARMA processes (see, e.g., Hart, 1991;Tran et al 1996;Truong and Patil, 2001); or a stationary Gaussian sequence with long-range dependence (see, e.g., Csörgö and Mielniczuk, 1995;Wang, 1996;Johnstone and Silverman, 1997;Johnstone, 1999); or a correlated and heteroscedastic noise sequence (Kovac and Silverman, 2000); or a correlated and nonstationary noise sequence (von Sachs and Macgibbon, 2000), just to mention a few. Regression models with long memory data are more appropriate for various phenomena in many fields which include agronomy, astronomy, economics, environmental sciences, geosciences, hydrology and signal and image processing.…”