Spline Smoothing in Regression Models and Asymptotic Efficiency in $L_2$

Nussbaum, Michael

doi:10.1214/aos/1176349651

Cited by 125 publications

(87 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cognitive value of this model had already been realized by Ibragimov and Khasminski (1977). These risk bounds have been established since then in a variety of other problems, e. g. density, nonparametric regression, spectral density, see Efroimovich and Pinsker (1982), Golubev (1984), Nussbaum (1985), and they have also been substantially extended conceptually (Korostelev (1993), Donoho, Johnstone, Kerkyacharian, Picard (1995)). The theory is now at a stage where the approximation of the various particular curve estimation problems by the white noise model could be made formal.…”

Section: Introduction and Main Resultsmentioning

confidence: 89%

Asymptotic equivalence of density estimation and Gaussian white noise

Nussbaum¹

1996

Ann. Statist.

Self Cite

215

221

View full text Add to dashboard Cite

Signal recovery in Gaussian white noise with variance tending to zero has served for some time as a representative model for nonparametric curve estimation, having all the essential traits in a pure form. The equivalence has mostly been stated informally, but an approximation in the sense of Le Cam's deficiency distance ∆ would make it precise. The models are then asymptotically equivalent for all purposes of statistical decision with bounded loss. In nonparametrics, a first result of this kind has recently been established for Gaussian regression (Brown and Low, 1993). We consider the analogous problem for the experiment given by n i. i. d. observations having density f on the unit interval. Our basic result concerns the parameter space of densities which are in a Hölder ball with exponent α > 1 2 and which are uniformly bounded away from zero. We show that an i. i. d. sample of size n with density f is globally asymptotically equivalent to a white noise experiment with drift f 1/2 and variance 1 4 n −1 . This represents a nonparametric analog of Le Cam's heteroscedastic Gaussian approximation in the finite dimensional case. The proof utilizes empirical process techniques related to the Hungarian construction. White noise models on f and log f are also considered, allowing for various "automatic" asymptotic risk bounds in the i. i. d. model from white noise. As first applications we discuss exact constants for L 2 and Hellinger loss.

show abstract

Section: Introduction and Main Resultsmentioning

confidence: 89%

Asymptotic equivalence of density estimation and Gaussian white noise

Nussbaum¹

1996

Ann. Statist.

Self Cite

215

221

View full text Add to dashboard Cite

show abstract

“…If all that we care about is attaining the minimax bound for a single specic ball F C , a great deal is known. For example, over certain L 2 Sobolev balls, special spline smoothers, with appropriate smoothness penalty terms chosen based on F C are asymptotically minimax [36,35]; over certain H older balls, Kernel methods with appropriate bandwidth, chosen with knowledge of F C are nearly minimax [40]; and it is known that no such linear methods can be nearly minimax over certain L p Sobolev balls, p < 2 [33,12]. However, nonlinear methods, such as the nonparametric method of maximum likelihood, are able to behave in a near-minimax way for L p Sobolev balls [32,19], but they require solution of a general n-dimensional nonlinear programming problem in general.…”

Section: Previous Adaptive Smoothing Workmentioning

confidence: 99%

De-noising by soft-thresholding

Donoho

1995

IEEE Trans. Inform. Theory

8,305

4,156

View full text Add to dashboard Cite

is at least as smooth as f, i n a n y of a wide variety of smoothness measures.[Adapt]: The estimator comes nearly as close in mean square to f as any measurable estimator can come, uniformly over balls in each o f t w o broad scales of smoothness classes. These two properties are unprecedented in several ways. Our proof of these results develops new facts about abstract statistical inference and its connection with an optimal recovery model.

show abstract

“…This approach has led to many theoretical developments which are of considerable interest: Stone (1982), Nussbaum (1985), Nemirovskii, Polyak, and Tsybakov (1985), ... But from a practical point of view, it has the diculty that it rarely corresponds with the usual situation where one is given data, but no knowledge of an a priori class F.…”

Section: Introductionmentioning

confidence: 99%

Adapting to Unknown Smoothness via Wavelet Shrinkage

Donoho

Johnstone

1995

Journal of the American Statistical Association

1,652

806

View full text Add to dashboard Cite

We attempt to recover a function of unknown smoothness from noisy, sampled data. We i n troduce a procedure, SureShrink, which suppresses noise by thresholding the empirical wavelet coecients. The thresholding is adaptive: a threshold level is assigned to each d y adic resolution level by the principle of minimizing the Stein Unbiased Estimate of Risk (Sure) for threshold estimates. The computational eort of the overall procedure is order N log(N) as a function of the sample size N.SureShrink is smoothness-adaptive: if the unknown function contains jumps, the reconstruction (essentially) does also; if the unknown function has a smooth piece, the reconstruction is (essentially) as smooth as the mother wavelet will allow. The procedure is in a sense optimally smoothness-adaptive: it is near-minimax simultaneously over a whole interval of the Besov scale; the size of this interval depends on the choice of mother wavelet. We know from a previous paper by the authors that traditional smoothing methods { kernels, splines, and orthogonal series estimates { even with optimal choices of the smoothing parameter, would be unable to perform in a near-minimax way o v er many spaces in the Besov scale.Examples of SureShrink are given: the advantages of the method are particularly evident when the underlying function has jump discontinuities on a smooth background.

show abstract

Spline Smoothing in Regression Models and Asymptotic Efficiency in $L_2$

Cited by 125 publications

References 0 publications

Asymptotic equivalence of density estimation and Gaussian white noise

Asymptotic equivalence of density estimation and Gaussian white noise

De-noising by soft-thresholding

Adapting to Unknown Smoothness via Wavelet Shrinkage

Contact Info

Product

Resources

About