Variational multiscale nonparametric regression: Smooth functions

Abstract

For the problem of nonparametric regression of smooth functions, we reconsider and analyze a constrained variational approach, which we call the MultIscale Nemirovski-Dantzig (MIND) estimator. This can be viewed as a multiscale extension of the Dantzig selector (Ann. Statist., 35(6): 2313-51, 2009) based on early ideas of Nemirovski (J. Comput. System Sci., 23:1-11, 1986). MIND minimizes a homogeneous Sobolev norm under the constraint that the multiresolution norm of the residual is bounded by a universal threshold. The main contribution of this paper is the derivation of convergence rates of MIND with respect to L q -loss, 1 ≤ q ≤ ∞, both almost surely and in expectation. To this end, we introduce the method of approximate source conditions. For a onedimensional signal, these can be translated into approximation properties of B-splines. A remarkable consequence is that MIND attains almost minimax optimal rates simultaneously for a large range of Sobolev and Besov classes, which provides certain adaptation. Complimentary to the asymptotic analysis, we examine the finite sample performance of MIND by numerical simulations.

Multiscale change-point segmentation: beyond step functions

“…which can be shown to be 2-normal, see Grasmair et al (2018). (ii) the scale penalties s I satisfy almost surely that sup I∈I |s I | ≤ δ log n for some constant δ > 0.…”

Efficient regularization with wavelet sparsity constraints in photoacoustic tomography

“…Such hybrid approaches have been introduced independently in [18,34,70,89] (see also [21,24]). It is also related to the Dantzig and multiscale estimators of [19,41,43,50,56,74].…”

On polyhedral estimation of signals via indirect observations

“…To the best of our knowledge, the idea of polyhedral estimate goes back to [34], see also [33,Chapter 2], where it was shown that when recovering smooth multivariate regression functions known to belong to Sobolev balls from their noisy observations taken along a regular grid Γ, a polyhedral estimate with ad hoc selected contrast matrix is near-optimal in a wide range of smoothness characterizations and norms · . Recently, the ideas underlying the results of [34] have been taken up in the MIND estimator of [18], then applied in the indirect observation setting in [37] in the context of multiple testing. The goal of this paper is to investigate characteristics of the polyhedral estimate, with a particular emphasis on efficiently computable upper bounds for the risk of the estimate w H (·) and design of the contrast matrix H resulting in the (nearly) best upper risk bounds.…”

“…Risk bound (20) allows for a straightforward design of contrast matrices. Recalling that Ψ is monotone on the nonnegative orthant, all we need is to select h 's satisfying (18) and resulting in the smallest possible ς 's, which is what we are about to do now.…”

“…Application. Let us apply the result of Proposition 3.3 to the "standard case" (cf., e.g., [9,10,18]…”