Adaptive risk bounds in univariate total variation denoising and trend filtering

Abstract

We study trend filtering, a relatively recent method for univariate nonparametric regression. For a given integer r ≥ 1, the r th order trend filtering estimator is defined as the minimizer of the sum of squared errors when we constrain (or penalize) the sum of the absolute r th order discrete derivatives of the fitted function at the design points. For r = 1, the estimator reduces to total variation regularization which has received much attention in the statistics and image processing literature. In this paper, we study the performance of the trend filtering estimator for every r ≥ 1, both in the constrained and penalized forms. Our main results show that in the strong sparsity setting when the underlying function is a (discrete) spline with few "knots", the risk (under the global squared error loss) of the trend filtering estimator (with an appropriate choice of the tuning parameter) achieves the parametric n −1 -rate, up to a logarithmic (multiplicative) factor. Our results therefore provide support for the use of trend filtering, for every r ≥ 1, in the strong sparsity setting.

Oracle inequalities for square root analysis estimators with application to total variation penalties

“…Due to the use of the bound given by Lemma 4.4, Corollary 4.1 assumes a minimal length condition. This condition does not depend on n and is therefore weaker than the one found in Guntuboyina et al (2017). Note that the choice of the tuning parameter depends both on σ and n max = n max (S).…”

New Risk Bounds for 2D Total Variation Denoising

“…This is the main contribution of this paper and to the best of our knowledge is the first of its kind in the literature. Guntuboyina et al (2017) As mentioned in Section I, one of our motivating factors behind investigating adaptivity of the 2D TVD estimator was its success in optimally estimating piecewise constant vectors in the 1D setting. Theorem 2.2 in Guntuboyina et al (2017) gives a ‹ O(k * /n) rate for the ideally tuned constrained 1D TVD estimator when the truth θ * is piecewise constant with k * pieces or blocks and each block satisfies a certain minimum length condition.…”

“…Thus, there seems to be two routes for obtaining fast rates for TVD. One goes through the route of bounding Gaussian width of an appropriate tangent cone to derive fast rates for the constrained TVD estimator; as done here in this manuscript as well as in Guntuboyina et al (2017). The other route; followed by Hütter and Rigollet (2016) and generalized by Ortelli and van de Geer (2019b) is based on bounding the so called compatibility factor.…”

“…The bounded aspect ratio condition says that the rectangular level sets of θ * should not be too skinny or too long. In our proof, the bounded aspect ratio is needed for similar reasons as a minimum length condition is needed for the length of the pieces in the 1D setting; see Guntuboyina et al (2017), Dalalyan et al (2017). O(V * / √ N )?…”

Modeling swine population dynamics at a finer temporal resolution

“…The non‐negative scalar is used to control for the size of the penalties. Guntuboyina et al 22 provided theoretical results and optimality conditions of the estimator obtained by minimizing (6).…”