Oracle inequalities for square root analysis estimators with application to total variation penalties

Abstract

Through the direct study of the analysis estimator we derive oracle inequalities with fast and slow rates by adapting the arguments involving projections by Dalalyan, Hebiri and Lederer (2017). We then extend the theory to the square root analysis estimator. Finally, we focus on (square root) total variation regularized estimators on graphs and obtain constant-friendly rates, which, up to log-terms, match previous results obtained by entropy calculations. We also obtain an oracle inequality for the (square root) total variation regularized estimator over the cycle graph.

New Risk Bounds for 2D Total Variation Denoising

“…The 2D TVD problem, while being much less studied than in its 1D counterpart, has enjoyed a recent surge of interest. Worst case performance of the TVD estimator has been studied in Hütter and Rigollet (2016), Sadhanala et al (2016), Ortelli and van de Geer (2019b). These results show that like in the 1D setting, the 2D TVD estimator is nearly minimax rate optimal over the class {θ ∈ R n×n : TV norm (θ) ≤ V} of bounded variation signals.…”

“…In fact, Sadhanala et al (2016) also generalize the result of Donoho and Johnstone (1998) and prove that no linear function of y can attain the minimax rate in the 2D setting as well. A representative of the state of the art risk bound for the TVD estimator in 2D setting is due to Hütter and Rigollet (2016) (see also Ortelli and van de Geer (2019b)). They studied the penalized form of the TVD estimator and proved that there exist universal constants C, c > 0 such that by setting λ = cσ log n (where σ is known), one gets Theorem I.1 (Hütter and Rigollet).…”

“…There are some proposals available in the literature; see Solo (1999), Osadebey et al (2014), Langer (2017. Soon after we uploaded our paper, a different tuning parameter free method appeared in Ortelli and van de Geer (2019b) which also achieves the optimal worst case ‹ O(V/ √ N ) rate of convergence. See Section III-C for a comparison of our method with the one proposed in Ortelli and van de Geer (2019b).…”

“…At the latter stages of preparation of this manuscript we became aware of an independent work by Ortelli and van de Geer (2019b) which is related to our manuscript. In Ortelli and van de Geer (2019b), the authors give a general technique to derive slow ( 1 ) and fast ( 0 ) rates for penalized TVD estimators and its square root version on general graphs. Thus, there seems to be two routes for obtaining fast rates for TVD.…”

“…One goes through the route of bounding Gaussian width of an appropriate tangent cone to derive fast rates for the constrained TVD estimator; as done here in this manuscript as well as in Guntuboyina et al (2017). The other route; followed by Hütter and Rigollet (2016) and generalized by Ortelli and van de Geer (2019b) is based on bounding the so called compatibility factor. Ortelli and van de Geer (2019b) show how to bound this compatibility factor for specific graphs such as the 1d grid graph and the 1d cycle graph.…”