Replicant compression coding in Besov spaces

Kerkyacharian, Gérard; Picard, Dominique

doi:10.1051/ps:2003011

Cited by 3 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the isotropic Sobolev space, Theorem 9 was obtained in the key paper Birman and Solomjak (1967) (see Theorem 5.2 herein), and for the isotropic Besov space, it can be found, among others, in Birgé and Massart (2000) and Kerkyacharian and Picard (2003).…”

Section: Notationsmentioning

confidence: 86%

“…Remark 3. A more constructive computation of the entropy of anisotropic Besov spaces can be done using the replicant coding approach, which is done for Besov bodies in Kerkyacharian and Picard (2003). Using this approach together with an anisotropic multiresolution analysis based on compactly supported wavelets or atoms, see Section 5.2 in Triebel (2006), we can obtain a direct computation of the entropy.…”

Section: Notationsmentioning

confidence: 99%

See 1 more Smart Citation

Hyper-sparse optimal aggregation

Gaïffas,

Lecué

2009

Preprint

View full text Add to dashboard Cite

In this paper, we consider the problem of hyper-sparse aggregation. Namely, given a dictionary F = {f1, . . . , fM } of functions, we look for an optimal aggregation algorithm that writes f = P M j=1 θjfj with as many zero coefficients θj as possible. This problem is of particular interest when F contains many irrelevant functions that should not appear in f . We provide an exact oracle inequality for f , where only two coefficients are non-zero, that entails f to be an optimal aggregation algorithm. Since selectors are suboptimal aggregation procedures, this proves that 2 is the minimal number of elements of F required for the construction of an optimal aggregation procedures in every situations. A simulated example of this algorithm is proposed on a dictionary obtained using LARS, for the problem of selection of the regularization parameter of the LASSO. We also give an example of use of aggregation to achieve minimax adaptation over anisotropic Besov spaces, which was not previously known in minimax theory (in regression on a random design).

show abstract

Section: Notationsmentioning

confidence: 86%

Section: Notationsmentioning

confidence: 99%

Hyper-sparse optimal aggregation

Gaïffas,

Lecué

2009

Preprint

View full text Add to dashboard Cite

show abstract

“…For the isotropic Sobolev space, Theorem 5 was obtained in the key paper [6] (see Theorem 5.2 herein), and for the isotropic Besov space, it can be found, among others, in [5] and [26].…”

Section: Proofs Of the Lemmasmentioning

confidence: 86%

“…This technique allows to prove the statements below. Note that another technique of proof based on replicant coding can be used, see [26]. This is commented below.…”

Section: Proofs Of the Lemmasmentioning

confidence: 99%

“…Remark. A more constructive computation of the entropy of anisotropic Besov spaces can be done using the replicant coding approach, which is done for Besov bodies in [26]. Using this approach together with an anisotropic multiresolution analysis based on compactly supported wavelets or atoms, see Section 5.2 in [39], we can obtain a direct computation of the entropy.…”

Section: Proofs Of the Lemmasmentioning

confidence: 99%

See 1 more Smart Citation

Aggregation of penalized empirical risk minimizers in regression

Gaïffas,

Lecué

2008

Preprint

View full text Add to dashboard Cite

We give a general result concerning the rates of convergence of penalized empirical risk minimizers (PERM) in the regression model. Then, we consider the problem of agnostic learning of the regression, and give in this context an oracle inequality and a lower bound for PERM over a finite class. These results hold for a general multivariate random design, the only assumption being the compactness of the support of its law (allowing discrete distributions for instance). Then, using these results, we construct adaptive estimators. We consider as examples adaptive estimation over anisotropic Besov spaces or reproductive kernel Hilbert spaces. Finally, we provide an empirical evidence that aggregation leads to more stable estimators than more standard cross-validation or generalized cross-validation methods for the selection of the smoothing parameter, when the number of observation is small.

show abstract

Wavelet kernel penalized estimation for non-equispaced design regression

2006

View full text Add to dashboard Cite

The paper considers regression problems with univariate design points. The design points are irregular and no assumptions on their distribution are imposed. The regression function is retrieved by a wavelet based reproducing kernel Hilbert space (RKHS) technique with the penalty equal to the sum of blockwise RKHS norms. In order to simplify numerical optimization, the problem is replaced by an equivalent quadratic minimization problem with an additional penalty term. The computational algorithm is described in detail and is implemented with both the sets of simulated and real data. Comparison with existing methods showed that the technique suggested in the paper does not oversmooth the function and is superior in terms of the mean squared error. It is also demonstrated that under additional assumptions on design points the method achieves asymptotic optimality in a wide range of Besov spaces.

show abstract

Replicant compression coding in Besov spaces

Cited by 3 publications

References 19 publications

Hyper-sparse optimal aggregation

Hyper-sparse optimal aggregation

Aggregation of penalized empirical risk minimizers in regression

Wavelet kernel penalized estimation for non-equispaced design regression

Contact Info

Product

Resources

About