Estimation of smooth functionals in high-dimensional models: bootstrap chains and Gaussian approximation

Koltchinskii, Vladimir

doi:10.48550/arxiv.2011.03789

Cited by 3 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some other results on bootstrap chain bias reduction and efficient estimation of functionals of unknown parameters for Gaussian shift models, more general random shift models, log-concave location models can be found in [15,17,14]. In [9], this method was studied in the case of general high-dimensional parametric models under the assumption that there exists an estimator of unknown parameter whose distribution could be approximated by Gaussian with sufficient accuracy. In [28], a version of Taylor expansions method of bias reduction was developed for estimation of functionals of the unknown mean with not necessarily i.i.d.…”

Section: A Brief Review Of Related Resultsmentioning

confidence: 99%

Estimation of smooth functionals of covariance operators: jackknife bias reduction and bounds in terms of effective rank

Koltchinskii¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Let E be a separable Banach space and let X, X 1 , . . . , Xn, . . . be i.i.d. Gaussian random variables taking values in E with mean zero and unknown covariance operator Σ : E * → E. The complexity of estimation of Σ based on observations X 1 , . . . , Xn is naturally characterized by the so called effective rank of Σ :, where Σ is the operator norm of Σ. Given a smooth real valued functional f defined on the space L(E * , E) of symmetric linear operators from E * into E (equipped with the operator norm), our goal is to study the problem of estimation of f (Σ) based on X 1 , . . . , Xn. The estimators of f (Σ) based on jackknife type bias reduction are considered and the dependence of their Orlicz norm error rates on effective rank r(Σ), the sample size n and the degree of Hölder smoothness s of functional f are studied. In particular, it is shown that, if r(Σ) n α for some α ∈ (0, 1) and s ≥ 1 1−α , then the classical √ n-rate is attainable and, if s > 1 1−α , then asymptotic normality and asymptotic efficiency of the resulting estimators hold. Previously, the results of this type (for different estimators) were obtained only in the case of finite dimensional Euclidean space E = R d and for covariance operators Σ whose spectrum is bounded away from zero (in which case, r(Σ) ≍ d).

show abstract

Section: A Brief Review Of Related Resultsmentioning

confidence: 99%

Estimation of smooth functionals of covariance operators: jackknife bias reduction and bounds in terms of effective rank

Koltchinskii¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Consider the estimation of θ 0 " f pµq, if E is Banach or θ 0 " f pµ, Σq, if E is Hilbert. Koltchinskii andZhilova (2019, 2021) provide asymptotically efficient estimators of θ 0 which have a Normal limiting distribution. Also, see Koltchinskii (2020).…”

Section: Semiparametric Estimationmentioning

confidence: 99%

“…Koltchinskii andZhilova (2019, 2021) provide asymptotically efficient estimators of θ 0 which have a Normal limiting distribution. Also, see Koltchinskii (2020). The asymptotic variance is xΣf 1 pµq, f 1 pµqy if θ 0 " f pµq and is…”

Section: Semiparametric Estimationmentioning

confidence: 99%

The HulC: Confidence Regions from Convex Hulls

Kuchibhotla,

Balakrishnan,

Wasserman

2021

Preprint

View full text Add to dashboard Cite

We develop and analyze the HulC, an intuitive and general method for constructing confidence sets using the convex hull of estimates constructed from subsets of the data. Unlike classical methods which are based on estimating the (limiting) distribution of an estimator, the HulC is often simpler to use and effectively bypasses this step. In comparison to the bootstrap, the HulC requires fewer regularity conditions and succeeds in many examples where the bootstrap provably fails. Unlike subsampling, the HulC does not require knowledge of the rate of convergence of the estimators on which it is based. The validity of the HulC requires knowledge of the (asymptotic) median-bias of the estimators. We further analyze a variant of our basic method, called the Adaptive HulC, which is fully data-driven and estimates the median-bias using subsampling. We show that the Adaptive HulC retains the aforementioned strengths of the HulC. In certain cases where the underlying estimators are pathologically asymmetric the HulC and Adaptive HulC can fail to provide useful confidence sets. We propose a final variant, the Unimodal HulC, which can salvage the situation in cases where the distribution of the underlying estimator is (asymptotically) unimodal. We discuss these methods in the context of several challenging inferential problems which arise in parametric, semi-parametric, and non-parametric inference. Although our focus is on validity under weak regularity conditions, we also provide some general results on the width of the HulC confidence sets, showing that in many cases the HulC confidence sets have near-optimal width.

show abstract

“…If functional f is k times differentiable and d is small comparing with n, one could therefore expect that (B k f )(θ) d n k/2 (based on the analogy with the behavior of k-th order differences of k times differentiable functions in the real line). The justification of this heuristic for general parametric models could be rather involved (see [14,15,18,16]), but it will be shown below that it is much simpler in the case of equivariant estimators θ (such as the MLE) (see also [17,19]).…”

Section: Also Note Thatmentioning

confidence: 99%

“…Similar results were obtained in [17] in the case of Gaussian shift models, in [19] in the case of more general Poincaré random shift models and in [14,15,18] in the case of Gaussian models with unknown covariance and unknown mean and covariance (the analysis becomes much more involved in the case when the functional depends on unknown covariance). In [16], the proposed higher order bias reduction method was studied in the case of general models with a high-dimensional parameter θ for which there exists an estimator θ admitting high-dimensional normal approximation.…”

mentioning

confidence: 99%

Functional estimation in log-concave location families

Koltchinskii¹,

Wahl²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Let {P θ : θ ∈ R d } be a log-concave location family with P θ (dx) = e −V (x−θ) dx, where V : R d → R is a known convex function and let X1, . . . , Xn be i.i.d. r.v. sampled from distribution P θ with an unknown location parameter θ. The goal is to estimate the value f (θ) of a smooth functional f : R d → R based on observations X1, . . . , Xn. In the case when V is sufficiently smooth and f is a functional from a ball in a Hölder space C s , we develop estimators of f (θ) with minimax optimal error rates measured by the L2(P θ )-distance as well as by more general Orlicz norm distances. Moreover, we show that if d ≤ n α and s > 1 1−α , then the resulting estimators are asymptotically efficient in Hájek-LeCam sense with the convergence rate √ n. This generalizes earlier results on estimation of smooth functionals in Gaussian shift models. The estimators have the form f k ( θ), where θ is the maximum likelihood estimator and f k : R d → R (with k depending on s) are functionals defined in terms of f and designed to provide a higher order bias reduction in functional estimation problem. The method of bias reduction is based on iterative parametric bootstrap and it has been successfully used before in the case of Gaussian models.

show abstract

Estimation of smooth functionals in high-dimensional models: bootstrap chains and Gaussian approximation

Abstract: Let X (n) be an observation sampled from a distribution P (n) θ

Cited by 3 publications

References 38 publications

Estimation of smooth functionals of covariance operators: jackknife bias reduction and bounds in terms of effective rank

Estimation of smooth functionals of covariance operators: jackknife bias reduction and bounds in terms of effective rank

The HulC: Confidence Regions from Convex Hulls

Functional estimation in log-concave location families

Contact Info

Product

Resources

About