On strict sub-Gaussianity, optimal proxy variance and symmetry for bounded random variables

Arbel, Julyan; Marchal, Olivier; Nguyen, Hien D.

doi:10.1051/ps/2019018

Cited by 15 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We now use the identity in (73) to answer if P X|Y =y is strictly sub-Gaussian. As shown in [26,Prop. 4.3] a necessary condition for strict sub-Gaussianity requires that the third cumulant is zero.…”

Section: Example a Random Variable U With Meanmentioning

confidence: 92%

A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applications

Dytso

Poor

Shitz

2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Consider a channel Y = X + N where X is an n-dimensional random vector, and N is a multivariate Gaussian vector with a full-rank covariance matrix KN. The object under consideration in this paper is the conditional mean of X given Y = y, that is y → E[X|Y = y]. Several identities in the literature connect E[X|Y = y] to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities.In the first part of the paper, a general derivative identity for the conditional mean estimator is derived. Specifically, for the Markov chain U ↔ X ↔ Y, it is shown that the Jacobian matrix of E[U|Y = y] is given by K −1 N Cov(X, U|Y = y) where Cov(X, U|Y = y) is the conditional covariance.In the second part of the paper, via various choices of the random vector U, the new identity is used to recover and generalize many of the known identities and derive some new identities. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. The Jaffer identity is then further explored, and several equivalent stamens are derived, such as an identity for the higher-order conditional expectation (i.e., E[X k |Y]) in terms of the derivatives of the conditional expectation. Third, a new fundamental connection between the conditional cumulants and the conditional expectation is demonstrated. In particular, in the univariate case, it is shown that the k-th derivative of the conditional expectation is proportional to the (k +1)-th conditional cumulant. A similar expression is derived in the multivariate case.The third part of the paper considers various applications of the derived identities (mostly in the scalar case). In a first application, using the new identity for higher-order derivatives of the conditional expectation, a power series representation of the conditional expectation is derived. The power series representation, together with the Lagrange inversion theorem, is then used to find an expression for the compositional inverse of y → E[X|Y = y]. In a second application, the conditional expectation is viewed as a random variable and the probability distribution of E[X|Y ] and probability distribution of the estimator error (X − E[X|Y ]) are derived. In the third application, the new identities are used to show that the higher-order conditional expectations and the conditional cumulants depended on the joint distribution only through the marginal of Y . This observation is then used to construct consistent estimators (known as the empirical Bayes estimators) of the higher-order conditional expectations and the conditional cumulants from an independent and identically distributed sequence Y1, . . . , Yn.

show abstract

Section: Example a Random Variable U With Meanmentioning

confidence: 92%

A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applications

Dytso

Poor

Shitz

2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…so that Σ †X ψ2 ≈ Σ † 2 holds. (This is the case for example if X ∼ N(μ, Σ), or more generally for strict sub-Gaussians [4]). The right hand side of (1.8) then scales linearly in Σ † 2 , whereas (1.6) shows a quadratic behavior.…”

Section: Overview and Contributionsmentioning

confidence: 99%

“…Such bounds have been recently provided 562Ž. Kereta and T. Klock in [48] under the assumption that X is strictly sub-Gaussian [4], i.e. for some K > 0, and for arbitrary matrices U ∈ R k×D , X satisfies…”

Section: Covariance Matrix Estimationmentioning

confidence: 99%

Estimating covariance and precision matrices along subspaces

Kereta¹,

Klock²

2021

Electron. J. Statist.

View full text Add to dashboard Cite

We study the accuracy of estimating the covariance and the precision matrix of a D-variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspond to desired subspaces or directions. This is relevant and important for problems where the behavior of data along a lower-dimensional space is of specific interest, such as dimension reduction or structured regression problems. We also show that estimation of precision matrices is almost independent of the condition number of the covariance matrix. The presented applications include direction-sensitive eigenspace perturbation bounds, relative bounds for the smallest eigenvalue, and the estimation of the single-index model. For the latter, a new estimator, derived from the analysis, with strong theoretical guarantees and superior numerical performance is proposed.

show abstract

“…not almost surely equal to a constant) for all nonzero v ∈ Im(A), where Z is the standardization of X. In Table 1 we denote this condition by RCP (random conditional projection), and by RCP 2…”

Section: Related Workmentioning

confidence: 99%

Estimating multi-index models with response-conditional least squares

Klock¹,

Lanteri²,

Vigogna³

2021

Electron. J. Statist.

View full text Add to dashboard Cite

The multi-index model is a simple yet powerful high-dimensional regression model which circumvents the curse of dimensionality assuming E[Y |X] = g(A X) for some unknown index space A and link function g. In this paper we introduce a method for the estimation of the index space, and study the propagation error of an index space estimate in the regression of the link function. The proposed method approximates the index space by the span of linear regression slope coefficients computed over level sets of the data. Being based on ordinary least squares, our approach is easy to implement and computationally efficient. We prove a tight concentration bound that shows N −1/2 -convergence, but also faithfully describes the dependence on the chosen partition of level sets, hence providing guidance on the hyperparameter tuning. The estimator's competitiveness is confirmed by extensive comparisons with state-of-the-art methods, both on synthetic and real data sets. As a second contribution, we establish minimax optimal generalization bounds for k-nearest neighbors and piecewise polynomial regression when trained on samples projected onto any N −1/2 -consistent estimate of the index space, thus providing complete and provable estimation of the multi-index model.

show abstract

On strict sub-Gaussianity, optimal proxy variance and symmetry for bounded random variables

Cited by 15 publications

References 20 publications

A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applications

A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applications

Estimating covariance and precision matrices along subspaces

Estimating multi-index models with response-conditional least squares

Contact Info

Product

Resources

About