The Hanson-Wright Inequality for Random Tensors

Bamberger, Stefan; Krahmer, Felix; Ward, Rachel

doi:10.48550/arxiv.2106.13345

Cited by 1 publication

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is extensive literature dedicated to extending and adapting the Hanson-Wright inequality (e.g., [29,1,46,5]). Yet among popular sketching methods, only Gaussian and sub-gaussian embeddings are known to exhibit this property with a small constant K = O(1).…”

Section: Sub-gaussian Limit Theorem Via New Hanson-wright Inequalitymentioning

confidence: 99%

Algorithmic Gaussianization through Sketching: Converting Data into Sub-gaussian Random Designs

Dereziński¹

2022

Preprint

View full text Add to dashboard Cite

Algorithmic Gaussianization is a phenomenon that can arise when using randomized sketching or sampling methods to produce smaller representations of large datasets: For certain tasks, these sketched representations have been observed to exhibit many robust performance characteristics that are known to occur when a data sample comes from a sub-gaussian random design, which is a powerful statistical model of data distributions. However, this phenomenon has only been studied for specific tasks and metrics, or by relying on computationally expensive methods. We address this by providing an algorithmic framework for gaussianizing data distributions via averaging, proving that it is possible to efficiently construct data sketches that are nearly indistinguishable (in terms of total variation distance) from sub-gaussian random designs. In particular, relying on a recently introduced sketching technique called Leverage Score Sparsified (LESS) embeddings, we show that one can construct an n × d sketch of an N × d matrix A, where n N , that is nearly indistinguishable from a sub-gaussian design, in time O(nnz(A) log N + nd 2 ), where nnz(A) is the number of non-zero entries in A. As a consequence, strong statistical guarantees and precise asymptotics available for the estimators produced from sub-gaussian designs (e.g., for least squares and Lasso regression, covariance estimation, low-rank approximation, etc.) can be straightforwardly adapted to our sketching framework. We illustrate this with a new approximation guarantee for sketched least squares, among other examples.

show abstract

Section: Sub-gaussian Limit Theorem Via New Hanson-wright Inequalitymentioning

confidence: 99%

Algorithmic Gaussianization through Sketching: Converting Data into Sub-gaussian Random Designs

Dereziński¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

The Hanson-Wright Inequality for Random Tensors

Cited by 1 publication

References 16 publications

Algorithmic Gaussianization through Sketching: Converting Data into Sub-gaussian Random Designs

Algorithmic Gaussianization through Sketching: Converting Data into Sub-gaussian Random Designs

Contact Info

Product

Resources

About