The Hanson–Wright inequality for random tensors

Abstract

We provide moment bounds for expressions of the type $$(X^{(1)} \otimes \cdots \otimes X^{(d)})^T A (X^{(1)} \otimes \cdots \otimes X^{(d)})$$ ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) T A ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) where $$\otimes $$ ⊗ denotes the Kronecker product and $$X^{(1)}, \ldots , X^{(d)}$$ X ( 1 ) , … , X ( d ) are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on d for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form $$\Vert B (X^{(1)} \otimes \cdots \otimes X^{(d)})\Vert _2$$ ‖ B ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) ‖ 2 .