Johnson-Lindenstrauss Embeddings with Kronecker Structure

Bamberger, Stefan; Krahmer, Felix; Ward, Rachel

doi:10.48550/arxiv.2106.13349

Cited by 3 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application [14] generalizes the result in [5] on constructing Johnson-Lindenstrauss embeddings from matrices satisfying the restricted isometry property to Johnson-Lindenstrauss embeddings with a fast transformation of Kronecker products. This leads to expressions of the type D ξ x 2 2 , where ∈ R m×N is a matrix, x ∈ R N a vector, and D ξ ∈ R N ×N is a diagonal matrix with entries from ξ = ξ (1) ⊗• • •⊗ξ (d) , where ξ (1) , .…”

Section: Discussionmentioning

confidence: 65%

“…Our bounds imply improved estimates for (4) and lay the foundations for an order-optimal analysis of fast Kronecker-structured Johnson-Lindenstrauss embeddings. We refer the reader to our companion paper [14] for a discussion of the implications in this regard. We nevertheless expect that our results should find broader use beyond these specific applications.…”

Section: Background and Studied Objectsmentioning

confidence: 99%

See 1 more Smart Citation

The Hanson–Wright inequality for random tensors

Bamberger

Krahmer

Ward

2022

Sampl. Theory Signal Process. Data Anal.

Self Cite

View full text Add to dashboard Cite

We provide moment bounds for expressions of the type $$(X^{(1)} \otimes \cdots \otimes X^{(d)})^T A (X^{(1)} \otimes \cdots \otimes X^{(d)})$$ ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) T A ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) where $$\otimes $$ ⊗ denotes the Kronecker product and $$X^{(1)}, \ldots , X^{(d)}$$ X ( 1 ) , … , X ( d ) are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on d for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form $$\Vert B (X^{(1)} \otimes \cdots \otimes X^{(d)})\Vert _2$$ ‖ B ( X ( 1 ) ⊗ ⋯ ⊗ X ( d ) ) ‖ 2 .

show abstract

Section: Discussionmentioning

confidence: 65%

Section: Background and Studied Objectsmentioning

confidence: 99%

The Hanson–Wright inequality for random tensors

Bamberger

Krahmer

Ward

2022

Sampl. Theory Signal Process. Data Anal.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other recent work involving the analysis of modewise maps for tensor data include, e.g., applications in kernel learning methods which effectively use modewise operators specialized to finite sets of rank-one tensors [2], as well as a variety of works in the computer science literature aimed at compressing finite sets of low-rank (with respect to, e.g., CP and tensor train decompositions [32]) tensors. More general results involving extensions of bounded orthonormal sampling results to the tensor setting [23,3] apply to finite sets of arbitrary tensors. With respect to norm-preserving modewise embeddings of infinite sets, prior work has been limited to oblivious subspace embeddings (see, e.g., [21,28]).…”

Section: Introduction and Prior Workmentioning

confidence: 99%

Modewise Operators, the Tensor Restricted Isometry Property, and Low-Rank Tensor Recovery

Iwen¹,

Needell²,

Perlmutter³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recovery of sparse vectors and low-rank matrices from a small number of linear measurements is well-known to be possible under various model assumptions on the measurements.The key requirement on the measurement matrices is typically the restricted isometry property, that is, approximate orthonormality when acting on the subspace to be recovered. Among the most widely used random matrix measurement models are (a) independent sub-gaussian models and (b) randomized Fourier-based models, allowing for the efficient computation of the measurements.For the now ubiquitous tensor data, direct application of the known recovery algorithms to the vectorized or matricized tensor is awkward and memory-heavy because of the huge measurement matrices to be constructed and stored. In this paper, we propose modewise measurement schemes based on sub-gaussian and randomized Fourier measurements. These modewise operators act on the pairs or other small subsets of the tensor modes separately. They require significantly less memory than the measurements working on the vectorized tensor, provably satisfy the tensor restricted isometry property and experimentally can recover the tensor data from fewer measurements and do not require impractical storage.

show abstract

“…The condition in Definition 2.3 considers length preservation of a single vector. A standard union bound argument can be used to show that a JL matrix with probability 1 − δ satisfies (2.4) for all u, v ∈ P where P contains m points, provided that d is chosen to be sufficiently large; see Remark 2.2 of [3] for a discussion about this.…”

mentioning

confidence: 99%

Construction of Hierarchically Semi-Separable matrix Representation using Adaptive Johnson-Lindenstrauss Sketching

Yaniv¹,

Malik²,

Ghysels³

et al. 2023

Preprint

View full text Add to dashboard Cite

We extend an adaptive partially matrix-free Hierarchically Semi-Separable (HSS) matrix construction algorithm by Gorman et al. [SIAM J. Sci. Comput. 41(5), 2019] which uses Gaussian sketching operators to a broader class of Johnson-Lindenstrauss (JL) sketching operators. We present theoretical work which justifies this extension. In particular, we extend the earlier concentration bounds to all JL sketching operators and examine this bound for specific classes of such operators including the original Gaussian sketching operators, subsampled randomized Hadamard transform (SRHT) and the sparse Johnson-Lindenstrauss transform (SJLT). We discuss the implementation details of applying SJLT efficiently and demonstrate experimentally that using SJLT instead of Gaussian sketching operators leads to 1.5-2.5× speedups of the HSS construction implementation in the STRUMPACK C++ library. The generalized algorithm allows users to select their own JL sketching operators with theoretical lower bounds on the size of the operators which may lead to faster run time with similar HSS construction accuracy.

show abstract

Johnson-Lindenstrauss Embeddings with Kronecker Structure

Cited by 3 publications

References 22 publications

The Hanson–Wright inequality for random tensors

The Hanson–Wright inequality for random tensors

Modewise Operators, the Tensor Restricted Isometry Property, and Low-Rank Tensor Recovery

Construction of Hierarchically Semi-Separable matrix Representation using Adaptive Johnson-Lindenstrauss Sketching

Contact Info

Product

Resources

About