Oblivious Sketching of High-Degree Polynomial Kernels

Ahle, Thomas D.; Kapralov, Michael; Knudsen, Jakob Bæk Tejs; Pagh, Rasmus; Velingker, Ameya; Woodruff, David P.; Zandieh, Amir

doi:10.1137/1.9781611975994.9

Cited by 25 publications

(85 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative approach would be to incorporate matrix sketching techniques to allow for more efficient computation of the SVD in the tensorized domain. For example, recent work that focuses on sketches for matrices with Kronecker product structure, such as TensorSketch [30] and its modifications [1] or the Kronecker fast Johnson--Lindenstrauss transform [21], could potentially be adapted to our setting.…”

Section: Discussionmentioning

confidence: 99%

Tensor Methods for Nonlinear Matrix Completion

Ongie¹,

Pimentel-Alarcón²,

Balzano³

et al. 2021

SIAM Journal on Mathematics of Data Science

View full text Add to dashboard Cite

In the low-rank matrix completion (LRMC) problem, the low-rank assumption means that the columns (or rows) of the matrix to be completed are points on a low-dimensional linear algebraic variety. This paper extends this thinking to cases where the columns are points on a low-dimensional nonlinear algebraic variety, a problem we call low algebraic dimension matrix completion (LADMC). Matrices whose columns belong to a union of subspaces are an important special case. We propose an LADMC algorithm that leverages existing LRMC methods on a tensorized representation of the data. For example, a second-order tensorized representation is formed by taking the Kronecker product of each column with itself, and we consider higher-order tensorizations as well. This approach will succeed in many cases where traditional LRMC is guaranteed to fail because the data are low-rank in the tensorized representation but are not in the original representation. We also provide a formal mathematical justification for the success of our method. In particular, we give bounds on the rank of these data in the tensorized representation, and we prove sampling requirements to guarantee uniqueness of the solution. We also provide experimental results showing that the new approach outperforms existing state-of-the-art methods for matrix completion under a union-of-subspaces model.

show abstract

Section: Discussionmentioning

confidence: 99%

Tensor Methods for Nonlinear Matrix Completion

Ongie¹,

Pimentel-Alarcón²,

Balzano³

et al. 2021

SIAM Journal on Mathematics of Data Science

View full text Add to dashboard Cite

show abstract

“…In the course of adapting fast Johnson-Lindenstrauss embeddings to data with Kronecker structure as introduced in [6] (see also [3,11]), one encounters expressions of the form (X (1) ⊗ • • • ⊗ X (d) ) T A(X (1) ⊗ • • • ⊗ X (d) ) which are somewhat intermediate between ( 1) and ( 2), as they can be expanded as n i 1 ,...,i 2d =1…”

Section: Background and Studied Objectsmentioning

confidence: 99%

“…Possible applications of such results include recent developments in norm-preserving maps for vectors with tensor structure in the context of machine learning methods using the kernel trick [6,3,11].…”

Section: Overview Of Our Contributionmentioning

confidence: 99%

The Hanson-Wright Inequality for Random Tensors

Bamberger¹,

Krahmer²,

Ward³

2021

Preprint

View full text Add to dashboard Cite

We provide moment bounds for expressions of the type (Xwhere ⊗ denotes the Kronecker product and X (1) , . . . , X (d) are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on d for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form B(X (1) ⊗ • • • ⊗ X (d) ) 2 .

show abstract

“…Modewise multiplication and j-mode products: For 1 ď j ď d, the j-mode product of d-mode tensor X P R n 1 ˆ¨¨¨ˆn j´1 ˆnj ˆnj`1 ˆ¨¨¨ˆn d with a matrix U P R m j ˆnj is another d-mode tensor X ˆj U P R n 1 ˆ¨¨¨ˆn j´1 ˆmj ˆnj`1 ˆ¨¨¨ˆn d . Its entries are given by (2) pX ˆj U q i 1 ,...,i j´1 , ,i j`1 ,...,i d "…”

Section: Tensor Prerequisitesmentioning

confidence: 99%

“…Other recent work involving the analysis of modewise maps for tensor data include, e.g., applications in kernel learning methods which effectively use modewise operators specialized to finite sets of rank-one tensors [2], as well as a variety of works in the computer science literature aimed at compressing finite sets of low-rank (with respect to, e.g., CP and tensor train decompositions [32]) tensors. More general results involving extensions of bounded orthonormal sampling results to the tensor setting [23,3] apply to finite sets of arbitrary tensors.…”

Section: Introduction and Prior Workmentioning

confidence: 99%

Modewise Operators, the Tensor Restricted Isometry Property, and Low-Rank Tensor Recovery

Iwen¹,

Needell²,

Perlmutter³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recovery of sparse vectors and low-rank matrices from a small number of linear measurements is well-known to be possible under various model assumptions on the measurements.The key requirement on the measurement matrices is typically the restricted isometry property, that is, approximate orthonormality when acting on the subspace to be recovered. Among the most widely used random matrix measurement models are (a) independent sub-gaussian models and (b) randomized Fourier-based models, allowing for the efficient computation of the measurements.For the now ubiquitous tensor data, direct application of the known recovery algorithms to the vectorized or matricized tensor is awkward and memory-heavy because of the huge measurement matrices to be constructed and stored. In this paper, we propose modewise measurement schemes based on sub-gaussian and randomized Fourier measurements. These modewise operators act on the pairs or other small subsets of the tensor modes separately. They require significantly less memory than the measurements working on the vectorized tensor, provably satisfy the tensor restricted isometry property and experimentally can recover the tensor data from fewer measurements and do not require impractical storage.

show abstract

Oblivious Sketching of High-Degree Polynomial Kernels

Cited by 25 publications

References 0 publications

Tensor Methods for Nonlinear Matrix Completion

Tensor Methods for Nonlinear Matrix Completion

The Hanson-Wright Inequality for Random Tensors

Modewise Operators, the Tensor Restricted Isometry Property, and Low-Rank Tensor Recovery

Contact Info

Product

Resources

About