Efficient Density Evaluation for Smooth Kernels

Bačkurs, Artūrs; Charikar, Moses; Indyk, Piotr; Siminelakis, Paris

doi:10.1109/focs.2018.00065

Cited by 11 publications

(33 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, this approach turns out to be constraining due to its reliance on black-box c-ANN calls, and in particular only applies to polynomial kernels. Our techniques in this paper recover the result of [BCIS18] up to µ −o (1) factors as a special case (see Section 4). Furthermore, the µ −o (1) factor loss that we incur is only due to the fact that we are using the powerful Euclidean LSH family in order to achieve strong bounds for kernels that exhibit fast decay (e.g., Gaussian, exponential and others) using the same algorithm.…”

Section: Related Worksupporting

confidence: 58%

“…We remark that the actual non-adaptive algorithm that we present in Section 4 is more general than the above and applies to a wide class of kernels. In particular, it simultaneously improves upon all prior work on radial kernels that exhibit fast tail decay (such as the exponential and the Gaussian kernels) [CS17] as well as matches the result of [BCIS18] on kernels with only inverse polynomial rate of decay up to µ −o (1) factors.…”

Section: Data-independent Algorithm (Section 4)mentioning

confidence: 61%

Section: Related Workmentioning

confidence: 99%

“…The work most closely related to ours is that of [BCIS18]. [BCIS18] introduced a technique, called Spherical Integration, that uses black-box calls to c-ANN data structures (constructed on sub-sampled versions of the data set) to sample points from "spherical annuli" (r, cr) around the query, for all annuli that had non-negligible contribution to the density of the query. For kernels with polynomial tails of degree t, their approach required O(c 5t ) calls to such data-structures (without counting the query time required for each such call) to estimate the density.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Kernel Density Estimation through Density Constrained Near Neighbor Search

Charikar¹,

Kapralov²,

Nouri³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we revisit the kernel density estimation problem: given a kernel K(x, y) and a dataset of n points in high dimensional Euclidean space, prepare a data structure that can quickly output, given a query q, a (1+ǫ)-approximation to µ := 1 |P | p∈P K(p, q). First, we give a single data structure based on classical near neighbor search techniques that improves upon or essentially matches the query time and space complexity for all radial kernels considered in the literature so far. We then show how to improve both the query complexity and runtime by using recent advances in data-dependent near neighbor search.We achieve our results by giving a new implementation of the natural importance sampling scheme. Unlike previous approaches, our algorithm first samples the dataset uniformly (considering a geometric sequence of sampling rates), and then uses existing approximate near neighbor search techniques on the resulting smaller dataset to retrieve the sampled points that lie at an appropriate distance from the query. We show that the resulting sampled dataset has strong geometric structure, making approximate near neighbor search return the required samples much more efficiently than for worst case datasets of the same size. As an example application, we show that this approach yields a data structure that achieves query time µ −(1+o(1))/4 and space complexity µ −(1+o(1)) for the Gaussian kernel. Our data dependent approach achieves query time µ −0.173−o(1) and space µ −(1+o(1)) for the Gaussian kernel. The data dependent analysis relies on new techniques for tracking the geometric structure of the input datasets in a recursive hashing process that we hope will be of interest in other applications in near neighbor search.

show abstract

Section: Related Worksupporting

confidence: 58%

Section: Data-independent Algorithm (Section 4)mentioning

confidence: 61%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Kernel Density Estimation through Density Constrained Near Neighbor Search

Charikar¹,

Kapralov²,

Nouri³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Over the last decade many such algorithms have been proposed. In our context, the most relevant ones are those solving the problem of kernel density evaluation [Charikar and Siminelakis, 2017, Backurs et al, 2018, Siminelakis et al, 2019, Charikar et al, 2020. Here, we are given two sets of vectors X = {x 1 , .…”

Section: Introductionmentioning

confidence: 99%

Faster Kernel Matrix Algebra via Density Estimation

Bačkurs¹,

Indyk²,

Musco³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We study fast algorithms for computing fundamental properties of a positive semidefinite kernel matrix K ∈ R n×n corresponding to n points x 1 , . . . , x n ∈ R d . In particular, we consider estimating the sum of kernel matrix entries, along with its top eigenvalue and eigenvector.We show that the sum of matrix entries can be estimated to 1 + relative error in time sublinear in n and linear in d for many popular kernels, including the Gaussian, exponential, and rational quadratic kernels. For these kernels, we also show that the top eigenvalue (and an approximate eigenvector) can be approximated to 1 + relative error in time subquadratic in n and linear in d.Our algorithms represent significant advances in the best known runtimes for these problems. They leverage the positive definiteness of the kernel matrix, along with a recent line of work on efficient kernel density estimation.

show abstract

Multi-resolution Hashing for Fast Pairwise Summations

Charikar

Siminelakis

2019

2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS)

Self Cite

View full text Add to dashboard Cite

A basic computational primitive in the analysis of massive datasets is summing simple functions over a large number of objects. Modern applications pose an additional challenge in that such functions often depend on a parameter vector y (query) that is unknown a priori. Given a set of points X ⊂ d and a pairwise function w : d × d → [0, 1], we study the problem of designing a data-structure that enables sublinear-time approximation of the summation Z w (y) 1 |X| x∈X w(x, y) for any query y ∈ d . By combining ideas from Harmonic Analysis (partitions of unity and approximation theory) with Hashing-Based-Estimators [Charikar, Siminelakis FOCS'17], we provide a general framework for designing such data structures through hashing that reaches far beyond what previous techniques allowed.A key design principle is a collection of T 1 hashing schemes with collision probabilities p 1 , . . . , p T such that sup t∈[T] {p t (x, y)} Θ( w(x, y)). This leads to a data-structure that approximates Z w (y) using a sub-linear number of samples from each hash family. Using this new framework along with Distance Sensitive Hashing [Aumuller, Christiani, Pagh, Silvestri PODS'18], we show that such a collection can be constructed and evaluated efficiently for any log-convex function w(x, y) e φ( x, y ) of the inner product on the unit sphere x, y ∈ S d−1 .Our method leads to data structures with sub-linear query time that significantly improve upon random sampling and can be used for Kernel Density or Partition Function Estimation. We provide extensions of our result from the sphere to d and from scalar functions to vector functions.

show abstract

Efficient Density Evaluation for Smooth Kernels

Cited by 11 publications

References 32 publications

Kernel Density Estimation through Density Constrained Near Neighbor Search

Kernel Density Estimation through Density Constrained Near Neighbor Search

Faster Kernel Matrix Algebra via Density Estimation

Multi-resolution Hashing for Fast Pairwise Summations

Contact Info

Product

Resources

About