New directions in nearest neighbor searching with applications to lattice sieving

Abstract

To solve the approximate nearest neighbor search problem (NNS) on the sphere, we propose a method using locality-sensitive filters (LSF), with the property that nearby vectors have a higher probability of surviving the same filter than vectors which are far apart. We instantiate the filters using spherical caps of height 1 − α, where a vector survives a filter if it is contained in the corresponding spherical cap, and where ideally each filter has an independent, uniformly random direction.For small α, these filters are very similar to the spherical locality-sensitive hash (LSH) family previously studied by Andoni et al. For larger α bounded away from 0, these filters potentially achieve a superior performance, provided we have access to an efficient oracle for finding relevant filters. Whereas existing LSH schemes are limited by a performance parameter of ρ ≥ 1/(2c 2 − 1) to solve approximate NNS with approximation factor c, with spherical LSF we potentially achieve smaller asymptotic values of ρ, depending on the density of the data set. For sparse data sets where the dimension is super-logarithmic in the size of the data set, we asymptotically obtain ρ = 1/(2c 2 − 1), while for a logarithmic dimensionality with density constant κ we obtain asymptotics of ρ ∼ 1/(4κc 2 ). To instantiate the filters and prove the existence of an efficient decoding oracle, we replace the independent filters by filters taken from certain structured random product codes. We show that the additional structure in these concatenation codes allows us to decode efficiently using techniques similar to lattice enumeration, and we can find the relevant filters with low overhead, while at the same time not significantly changing the collision *

NTRU Prime: Reducing Attack Surface at Low Cost

“…The best squeezing method we found uses vpmulhrsw, which performs 16 separate copies of the following operation: multiply two integers between −2 15 and 2 15 , divide by 2 15 , and round to an integer. We take the second integer as 7; then the output is round(7x/2 15 ) where x is the first integer.…”

“…Now we rewrite f m as p−1 i=0 f i (x i m). Since this sum adds at most 2t terms of each degree, it just remains to be shown that the coefficients of [15,79]) has pushed the heuristic complexity down to 2 0.292...β+o(β) .…”

Sieving for Shortest Vectors in Lattices Using Angular Locality-Sensitive Hashing

“…The remaining part of this subsection is dedicated to proving (11) and (12). We first prove that nearby vectors often collide in at least one of the hash tables, given that k is a suitable function of t. We then show how p * 2 scales as a function of k and t and how to choose k and t to minimize the asymptotic time complexity.…”

“…Although this work focuses on applying angular LSH to sieving, more generally this work could be considered the first to succeed in applying LSH to lattice algorithms. Various recent followup works have already further investigated the use of different LSH methods [7,8] and other nearest neighbor search methods [9,11,38] in the context of lattice sieving [11][12][13]30,37], and an open problem is whether other lattice algorithms (e.g. provable sieving algorithms, the Voronoi cell algorithm [39]) can benefit from related techniques as well.…”

Optimal Hashing-based Time-Space Trade-offs for Approximate Near Neighbors

“…This is the first data structure that achieves sublinear query time and near-linear space for every approximation factor c > 1, improving upon [Kap15]. The data structure is a culmination of a long line of work on the problem for all space regimes; it builds on Spherical Locality-Sensitive Filtering [BDGL16] and datadependent hashing [AINR14,AR15]. Our matching lower bounds are of two types: conditional and unconditional.…”