Parallel cosine nearest neighbor graph construction

Abstract

The nearest neighbor graph is an important structure in many data mining methods for clustering, advertising, recommender systems, and outlier detection. Constructing the graph requires computing up to n 2 similarities for a set of n objects. This high complexity has led researchers to seek approximate methods, which find many but not all of the nearest neighbors. In contrast, we leverage shared memory parallelism and recent advances in similarity joins to solve the problem exactly. Our method considers all pairs of potential neighbors but quickly filters pairs that could not be a part of the nearest neighbor graph, based on similarity upper bound estimates. The filtering is data dependent and not easily predicted, which poses load balance challenges in parallel execution. We evaluated our methods on several real-world datasets and found they work up to two orders of magnitude faster than existing methods, display linear strong scaling characteristics, and incur less than 1% load imbalance during filtering.

One-Nearest-Neighbor Search is All You Need for Minimax Optimal Regression and Classification

“…We remark, however, that they typically involve onerous and rather complex preprocessing steps, which may not be suitable for a large-scale data. Approximate NN (ANN) search algorithms (Indyk & Motwani, 1998;Slaney & Casey, 2008;Har-Peled et al, 2012) are yet another practical solution to reduce the query complexity, but ANN-search-based rules such as (Alabduljalil et al, 2013;Anastasiu & Karypis, 2019) hardly have any statistical guarantee (Dasgupta & Kpotufe, 2019) with few exception (Gottlieb et al, 2014;Efremenko et al, 2020). Gottlieb et al (2014) proposed an ANN-based classifier for general doubling spaces with generalization bounds.…”

Rates of Convergence for Large-scale Nearest Neighbor Classification

“…As the definition of "large-scale" evolves, it would be good to know if the divide-andconquer scheme described above may be used in conjunction to these methods. There are related methods such as those that rely on efficient kNNs or approximate nearest neighbor (ANN) methods [2,1,26,39]. However, little theoretical understanding in terms of the classification accuracy of these approximate methods has been obtained (with rare exceptions like [21].)…”