Maximal metric margin partitioning for similarity search indexes

Fukagawa

IEICE Trans. Inf. & Syst.

et al. 2010

Self Cite

SUMMARYWhen developing an index for a similarity search in metric spaces, how to divide the space for effective search pruning is a fundamental issue. We present Maximal Metric Margin Partitioning (MMMP), a partitioning scheme for similarity search indexes. MMMP divides the data based on its distribution pattern, especially for the boundaries of clusters. A partitioning boundary created by MMMP is likely to be located in a sparse area between clusters. Moreover, the partitioning boundary is at maximum distances from the two cluster edges. We also present an indexing scheme, named the MMMP-Index, which uses MMMP and pivot filtering. The MMMP-Index can prune many objects that are not relevant to a query, and it reduces the query execution cost. Our experimental results show that MMMP effectively indexes clustered data and reduces the search cost. For clustered data in a vector space, the MMMP-Index reduces the computational cost to less than two thirds that of comparable schemes.

Section: Introductionmentioning

confidence: 89%

Margin-Based Pivot Selection for Similarity Search Indexes

Fukagawa

IEICE Trans. Inf. & Syst.

et al. 2010

Self Cite

“…All existing index-based studies focus on improving the index. While general metric indexes are designed to deal with any distances in the range queries [10,18,11], the indexes for similarity join queries assume fixed-distance range queries. eD-index [8] constructs an index that is an extension of D-index [7].…”

Section: Definition 3 (Modified Ball Partitioning) For a Metric Spacementioning

confidence: 99%

Finding the k-closest pairs in metric spaces

Proceedings of the 1st Workshop on New Trends in Similarity Search

Adachi

2011

Self Cite

We investigated the problem of reducing the cost of searching for the k closest pairs in metric spaces. In general, a k-closest pair search method initializes the upper bound distance between the k closest pairs as infinity and repeatedly updates the upper bound distance whenever it finds pairs of objects whose distances are shorter than that distance. Furthermore, it prunes dissimilar pairs whose distances are estimated as longer than the upper bound distance based on the distances from the pivot to objects and the triangle inequality. The cost of a k-closest pair query is smaller for a shorter upper bound distance and a sparser distribution of distances between the pivot and objects. We propose a new divide-and-conquer-based k-closest pair search method in metric spaces, called Adaptive Multi-Partitioning (AMP). AMP repeatedly divides and conquers objects from the sparser distance-distribution space and speeds up the convergence of the upper bound distance before partitioning the denser space. As a result, AMP can prune many dissimilar pairs compared with ordinary divide-and-conquer-based method. We compare our method with other partitioning method and show that AMP reduces distances computations.

“…OMNI-Family [15] chooses a set of pivots based on the minimum bounding region. MMMP [17] has a pivot selection based on the maximal margin, and it classifies dense regions. Moreover, some methods combine different pivot selection techniques [19], [27].…”

Section: Related Workmentioning

confidence: 99%

“…However, they don't consider the partitioning boundary of the pivot and the selected pivot may not be good for pruning. The clustering-based approach [13], [17] aims at selecting a pivot and its partitioning boundary based on the clusters in the dataset. Although it can select better pivots for pruning than the simple statistics-based approach, it doesn't take into consideration the tree balancing.…”

Section: Related Workmentioning

confidence: 99%

Optimal Pivot Selection Method Based on the Partition and the Pruning Effect for Metric Space Indexes

Fukagawa

IEICE Trans. Inf. & Syst.

et al. 2011

Self Cite

Hisashi KURASAWA †a) , Daiji FUKAGAWA † †b) , Atsuhiro TAKASU † † †c) , and Jun ADACHI † † †d) , Members SUMMARY This paper proposes a new method to reduce the cost of nearest neighbor searches in metric spaces. Many similarity search indexes recursively divide a region into subregions by using pivots, and construct a tree-structured index. Most of recently developed indexes focus on pruning objects and do not pay much attention to the tree balancing. As a result, indexes having imbalanced tree-structure may be constructed and the search cost is degraded. We propose a similarity search index called the Partitioning Capacity (PC) Tree. It selects the optimal pivot in terms of the PC that quantifies the balance of the regions partitioned by a pivot as well as the estimated effectiveness of the search pruning by the pivot. As a result, PCTree reduces the search cost for various data distributions. We experimentally compared PCTree with four indexes using synthetic data and five real datasets. The experimental results shows that the PCTree successfully reduces the search cost.