A comparison of extended fingerprint hashing and locality sensitive hashing for binary audio fingerprints

Abstract

Hash tables have been proposed for the indexing of highdimensional binary vectors, specifically for the identification of media by fingerprints. In this paper we develop a new model to predict the performance of a hash-based method (Fingerprint Hashing) under varying levels of noise. We show that by the adjustment of two parameters, robustness to a higher level of noise is achieved. We extend Fingerprint Hashing to a multi-table range search (Extended Fingerprint Hashing) and show this approach also increases robustness to noise. We then show the relationship between Extended Fingerprint Hashing and Locality Sensitive Hashing and investigate design choices for dealing with higher noise levels. If index size must be held constant, the Extended Fingerprint Hash is a superior method. We also show that to achieve similar performance at a given level of noise a Locality Sensitive Hash requires nearly a six-fold increase in index size which is likely to be impractical for many applications.

HDKV: supporting efficient high‐dimensional similarity search in key‐value stores

“…Substantial work has been conducted on database scalability issues, and various mechanisms are proposed. Among them, distributed key‐value stores exhibit great efficiency to store and retrieve large‐volume of data, and have been widely used in various applications such as content‐based image processing , bio‐info data mining , spatial data processing . There are several open‐source implementations available, including HBase , Cassandra and Voldemort .…”

“…We design a strict order‐preserving hash function to map the nearby objects in high‐dimensional spaces onto adjacent keys in key‐value stores using locality sensitive hashing (LSH) . LSH has been proven to be very efficient for many applications, where a transformation from a higher‐dimensional space to a lower‐dimensional space is needed.…”

“…Locality sensitive hash algorithm is an effective approach that has been widely used in large‐scale data management systems such as graph processing , data mining and information retrieval . The basic idea of LSH is to group objects with similar attributes in the high‐dimensional space into the same bucket.…”