b-bit minwise hashing in practice

Abstract

Minwise hashing is a standard technique in the context of search for approximating set similarities. The recent work [26,32] demonstrated a potential use of b-bit minwise hashing [23,24] for efficient search and learning on massive, high-dimensional, binary data (which are typical for many applications in Web search and text mining). In this paper, we focus on a number of critical issues which must be addressed before one can apply b-bit minwise hashing to the volumes of data often used industrial applications.Minwise hashing requires an expensive preprocessing step that computes k (e.g., 500) minimal values after applying the corresponding permutations for each data vector. We developed a parallelization scheme using GPUs and observed that the preprocessing time can be reduced by a factor of 20 ∼ 80 and becomes substantially smaller than the data loading time. Reducing the preprocessing time is highly beneficial in practice, e.g., for duplicate Web page detection (where minwise hashing is a major step in the crawling pipeline) or for increasing the testing speed of online classifiers.Another critical issue is that for very large data sets it becomes impossible to store a (fully) random permutation matrix, due to its space requirements. Our paper is the first study to demonstrate that b-bit minwise hashing implemented using simple hash functions, e.g., the 2-universal (2U) and 4-universal (4U) hash families, can produce very similar learning results as using fully random permutations. Experiments on datasets of up to 200GB are presented.

In Defense of MinHash Over SimHash

“…SimHash generates a single bit output (only the signs) whereas MinHash generates an integer value. Recently proposed b-bit minwise hashing [22] provides simple strategy to generate an informative single bit output from MinHash, by using the parity of MinHash values:…”

“…For example, the paper on Conditional Random Sampling (CRS) [19] showed that random projections can be very inaccurate especially in binary data, for the task of inner product estimation (which is not the same as near neighbor search). A more recent paper [26] empirically demonstrated that b-bit minwise hashing [22] outperformed SimHash and spectral hashing [30].…”

On b-bit min-wise hashing for large-scale regression and classification with sparse data

“…In practice, one would not store the entire matrix of signs nor all the random permutations. In an implementation, hash functions [Carter and Wegman, 1979] would be used to create the matrix S deterministically, though it is beyond the scope of this paper to go into the details; see Li et al [2013] for more information and further computational improvements. With this approach, S would be created row-by-row, and only a single observation from X would need to be kept in memory at any one time.…”

“…The empirical performance of regression and classification procedures following b-bit min-wise hashing [Li et al, , 2013 is particularly impressive. Existing theory on b-bit min-wise hashing has focused on the variance and bias in the approximation of the kernel.…”

WordNet-based hybrid VSM for Document Classification

“…Many methods of document representation based on TF-IDF can construct Vector Space Model (VSM) of text corpus. Similarly, many methods of document representation exploit statistical term measures, such as BoS (Bag-of-Words) [3] and Minwise hashing [4]. For document representation, these methods are perceived as statistical methods of feature extraction.…”