Approximately Minwise Independence with Twisted Tabulation

Dahlgaard, Søren; Thorup, Mikkel

doi:10.1007/978-3-319-08404-6_12

Cited by 11 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Minwise independence Concerning minwise hashing, Dahlgaard and Thorup [15] have proved that twisted tabulation yields the following strengthening of Theorem 4 for simple tabulation.…”

Section: Theorem 5 ([42]) Choose a Random C-character Twisted Tabulatmentioning

confidence: 96%

“…In this paper we will survey recent results from [12,13,14,15,41,42,50] showing how simple realistic hashing schemes based on tabulation provide unexpectedly strong guarantees for many popular randomized algorithms, e.g., linear probing, Cuckoo hashing, min-wise independence, treaps, planar partitions, powerof-two-choices, Chernoff-style concentration bounds, and even high independence. The survey is from a users perspective, explaining how these tabulation schemes can be applied.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast and powerful hashing using tabulation

Thorup

2017

Commun. ACM

Self Cite

View full text Add to dashboard Cite

show abstract

“…Minwise independence Concerning minwise hashing, Dahlgaard and Thorup [15] have proved that twisted tabulation yields the following strengthening of Theorem 4 for simple tabulation.…”

Section: Theorem 5 ([42]) Choose a Random C-character Twisted Tabulatmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

Fast and powerful hashing using tabulation

Thorup

2017

Commun. ACM

Self Cite

View full text Add to dashboard Cite

show abstract

“…Min-wise independence unfortunately does not follow as a consequence of strong universality, though you can approximate it with sufficiently high degrees of k-independence [24] or by using a twisted variant of tabulation hashing [10].…”

Section: Hashing and Permutationsmentioning

confidence: 99%

On minimizers and convolutional filters: a partial justification for the unreasonable effectiveness of CNNs in categorical sequence analysis

Yu¹

2021

Preprint

View full text Add to dashboard Cite

Minimizers and convolutional neural networks (CNNs) are two quite distinct popular techniques that have both been employed to analyze biological sequences. At face value, the methods seem entirely dissimilar. Minimizers use min-wise hashing on a rolling window to extract a single important k-mer feature per window. CNNs start with a wide array of randomly initialized convolutional filters, paired with a pooling operation, and then multiple additional neural layers to learn both the filters themselves and how those filters can be used to classify the sequence. In this manuscript, I demonstrate through a careful mathematical analysis of hash function properties that there are deep theoretical connections between minimizers and convolutional filters-in short, for sequences over a categorical alphabet, random Gaussian initialization of convolutional filters with max-pooling is equivalent to choosing minimizers from a random hash function biased towards more distinct k-mers. This provides a partial explanation for the unreasonable effectiveness of CNNs in categorical sequence analysis.

show abstract

“…Generating m hash functions simply requires the generation of m sets of lookup tables. A more recent variation of simple tabulation hashing, named twisted tabulation hashing, is reported to be even better for minwise hashing [15], although it will not be described because it is not used in our current implementation. Table 1 shows simple tabulation minwise hashes for the bigram-tokenised strings 'john' and 'jon'.…”

Section: 10101011 01110101 01010011mentioning

confidence: 99%

Secure pseudonymisation for privacy-preserving probabilistic record linkage

Smith

2017

Journal of Information Security and Applications

View full text Add to dashboard Cite

Record linkage is becoming an increasingly important tool in many areas of research -particularly medical research, where the relevant data often reside in more than one location. In the absence of a reliable and unique identifier probabilistic approaches to linkage are often employed. This linkage generally exploits the information contained in the fields that are common to a record pair. In classical record linkage the values in common fields are simply compared for equality. As values might contain typographical (or other) errors the performance of classical record linkage can often be significantly improved if similarities between value pairs are also exploited.In applications where the data used for matching must be kept private the raw values are replaced by pseudonyms. For better linkage performance these pseudonyms should also convey information regarding similarities. Existing approaches are often based on Bloom filters, yet these are susceptible to attack. Secure schemes based on Bloom filters inevitably involve additional security measures. Here we introduce a new scheme that produces pseudonyms that are far more secure than Bloom filters. It can be used a drop-in replacement for many schemes that use Bloom filters. The new scheme allows similarity scores to be estimated from pairs of pseudonyms with negligible bias and with known variance for a given similarity score.©

show abstract

Approximately Minwise Independence with Twisted Tabulation

Cited by 11 publications

References 22 publications

Fast and powerful hashing using tabulation

Fast and powerful hashing using tabulation

On minimizers and convolutional filters: a partial justification for the unreasonable effectiveness of CNNs in categorical sequence analysis

Secure pseudonymisation for privacy-preserving probabilistic record linkage

Contact Info

Product

Resources

About