Independence of Tabulation-Based Hash Classes

Abstract

A tabulation-based hash function maps a key into d derived characters indexing random values in tables that are then combined with bitwise xor operations to give the hash. Thorup and Zhang [8] presented d-wise independent tabulation-based hash classes that use linear maps over finite fields to map a key, considered as a vector (a, b), to derived characters. We show that a variant where the derived characters are a + b · i for i = 0, . . . , q − 1 (using integer arithmetic) yielding (2d − 1)-wise independence. Our analysis is based on an algebraic property that characterizes k-wise independence of tabulation-based hashing schemes, and combines this characterization with a geometric argument. We also prove a nontrivial lower bound on the number of derived characters necessary for k-wise independence with our and related hash classes. * Supported in part by an NSERC USRA (in the summer of 2010) and a PURE award from the University of Calgary (in summer 2011). † Supported in part by NSERC family (where the range is [m] := {0, 1, . . . , m−1}), is constructed by choosing a prime p ≥ m and taking as the family the set of all mappings x → P (x) mod m, where P is a polynomial of degree k − 1 over Z p . Evaluating the polynomial is usually inefficient, especially for large values of k. Therefore, there has been interest in faster methods trading off arithmetic operations for a few lookups in tables filled with random values.

Simple Tabulation, Fast Expanders, Double Tabulation, and High Independence

“…If we unravel the recursion (to be presented in Section 4), for some D = o(c lg c ) and k = u Ω(1/c) , we get a function f : Proposition 2] or [8,Theorem 3] that f has the property that some output position character appears an odd number of times.…”

Twisted Tabulation Hashing

“…Our twisted tabulation follows the generic approach of derived characters to improve on simple tabulation [10,12,20,25]. Above we mentioned [20] as a theoretical option for high independence.…”

“…Above we mentioned [20] as a theoretical option for high independence. For moderate independence d over c character keys, [10,12,25] use Θ(cd) derived characters and character lookups. The cost is linear in d, just as with degree d polynomials [27], but the evaluation is faster using character lookups instead of multiplications.…”

“…The cost is linear in d, just as with degree d polynomials [27], but the evaluation is faster using character lookups instead of multiplications. The derived characters in [10] are randomized like our twist, but the later deterministic constructions from [12,25] are more efficient. Recall here that twisted tabulation does not give independence above 3.…”

“…Twisted tabulation is designed to provide the general distributional properties discussed above. With sub-logarithmic d, none of the schemes from [10,12,25] are known to provide any of these distributional guarantees. Of course, it is hard to know if these schemes have stronger hidden properties.…”