Automatic Design of Noncryptographic Hash Functions Using Genetic Programming

Abstract

Noncryptographic hash functions have an immense number of important practical applications owing to their powerful search properties. However, those properties critically depend on good designs: Inappropriately chosen hash functions are a very common source of performance losses. On the other hand, hash functions are difficult to design: They are extremely nonlinear and counterintuitive, and relationships between the variables are often intricate and obscure. In this work, we demonstrate the utility of genetic programming (GP) and avalanche effect to automatically generate noncryptographic hashes that can compete with state-of-the-art hash functions. We describe the design and implementation of our system, called GP-hash, and its fitness function, based on avalanche properties. Also, we experimentally identify good terminal and function sets and parameters for this task, providing interesting information for future research in this topic. Using GP-hash, we were able to generate two different families of noncryptographic hashes. These hashes are able to compete with a selection of the most important functions of the hashing literature, most of them widely used in the industry and created by world-class hashing experts with years of experience.

Synthetic and real data sets for benchmarking non-cryptographic hash functions

“…When using these data, NCHFs can be applied with different padding schemes, load factors, and table sizes [2] , but we recommend always using them equally for sake of consistency when comparing performances. In the case of using these data sets for generating new NCHFs with machine learning techniques, then they are split into training and test datasets in a random 70%–30% proportion, a minimum 5 k-fold is recommended for cross validation.…”

An Enhanced Non-Cryptographic Hash Function

“…Independent of the inputs of a hash functions, they are optimized to work very well in different scenarios. The criteria for optimization is based on the assertion that, with hash functions, there should be equal probability with the generation of each output and a little change in inputs, must result in a huge change in outputs [17].…”