Morton filters

Abstract

Approximate set membership data structures (ASMDSs) are ubiquitous in computing. They trade a tunable, often small, error rate () for large space savings. The canonical ASMDS is the Bloom filter, which supports lookups and insertions but not deletions in its simplest form. Cuckoo filters (CFs), a recently proposed class of ASMDSs, add deletion support and often use fewer bits per item for equal. This work introduces the Morton filter (MF), a novel AS-MDS that introduces several key improvements to CFs. Like CFs, MFs support lookups, insertions, and deletions, but improve their respective throughputs by 1.3× to 2.5×, 0.9× to 15.5×, and 1.3× to 1.6×. MFs achieve these improvements by (1) introducing a compressed format that permits a logically sparse filter to be stored compactly in memory, (2) leveraging succinct embedded metadata to prune unnecessary memory accesses, and (3) heavily biasing insertions to use a single hash function. With these optimizations, lookups, insertions, and deletions often only require accessing a single hardware cache line from the filter. These improvements are not at a loss in space efficiency, as MFs typically use comparable to slightly less space than CFs for the same .

A Bloom filter variant for Blockchain

“…Comparison methods. We compare three Bloom filters, BF, 1 UFBF, 10 and MF 11 with our proposed BBF. BF is the standard Bloom filter.…”

“…Although UFBF develop a novel hash computation algorithm which can compute multiple hash functions in parallel with the use of SIMD instructions, its properties are achieved while also using comparable or fewer bits per item than a BBF for a target false positive rate. Morton filter (MF) 11 utilizes SIMD implementations of popcounts to implement fast reductions for determining bucket boundaries in the FSA, while it has not made full use of the SIMD instruction sets for lookups, insertions, and deletions.…”

Xor Filters

“…Table 5 shows that the query time of the reference quotient-filter implementation is several times the query time of competitive approaches like cuckoo or xor filters. We also consider Morton filters [8] with the reference implementation [7]. Morton filters answer one-at-a-time queries at half the speed of 8-bit xor filters, despite similar false-positive probabilities and memory usage.…”