Xor Filters

Abstract

The Bloom filter provides fast approximate set membership while using little memory. Engineers often use these filters to avoid slow operations such as disk or network accesses. As an alternative, a cuckoo filter may need less space than a Bloom filter and it is faster. Chazelle et al. proposed a generalization of the Bloom filter called the Bloomier filter. Dietzfelbinger and Pagh described a variation on the Bloomier filter that can answer approximate membership queries over immutable sets. It has never been tested empirically, to our knowledge. We review an efficient implementation of their approach, which we call the xor filter. We find that xor filters can be faster than Bloom and cuckoo filters while using less memory. We further show that a more compact version of xor filters (xor+) can use even less space than highly compact alternatives (e.g., Golomb-compressed sequences) while providing speeds competitive with Bloom filters.

Attacking the Privacy of Approximate Membership Check Filters by Positive Concentration

“…Instead a false negative occurs when a negative is returned for an element that is in S, so a positive should have been returned. These data structures are generally known as approximate membership check filters and many types have been proposed over the years, starting with the Bloom filter [5] and more recently, the quotient [7], [8], cuckoo [6], or xor [15] filters.…”

A Case for Partitioned Bloom Filters

“…So, the collision problem occurs in practice for BBFs. It should be emphasized that using blocking is the only way that Bloom filters can remain performance-wise competitive [14] with dictionary-based approaches (such as Cuckoo Filters [15], Morton Filters [16]), or Xor Filters [17]. Therefore, the scenario of a small Bloom filter (a block of a BBF) is important, even for scenarios with huge (on the whole) filters.…”

“…Partitioned Bloom filters may have several feature advantages, compared with the standard, but is it enough for them to be competitive with alternative approaches? A common view is that Bloom filters have been superseded by fingerprint-based mechanisms, such as Cuckoo [15] or Xor [17] filters. This is not necessarily the case: while that is true if the main concern is memory consumption and high accuracy, for moderate accuracy and when query time is important but memory less of a concern, Bloom filters, in blocked variants, remain the best [14].…”

“…We now summarize how the Bloom filter variants compare among themselves, and with some fingerprint-based approaches: the well known Cuckoo and Xor filters, and two recent state-of-the-art mechanisms, Ribbon filters [38] and binary fuse filters (BFF) [39]. Table 5 presents a feature-wise and qualitative comparison of filters (detailed quantitative results can be found elsewhere [17], [38], [39]). The columns, from left to right depict Bloom filters, in standard and partitioned variants, blocked Bloom filters using either standard or partitioned filters in blocks, and then fingerprint-based mechanisms: Cuckoo, Xor, Ribbon and binary fuse filters.…”

A blockchain-based infection tracing and notification system by non-fungible tokens

“…In particular, considering a device that generates 2000 addresses in a time window of 14 days (that roughly corresponds to a new address every 10 min), a kB XOR filter has to be transmitted from the device of the user tested positive, while considering contagions, MB of XOR filters should be downloaded on the device. Moreover, XOR filters are smaller and faster [32] than Bloom fiters used in [16] , [33] .…”