Theory and Network Applications of Dynamic Bloom Filters

Abstract

A bloom filter is a simple, space-efficient, randomized data structure for concisely representing a static data set, in order to support approximate membership queries. It has great potential for distributed applications where systems need to share information about what resources they have. The space efficiency is achieved at the cost of a small probability of false positive in membership queries. However, for many applications the space savings and short locating time consistently outweigh this drawback. In this paper, we introduce dynamic bloom filters (DBF) to support concise representation and approximate membership queries of dynamic sets, and study the false positive probability and union algebra operations. We prove that DBF can control the false positive probability at a low level by adjusting the number of standard bloom filters used according to the actual size of current dynamic set. The space complexity is also acceptable if the actual size of dynamic set does not deviate too much from the predefined threshold. Furthermore, we present multidimension dynamic bloom filters (MDDBF) to support concise representation and approximate membership queries of dynamic sets in multiple attribute dimensions, and study the false positive probability and union algebra operations through mathematic analysis and experimentation. We also explore the optimization approach and three network applications of bloom filters, namely bloom joins, informed search, and global index implementation. Our simulation shows that informed search based on bloom filters can obtain higher recall and success rate of query than the blind search protocol.

Tối Ưu Hóa Join Đệ Quy Trên Tập Dữ Liệu Lớn Trong Môi Trường Spark

“… Không gian lưu trữ hiệu quả (space eciency): Kích thước của bộ lọc là cố định, không phụ thuộc vào số lượng phần tử n nhưng nó có mối liên hệ giữa mảng bit m và dương tính giả theo xác suất [40]:…”

Location privacy without mutual trust: The spatial Bloom filter

“…These variants allow to implement a delete operation on the filter without recreating it from scratch. Compressed Bloom Filters are designed to control precisely the memory consumption through a parameter z [29], while Dynamic Bloom Filters allow to create new filters in real-time as the originating set varies [17]. However, none of these variants is suitable to store location data as multiple sets of areas nor to efficiently perform location queries upon them.…”

“…The former case is always true, for the properties of Definition 3, as explained in Section 4. In the latter case, the false positive probability p i for each area i is determined by Paul according to (17) during filter creation. It is therefore Paul himself who decides the correctness bounds of the protocol.…”

Improving distributed join efficiency with extended bloom filter operations

“…While on the surface there seems to be a similarity between block-partitioned bloom filters and dynamic bloom filters [12], the approaches are actually different. The purpose of dynamic bloom filters is to grow, as more and more objects are added.…”

“…Another approach to multiset representation, the space-code bloom filter is described in [13]. Finally, [12] presents dynamic bloom filters, which dynamically adapt their size to the number of inserted elements.…”