Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Luo, Lailong; Guo, Deke; Richard, T. B.; Rottenstreich, Ori; Xue-shan, Luo

doi:10.1109/comst.2018.2889329

Cited by 178 publications

(104 citation statements)

References 153 publications

(681 reference statements)

Supporting

Mentioning

103

Contrasting

Unclassified

Order By: Relevance

“…Top flow keys: When reporting the identities of the most influential flows, we cannot leverage data structures such as heap or priority queue to store the flow identities in the data plane. Instead, we leverage a packet mirroring feature in the switch with a Bloom filter [30,31,32]. Once the estimated statistic for a flow exceeds some threshold, the switch duplicates this packet and reports the copy to the controller.…”

Section: Methodsmentioning

confidence: 99%

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

Self Cite

View full text Add to dashboard Cite

Network performance problems are notoriously difficult to diagnose. Prior profiling systems collect performance statistics by keeping information about each network flow, but maintaining per-flow state is not scalable on resource-constrained NIC and switch hardware. Instead, we propose sketch-based performance monitoring using memory that is sublinear in the number of flows. Existing sketches estimate flow monitoring metrics based on flow sizes. In contrast, performance monitoring typically requires combining information across pairs of packets, such as matching a data packet with its acknowledgment to compute a round-trip time. We define a new class of lean algorithms that use memory sublinear in both the size of input data and the number of flows. We then introduce lean algorithms for a set of important statistics, such as identifying flows with high latency, loss, out-of-order, or retransmitted packets. We implement prototypes of our lean algorithms on a commodity programmable switch using the P4 language. Our experiments show that lean algorithms detect ∼82% of top 100 problematic flows among real-world packet traces using just 40KB memory.

show abstract

Section: Methodsmentioning

confidence: 99%

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Liu¹,

Zhou²,

Rottenstreich³

et al. 2020

Symposium on Algorithmic Principles of Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…We focus on the following network measurement tasks, and more tasks can be found in [7], [44]- [47].…”

Section: B Generic Methods For Measurementsmentioning

confidence: 99%

Adaptive Measurements Using One Elastic Sketch

Yang

Jiang

Liu

et al. 2019

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

When network is undergoing problems such as congestion, scan attack, DDoS attack, etc, measurements are much more important than usual. In this case, traffic characteristics including available bandwidth, packet rate, and flow size distribution vary drastically, significantly degrading the performance of measurements. To address this issue, we propose the Elastic sketch. It is adaptive to currently traffic characteristics. Besides, it is generic to measurement tasks and platforms. We implement the Elastic sketch on six platforms: P4, FPGA, GPU, CPU, multi-core CPU, and OVS, to process six typical measurement tasks. Experimental results and theoretical analysis show that the Elastic sketch can adapt well to traffic characteristics. Compared to the state-of-the-art, the Elastic sketch achieves 44.6 ∼ 45.2 times faster speed and 2.0 ∼ 273.7 smaller error rate.

show abstract

“…Compared to other data structures, a Bloom filter has significant advantages in terms of space and time because the storage space and query time in the filter are constants [24,25]. In addition, the hash functions have no relationship with each other and can be implemented in parallel by hardware.…”

Section: Time-distance Bloom Filter Algorithmmentioning

confidence: 99%

A Temporal and Spatial Data Redundancy Processing Algorithm for RFID Surveillance Data

Wang

Cao

Zhang

et al. 2020

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The Radio Frequency Identification (RFID) data acquisition rate used for monitoring is so high that the RFID data stream contains a large amount of redundant data, which increases the system overhead. To balance the accuracy and real-time performance of monitoring, it is necessary to filter out redundant RFID data. We propose an algorithm called Time-Distance Bloom Filter (TDBF) that takes into account the read time and read distance of RFID tags, which greatly reduces data redundancy. In addition, we have proposed a measurement of the filter performance evaluation indicators. In experiments, we found that the performance score of the TDBF algorithm was 5.2, while the Time Bloom Filter (TBF) score was only 0.03, which indicates that the TDBF algorithm can achieve a lower false negative rate, lower false positive rate, and higher data compression rate. Furthermore, in a dynamic scenario, the TDBF algorithm can filter out valid data according to the actual scenario requirements.

show abstract

Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Cited by 178 publications

References 153 publications

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Memory-Efficient Performance Monitoring on Programmable Switches with Lean Algorithms

Adaptive Measurements Using One Elastic Sketch

A Temporal and Spatial Data Redundancy Processing Algorithm for RFID Surveillance Data

Contact Info

Product

Resources

About