Independent counter estimation buckets

Einziger, Gil; Fellman, Benny; Kassner, Yaron

doi:10.1109/infocom.2015.7218646

Cited by 21 publications

(8 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other methods such as Counter Braids [42] reduce the meta data size but require a long decode operation and are therefore not applicable to caching. Another approach is to compress the counter representation itself [52,34,54,20]. In TinyLFU we manage to represent the histogram using short counters, and thus these methods do not help us.…”

Section: Approximate Counting Architecturesmentioning

confidence: 99%

TinyLFU: A Highly Efficient Cache Admission Policy

Einziger

Friedman

2014

2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

110

View full text Add to dashboard Cite

This paper proposes to use a frequency based cache admission policy in order to boost the effectiveness of caches subject to skewed access distributions. Given a newly accessed item and an eviction candidate from the cache, our scheme decides, based on the recent access history, whether it is worth admitting the new item into the cache at the expense of the eviction candidate.Realizing this concept is enabled through a novel approximate LFU structure called TinyLFU, which maintains an approximate representation of the access frequency of a large sample of recently accessed items. TinyLFU is very compact and light-weight as it builds upon Bloom filter theory.We study the properties of TinyLFU through simulations of both synthetic workloads as well as multiple real traces from several sources. These simulations demonstrate the performance boost obtained by enhancing various replacement policies with the TinyLFU eviction policy. Also, a new combined replacement and eviction policy scheme nicknamed W-TinyLFU is presented. W-TinyLFU is demonstrated to obtain equal or better hit-ratios than other state of the art replacement policies on these traces. It is the only scheme to obtain such good results on all traces.

show abstract

Section: Approximate Counting Architecturesmentioning

confidence: 99%

TinyLFU: A Highly Efficient Cache Admission Policy

Einziger

Friedman

2014

2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing

Self Cite

110

View full text Add to dashboard Cite

show abstract

“…By separating flows into different buckets with different scales, the Independent Counter Estimation Buckets (ICE-BUCKETS) [34] algorithm allows to further improve the upper bound on the relative counting error of up to 57 times over real traffic traces.…”

Section: Related Workmentioning

confidence: 99%

A Probabilistic Counting Framework for Distributed Measurements

2019

View full text Add to dashboard Cite

The technological maturity attained by general purpose processors and network interface cards makes today's commodity PCs viable and high performing alternatives to specialized hardware for deploying network devices, such as switches, routers, and generic middleboxes. In addition, the flexibility of the software solution seems to be perfectly in line with the emerging trend towards the data-plane programming abstractions brought by recent proposals such as Openflow and the P4 language. However, if programming abstractions provide the way elementary instructions (primitives) are combined together, the development of such processing primitives is left to the network programmer. Although the type of such functions is strongly domain specific, we can safely assume that the counting primitive is easily required in a great deal of practical contexts. This paper presents a counting framework based on probabilistic sketches and LogLog counters for estimating the cardinality of large multi-sets of data. The proposed data structure is designed to be fast and compact for ready use in the on-line chain of processing of network devices running at multi-gigabit speeds. The complete implementation is provided within the probabilistic data structures (pds) library which has been designed, developed, experimentally assessed, and released as open-source for free download. Although the paper specifically presents two possible use-cases, the pds library can be used in rather general scenarios, even outside the networking domain. INDEX TERMS Approximate counting, loglog counter, multi-set cardinality, network applications, reversible sketches.

show abstract

“…Estimators reduce the size of counters using probabilistic techniques [9]- [11]. This enables maintaining one counter per flow in SRAM at the cost of reduced accuracy.…”

Section: A Backgroundmentioning

confidence: 99%

“…These methods trade precision for space and allow more counters to be contained in SRAM. This idea was first introduced by Approximate Counting [35] and was adapted to networking devices [9]- [11], [36]. The downside of estimators is that they require storing a flow-to-counter mapping for every flow, a requirement that has many overheads.…”

Section: Network Monitoring Architecturesmentioning

confidence: 99%

Randomized admission policy for efficient top-k and frequency estimation

Basat

Einziger

Friedman

et al. 2017

IEEE INFOCOM 2017 - IEEE Conference on Computer Communications

Self Cite

View full text Add to dashboard Cite

Abstract-Network management protocols often require timely and meaningful insight about per flow network traffic. This paper introduces Randomized Admission Policy (RAP) -a novel algorithm for the frequency and top-k estimation problems, which are fundamental in network monitoring. We demonstrate space reductions compared to the alternatives by a factor of up to 32 on real packet traces and up to 128 on heavy-tailed workloads. For top-k identification, RAP exhibits memory savings by a factor of between 4 and 64 depending on the workloads' skewness. These empirical results are backed by formal analysis, indicating the asymptotic space improvement of our probabilistic admission approach. Additionally, we present d-Way RAP, a hardware friendly variant of RAP that empirically maintains its space and accuracy benefits.

show abstract

Independent counter estimation buckets

Cited by 21 publications

References 21 publications

TinyLFU: A Highly Efficient Cache Admission Policy

TinyLFU: A Highly Efficient Cache Admission Policy

A Probabilistic Counting Framework for Distributed Measurements

Randomized admission policy for efficient top-k and frequency estimation

Contact Info

Product

Resources

About