Estimators also need shared values to grow together

Abstract

Network management applications require large numbers of counters in order to collect traffic characteristics for each network flow. However, these counters often barely fit into on-chip SRAM memories. Past papers have proposed using counter estimators instead, thus trading off counter precision for a lower number of bits. But these estimators do not achieve optimal estimation error, and cannot always scale to arbitrary counter values.In this paper, we introduce the CEDAR algorithm for decoupling the counter estimators from their estimation values, which are quantized into estimation levels and shared among many estimators. These decoupled and shared estimation values enable us to easily adjust them without needing to go through all the counters. We demonstrate how our CEDAR scheme achieves the min-max relative error, i.e., can guarantee the best possible relative error over the entire counter scale. We also explain how to use dynamic adaptive estimation values in order to support counter up-scaling and adjust the estimation error depending on the current maximal counter.Finally we implement CEDAR on FPGA and explain how it can run at line rate. We further analyze its performance and size requirements.

TinyLFU: A Highly Efficient Cache Admission Policy

“…Other methods such as Counter Braids [42] reduce the meta data size but require a long decode operation and are therefore not applicable to caching. Another approach is to compress the counter representation itself [52,34,54,20]. In TinyLFU we manage to represent the histogram using short counters, and thus these methods do not help us.…”

ICE Buckets: Improved Counter Estimation for Network Measurement

“…We used three traces from this dataset, CHI08 from 2008 [35], CHI14 from 2014 [36] and CHI15 from 2015 [37]. CHI08 was previously used to evaluate CEDAR in [31]. Trace (SJ13) was taken in 2013 from an Internet data collection monitor that is connected to a link between San Jose and Los Angeles, CA [38].…”

“…2) Optimality: An estimation function is considered optimal if it minimizes δ max given M , the desired counting capacity. To show that our estimation function is optimal, we rely on Theorems 3 and 4 in [31], stating that a function that satisfies the following recursive formula is optimal with δ max = δ.…”

A formal analysis of conservative update based approximate counting

“…However, CounterBraids suffers from very slow decode time. Alternatively, [16,25] tackle the space overhead problem of a single counter by introducing compressed counters that require fewer bits per counter but introduce an additional estimation error. These methods are complementary to the sketches discussed in this work, and can easily be deployed together.…”