Fast and accurate mining of correlated heavy hitters

Epicoco, Italo; Cafaro, Massimo; Pulimeno, Marco

doi:10.1007/s10618-017-0526-x

Cited by 13 publications

(11 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent results on finding frequent items in streams include models where items that are more recent in the stream have higher weight [25,3], as well as many applications, e.g., finding frequent elements in two-dimensional data streams [20,8].…”

Section: Previous and Related Workmentioning

confidence: 99%

Efficient Algorithm for Deterministic Search of Hot Elements

Kowalski¹,

Pająk²

2022

Preprint

View full text Add to dashboard Cite

When facing a very large stream of data, it is often desirable to extract most important statistics online in a short time and using small memory. For example, one may want to quickly find the most influential users generating posts online or check if the stream contains many identical elements. In this paper, we study streams containing insertions and deletions of elements from a possibly large set N of size |N | = n, that are being processed by online deterministic algorithms. At any point in the stream the algorithm may be queried to output elements of certain frequency in the already processed stream. More precisely, the most frequent elements in the stream so far. The output is considered correct if the returned elements it contains all elements with frequency greater than a given parameter ϕ and no element with frequency smaller than ϕ − ǫ. We present an efficient online deterministic algorithm for solving this problem using O(min(n, polylog (n) ǫ )) memory and O( polylog (n)) time per processing and outputting an element. It is the first such algorithm as the previous algorithms were either randomized, or processed elements in substantially larger time Ω(min(n, log n ǫ )), or handled only insertions and required two passes over the stream (i.e., were not truly online). Our solution is almost-optimally scalable (with only a polylogarithmic overhead) and does not require randomness or scanning twice through the stream. We complement the algorithm analysis with a lower bound Ω(min(n, 1 ǫ )) on required memory.

show abstract

Section: Previous and Related Workmentioning

confidence: 99%

Efficient Algorithm for Deterministic Search of Hot Elements

Kowalski¹,

Pająk²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Pairwise independent hash functions are used to map stream's items to corresponding cells in the sketch. Sketch-based algorithms include COUNTSKETCH by Charikar et al [5], GROUP TEST [22] and COUNT-MIN [23] by Cormode and Muthukrishnan, HCOUNT [24] by Jin et al and CMSS [11] by Cafaro et al Algorithms for Correlated Heavy Hitters (CHHs) have been recently proposed by Lahiri et al [25] and by Epicoco et al [26] in which a fast and more accurate algorithm for mining CHHs is presented.…”

Section: Related Workmentioning

confidence: 99%

“…Accelerator based algorithms include Govindaraju et al [42], Erra and Frola [43] and Cafaro et al [41,44]. Pulimeno et al [45] present a message-passing based version of the CHHs algorithm [26]; a parallel message-passing based version of [32] is presented in [46].…”

Section: Related Workmentioning

confidence: 99%

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

et al. 2020

Self Cite

View full text Add to dashboard Cite

A stream can be thought of as a very large set of data, sometimes even infinite, which arrives sequentially and must be processed without the possibility of being stored. In fact, the memory available to the algorithm is limited and it is not possible to store the whole stream of data which is instead scanned upon arrival and summarized through a succinct data structure in order to maintain only the information of interest. Two of the main tasks related to data stream processing are frequency estimation and heavy hitter detection. The frequency estimation problem requires estimating the frequency of each item, that is the number of times or the weight with which each appears in the stream, while heavy hitter detection means the detection of all those items with a frequency higher than a fixed threshold. In this work we design and analyze ACMSS, an algorithm for frequency estimation and heavy hitter detection, and compare it against the state of the art ASketch algorithm. We show that, given the same budgeted amount of memory, for the task of frequency estimation our algorithm outperforms ASketch with regard to accuracy. Furthermore, we show that, under the assumptions stated by its authors, ASketch may not be able to report all of the heavy hitters whilst ACMSS will provide with high probability the full list of heavy hitters.

show abstract

“…In [24], a faster and more accurate algorithm for mining CHHs is proposed. The Cascading Space-Saving Correlated Heavy Hitters (CSSCHH) algorithm exploits the basic ideas of Space-Saving, combining two summaries for tracking the primary item frequencies and the tuple frequencies.…”

Section: Related Workmentioning

confidence: 99%

Mining frequent items in unstructured P2P networks

Cafaro

Epicoco

Pulimeno

2019

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

Large scale decentralized systems, such as P2P, sensor or IoT device networks are becoming increasingly common, and require robust protocols to address the challenges posed by the distribution of data and the large number of peers belonging to the network. In this paper, we deal with the problem of mining frequent items in unstructured P2P networks. This problem, of practical importance, has many useful applications. We design P2PSS, a fully decentralized, gossip-based protocol for frequent items discovery, leveraging the Space-Saving algorithm. We formally prove the correctness and theoretical error bound. Extensive experimental results clearly show that P2PSS provides very good accuracy and scalability, also in the presence of highly dynamic P2P networks with churning. To the best of our knowledge, this is the first gossip-based distributed algorithm providing strong theoretical guarantees for both the Approximate Frequent Items Problem in Unstructured P2P Networks and for the frequency estimation of discovered frequent items.

show abstract

Fast and accurate mining of correlated heavy hitters

Cited by 13 publications

References 43 publications

Efficient Algorithm for Deterministic Search of Hot Elements

Efficient Algorithm for Deterministic Search of Hot Elements

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

Mining frequent items in unstructured P2P networks

Contact Info

Product

Resources

About