Parallel mining of time-faded heavy hitters

Cafaro, Massimo; Pulimeno, Marco; Epicoco, Italo

doi:10.1016/j.eswa.2017.11.021

Cited by 12 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accelerator based algorithms include Govindaraju et al [42], Erra and Frola [43] and Cafaro et al [41,44]. Pulimeno et al [45] present a message-passing based version of the CHHs algorithm [26]; a parallel message-passing based version of [32] is presented in [46].…”

Section: Related Workmentioning

confidence: 99%

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

et al. 2020

Self Cite

View full text Add to dashboard Cite

A stream can be thought of as a very large set of data, sometimes even infinite, which arrives sequentially and must be processed without the possibility of being stored. In fact, the memory available to the algorithm is limited and it is not possible to store the whole stream of data which is instead scanned upon arrival and summarized through a succinct data structure in order to maintain only the information of interest. Two of the main tasks related to data stream processing are frequency estimation and heavy hitter detection. The frequency estimation problem requires estimating the frequency of each item, that is the number of times or the weight with which each appears in the stream, while heavy hitter detection means the detection of all those items with a frequency higher than a fixed threshold. In this work we design and analyze ACMSS, an algorithm for frequency estimation and heavy hitter detection, and compare it against the state of the art ASketch algorithm. We show that, given the same budgeted amount of memory, for the task of frequency estimation our algorithm outperforms ASketch with regard to accuracy. Furthermore, we show that, under the assumptions stated by its authors, ASketch may not be able to report all of the heavy hitters whilst ACMSS will provide with high probability the full list of heavy hitters.

show abstract

Section: Related Workmentioning

confidence: 99%

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Mining time faded frequent items has been investigated in [14], [7], [49], [4]. A parallel message-passing based algorithm has been recently proposed in [6]. Regarding the Correlated Heavy Hitters Problem (CHHs), an algorithm based on the nested application of Frequent has been recently presented in [36].…”

Section: Related Workmentioning

confidence: 99%

Mining frequent items in unstructured P2P networks

Cafaro

Epicoco

Pulimeno

2019

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

Large scale decentralized systems, such as P2P, sensor or IoT device networks are becoming increasingly common, and require robust protocols to address the challenges posed by the distribution of data and the large number of peers belonging to the network. In this paper, we deal with the problem of mining frequent items in unstructured P2P networks. This problem, of practical importance, has many useful applications. We design P2PSS, a fully decentralized, gossip-based protocol for frequent items discovery, leveraging the Space-Saving algorithm. We formally prove the correctness and theoretical error bound. Extensive experimental results clearly show that P2PSS provides very good accuracy and scalability, also in the presence of highly dynamic P2P networks with churning. To the best of our knowledge, this is the first gossip-based distributed algorithm providing strong theoretical guarantees for both the Approximate Frequent Items Problem in Unstructured P2P Networks and for the frequency estimation of discovered frequent items.

show abstract

“…Finally, with regard to the running time, FDCMSS is the fastest owing to its data structures and update strategy. A parallel message-passing based algorithm has been recently proposed in [4].…”

Section: Related Workmentioning

confidence: 99%

“…Before proceeding with our analysis, we need to recall the results by Jelasity et al in [24], related to the convergence factor of the uniform gossiping protocol, and the main properties exhibited by the algorithm FDCMSS and by the procedure for merging FDCMSS sketches [4], that we use in Algorithm 4.…”

Section: Correctnessmentioning

confidence: 99%

“…P2PTFHH follows the same structure of the gossip-based averaging protocol by Jelasity et al, where, instead of single values, two nodes communicating provide two sketches to be merged and averaged. The merge operation refers to the procedure introduced by Cafaro et al in [4], where the authors proved that the merge procedure of two augmented sketch data structures preserves all of the properties of the sketch, including the fact that an FDCMSS sketch is 1-norm equivalent to a classical Count-Min sketch for the same input stream, i.e. the value of a cell in the Count-Min sketch is equal to the sum of the Space-Saving counters in the same cell of the FDCMSS sketch.…”

Section: Fdcmss Algorithm and Merging Of Augmented Sketchesmentioning

confidence: 99%

See 1 more Smart Citation

Distributed mining of time--faded heavy hitters

Pulimeno¹,

Epicoco²,

Cafaro³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

We present P2PTFHH (Peer-to-Peer Time-Faded Heavy Hitters) which, to the best of our knowledge, is the first distributed algorithm for mining time-faded heavy hitters on unstructured P2P networks. P2PTFHH is based on the FDCMSS (Forward Decay Count-Min Space-Saving) sequential algorithm, and efficiently exploits an averaging gossip protocol, by merging in each interaction the involved peers' underlying data structures. We formally prove the convergence and correctness properties of our distributed algorithm and show that it is fast and simple to implement. Extensive experimental results confirm that P2PTFHH retains the extreme accuracy and error bound provided by FDCMSS whilst showing excellent scalability. Our contributions are three-fold: (i) we prove that the averaging gossip protocol can be used jointly with our augmented sketch data structure for mining time-faded heavy hitters;(ii) we prove the error bounds on frequency estimation; (iii) we experimentally prove that P2PTFHH is extremely accurate and fast, allowing near real time processing of large datasets.

show abstract

Parallel mining of time-faded heavy hitters

Cited by 12 publications

References 33 publications

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

On Frequency Estimation and Detection of Heavy Hitters in Data Streams

Mining frequent items in unstructured P2P networks

Distributed mining of time--faded heavy hitters

Contact Info

Product

Resources

About