An efficient framework for parallel and continuous frequent item monitoring

Abstract

SUMMARYIn high-speed network monitoring, the ever-growing traffic calls for a high-performance solution for the computation of frequent items. The increasing number of cores in the current commodity multi-core processors opens up new opportunities in parallelization. In this paper, we present a novel precision integrated framework (PRIF) that exploits the great parallel capability of multi-cores to speed up the famous frequent algorithm. PRIF equally distributes the input data stream into sub-threads that use the optimized weighted frequent algorithm to track local frequent items. The items with frequency increments exceeding a predefined threshold are sent to a merging thread which is able to return the global continuous -deficient frequent items. The theoretical correctness and complexity analyses are presented. Experiments with real and synthetic traces confirm the theoretical analyses and demonstrate the excellent performance as well as the effects of parameters and data skewness. The results show that PRIF is able to provide continuous frequent items and near-linear speedup at the cost of greater memory use.

Mining frequent items in unstructured P2P networks

“…Regarding parallel algorithms, [10] (slightly improved in [5]) and [9] gorithms. Among the algorithms for shared-memory architectures we recall here a parallel version of Frequent [52], a parallel version of Lossy Counting [51], and parallel versions of Space-Saving [45] and [19]. Novel shared-memory parallel algorithms for frequent items were recently proposed in [47].…”

Parallel social network mining for interesting ‘following’ patterns

“…Several data mining algorithms and techniques have been proposed over the past few years for mining social networks such as the discovery of special events , detection of communities , subgraph mining , as well as discovery of popular friends , influential friends and strong friends . Given that we will reduce our social network analysis problem into a specific data mining problem of frequent pattern mining , we review some related works on frequent pattern mining in this section.…”

Parallel mining of time-faded heavy hitters

“…Relevant parallel algorithms include [6], [3] and [5] which are message-passing based parallel versions of the Frequent and Space Saving algorithms. Shared-memory algorithms have been designed as well, including a parallel version of Frequent [31], a parallel version of Lossy Counting [30], and parallel versions of Space Saving [28] [13].…”