Stream mining of frequent sets with limited memory

Abstract

With advances in technology, streams of data are produced in many applications. Efficient techniques for extracting implicit, previously unknown, and potentially useful information (e.g., in the form frequent sets) from data streams are in demand. Many existing stream mining algorithms capture important streaming data and assume that the captured data can fit into main memory. However, problem arose when the available memory is so limited that such an assumption does not hold. In this paper, we propose a novel data structure called DSTable to capture important data from the streams onto the disk. The DSTable can be easily maintained; it can be applicable for mining frequent sets from datasets, especially in limited memory environments.

Effectively and Efficiently Mining Frequent Patterns from Dense Graph Streams on Disk

“…Hence, algorithms for mining frequent patterns with limited memory are still in demand, so as to deal with the case of streams generated by graph data sources. For instance, Cameron et al 7 studied this topic and proposed an algorithm that works well for sparse data streams in limited memory space. In contrast, the mining algorithms we propose in the current paper are designed to mine dense graph streams in limited memory space.…”

An Algorithm for Mining Frequent Itemsets from Library Big Data

“…While many algorithms [7][8][9][10][11][12][13] have significantly increased their performance, they are still not fast enough for dealing with large datasets. This situation gave rise to the development of parallel algorithms.…”