An Efficient Algorithm for Frequent Itemset Mining on Data Streams

Abstract

In order to mining frequent itemsets on data stream efficiently, a new approach was proposed in this paper. The memory efficient and accurate one-pass algorithm divides all the frequent itemsets into frequent equivalence classes and prune all the redundant itemsets except for those represent the GLB(Greatest Lower Bound) and LUB(Least Upper Bound) of the frequent equivalence class and the number of GLB and LUB is much less than that of frequent itemsets. In order to maintain these equivalence classes, A compact data structure, the frequent itemset enumerate tree (FIET) was proposed in the paper. The detailed experimental evaluation on synthetic and real datasets shows that the algorithm is very accurate in practice and requires significantly lower memory than Jin and Agrawal's one pass algorithm.

Mining Regular Patterns in Transactional Databases

“…On the other hand, the best case is when every transaction is the same, with the number of tail-nodes being one. Moreover, keeping the tid information in a tree structure has also been found in literature discussing the efficient mining of frequent patterns [5]- [7]. To a certain extent, some of those approaches additionally maintain a support count and/or the tid information [6], [7] in each tree node.…”

“…Mining patterns that appear frequently in transactional databases [1], [2], [7], [14] has been widely studied for over a decade. The rationale behind mining frequent patterns is that only patterns occurring at a high frequency are of interest to users.…”

Sliding window-based frequent pattern mining over data streams

“…Most studies about finding frequent patterns in a data stream are based on the landmark window model [25,41,44] or the sliding window model [3,6,22,27,29,24]. The first attempt to mine frequent patterns over the entire history of streaming data was proposed by Manku and Motwani [28].…”

“…They developed two single-pass algorithms, Sticky-Sampling and Lossy Counting, both of which are based on the anti-monotone 1 property; these algorithms provide approximate results with an error bound. Zhi-Jun et al [44] used a lattice structure, referred to as a frequent enumerate tree, which is divided into several equivalent classes of stored patterns with the same transaction-ids in a single class. Frequent patterns are divided into equivalent classes, and only those frequent patterns that represent the two borders of each class are maintained; other frequent patterns are pruned.…”

EclatDS: An efficient sliding window based frequent pattern mining method for data streams

“…Most of them are based on the landmark window model [17,29,34] and the sliding window model [14,16,18,25]. DSM-FI [17] is a landmark based algorithm.…”