DSM-FI: an efficient algorithm for mining frequent itemsets in data streams

Li, Hua-Fu; Shan, Man-Kwan; Lee, Suh-Yin

doi:10.1007/s10115-007-0112-4

Cited by 64 publications

(45 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They proposed two singlepass algorithms, Sticky-Sampling and Lossy Counting, both of which are based on the anti-monotone property; these algorithms provide approximate results with an error bound. Li et al [15,16] proposed DSM-FI and DSM-MFI to mine frequent patterns using a landmark window. Each transaction is converted into k small transactions and inserted into an extended prefix-tree-based summary data structure called the item-suffix frequent itemset forest.…”

Section: Fig 2 Cantree In Lexicographic Ordermentioning

confidence: 99%

Efficient Single Pass Ordered Incremental Pattern Mining

Koh

Dobbie

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Since the introduction of FP-growth using FP-tree there has been a lot of research into extending its usage to data stream or incremental mining. Most incremental mining adapts the Apriori algorithm. However, we believe that using a tree based approach would increase performance as compared to the candidate generation and testing mechanism used in Apriori. Despite this, FP-tree still requires two scans through a dataset. In this paper we present a novel tree structure called Single Pass Ordered Tree, SPO-Tree, that captures information with a single scan for incremental mining. All items in a transaction are inserted/sorted based on their frequency. The tree is reorganized dynamically when necessary. SPO-Tree allows for easy maintenance in an incremental or data stream environment.

show abstract

Section: Fig 2 Cantree In Lexicographic Ordermentioning

confidence: 99%

Efficient Single Pass Ordered Incremental Pattern Mining

Koh

Dobbie

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…They also define the first singlepass algorithm for data streams based on the anti-monotonic property. Li et al [19] use an extended prefix-tree-based representation and a top-down frequent itemset discovery scheme. In [29], authors propose a regressionbased algorithm to find frequent itemsets in sliding windows.…”

Section: Related Workmentioning

confidence: 99%

Speed up gradual rule mining from stream data! A B-Tree and OWA-based approach

2009

View full text Add to dashboard Cite

Gradual rules allow users to be provided with rules describing the ordering correlations among attributes. Such a rule is for instance given by the higher the salary and the lower the number of cars, the higher the number of tourist travels. Previously intensively used in fuzzy command systems, these rules were manually provided to the system. More recently, they have received attention from the data mining community and methods have been defined to automatically extract and maintain gradual rules from numerical databases. However, no method has been shown to be able to handle data streams, as no method is scalable enough to manage the high rate which stream data arrive at. In this paper, we thus propose an original approach to mine data streams for gradual rules. Our method is based on B-Trees and OWA (Ordered Weighted Aggregation) operator in order to speed up the process. B-Trees are used to store already-known gradual rules in order to maintain the knowledge over time, while OWA operators provide a fast way to discard non relevant data.

show abstract

“…Itemset {ABCD} is the superset of itemset {AB} and itemset {CD}. Therefore, we will process function CheckSet in Figure 3.8, lines [13][14][15][16][17][18]. Itemsets {AB} and {CD} become itemset {ABCD}'s child.…”

Section: Data Insertionmentioning

confidence: 99%

“…According to different stream processing models, the research of mining frequent itemsets in data streams can be divided into three categories: landmark windows [15] as shown in Figure 1.2, sliding windows [9,11,12,16] as shown in Figure 1.3, and damped windows [5,18] as shown in Figure 1.4. In the landmark windows model, knowledge discovery is performed based on the values between a specific time stamp called landmark and the present.…”

Section: Window Models In Data Streamsmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Subset-Lattice Algorithm for Mining Closed Frequent Itemsets in Data Streams

Chang

Peng

2012

2012 Conference on Technologies and Applications of Artificial Intelligence

View full text Add to dashboard Cite

Online mining association rules over data streams is an important issue in the area of data mining, where an association rule means that the presence of some items in a transaction will imply the presence of other items in the same transaction. There are many applications of using association rules in data streams, such as market analysis, network security, sensor networks and web tracking. Mining closed frequent itemsets is a further work of mining association rules, which aims to find the subsets of frequent itemsets that could extract all frequent itemsets. Formally, a closed frequent itemset is an frequent itemset which has no superset with the same support as it. Since data streams are continuous, high-speed, and unbounded, archiving everything from data streams is impossible. That is, we can only scan once for the data streams and it is a main-memory database. Therefore, previous algorithms to mine closed frequent itemsets in the traditional database are not suitable for data streams. On the other hand, many applications are interested in the most recent data, and there is a model to deal with the most recent data in data streams, called Sliding Window Model, which acquires the recent data with a window size meets this characteristic. One of well-known algorithms for mining closed frequent itemsets which based on the sliding window model is the NewMoment algorithm. However, the NewMoment algorithm could not efficiently mine closed frequent itemsets in data streams, since they will generate closed frequent itemsets and many unclosed frequent itemsets. Moreover, when data in the sliding window is incrementally updated, the NewMoment algorithm needs to reconstruct the whole tree structure. Therefore, in this thesis, we propose a sliding window approach, the Subset-Lattice algorithm, which embeds the subset property into the lattice structure to efficiently mine closed frequent itemsets. Basically, Our proposed algorithm considers five kinds of set concepts : (1) equivalent, (2) superset, (3) subset, (4) intersection, (5) empty relation, when data items are inserted. We judge closed frequent itemsets without generating unclosed frequent itemsets by these five kinds of set concepts. Moreover, when data in the sliding window is incrementally updated, our Subset-Lattice algorithm will not reconstruct the whole lattice structure. Therefore, our Subset-Lattice algorithm is more efficient than the Moment algorithm. Furthermore, we use the bit-pattern to represent the itemsets, and use bit-operations to speed up the set-checking. From our simulation results, we show that our Subset-Lattice algorithm needs less memory and less processing time than the NewMoment algorithm. When window slides, the execution time could be saved up to 50%.

show abstract

DSM-FI: an efficient algorithm for mining frequent itemsets in data streams

Cited by 64 publications

References 10 publications

Efficient Single Pass Ordered Incremental Pattern Mining

Efficient Single Pass Ordered Incremental Pattern Mining

Speed up gradual rule mining from stream data! A B-Tree and OWA-based approach

An Efficient Subset-Lattice Algorithm for Mining Closed Frequent Itemsets in Data Streams

Contact Info

Product

Resources

About