Mining frequent patterns without candidate generation

Abstract

Abstract. Mining frequent patterns in transaction databases, time-series databases, and many other kinds of databases has been studied popularly in data mining research. Most of the previous studies adopt an Apriori-like candidate set generation-and-test approach. However, candidate set generation is still costly, especially when there exist a large number of patterns and/or long patterns.In this study, we propose a novel frequent-pattern tree (FP-tree) structure, which is an extended prefix-tree structure for storing compressed, crucial information about frequent patterns, and develop an efficient FP-treebased mining method, FP-growth, for mining the complete set of frequent patterns by pattern fragment growth. Efficiency of mining is achieved with three techniques: (1) a large database is compressed into a condensed, smaller data structure, FP-tree which avoids costly, repeated database scans, (2) our FP-tree-based mining adopts a pattern-fragment growth method to avoid the costly generation of a large number of candidate sets, and (3) a partitioning-based, divide-and-conquer method is used to decompose the mining task into a set of smaller tasks for mining confined patterns in conditional databases, which dramatically reduces the search space. Our performance study shows that the FP-growth method is efficient and scalable for mining both long and short frequent patterns, and is about an order of magnitude faster than the Apriori algorithm and also faster than some recently reported new frequent-pattern mining methods.

Mining Sequential Access Pattern with Low Support From Large Pre-Processed Web Logs

“…Study of WAP-mine algorithm (Han et al 2000)-Pattern-Growth miner with Tree Projection At the same time of FreeSpan and PrefixSpan in 2000/2001, another major contribution was made as a pattern growth and tree structure mining technique, which is the WAP-mine algorithm (Han et al, 2000) with its WAP-tree structure. Here, the sequence database is scanned only twice to build the WAPtree from frequent sequences along with their support, a "header table" is maintained to point at the first occurrence for each item in a frequent item set, which is later tracked in a threaded way to mine the tree for frequent sequences, building on the suffix.…”

“…5 and 6 with complete set of frequent sequence fs ={e, be, abe, ae, b, bb, ab, a, ba}. WAP-mine algorithm is reported to have better scalability than GSP and to outperform it by a margin (Han et al, 2000). Although it scans the database only twice and can avoid the problem of generating explosive candidates as in Apriori-based and candidate generate-and-test methods, WAP-mine suffers from a memory consumption problem as it recursively reconstructs numerous intermediate WAP-trees during mining and in particular, as the number of mined frequent patterns increases.…”

“…A sequential pattern mining algorithm mines the sequence database looking for repeating patterns (known as frequent sequences) that can be used later by end users or management to find associations between the different items or events in their data for purposes such as marketing campaigns, business reorganization, prediction and planning. With increase in the use of the world wide web for E-Commerce businesses, web services and others, web usage mining has appeared in the literature as one of the most prevalent application areas of sequential pattern mining (Han et al, 2000;Ezeife and Lu, 2005;El-Sayed et al, 2004;Han et al, 2004;Madan and Madan, 2010) This study focuses on sequential pattern mining techniques including those applicable to web usage mining.…”

Frequent Itemset Mining for Big Data Using Greatest Common Divisor Technique

“…Han, 2000). "The main aim of this algorithm was to remove the bottlenecks of the Apriori algorithm in generating and testing candidate sets" (Pramod S., 2015).…”

Using Unique-Prime-Factorization Theorem to Mine Frequent Patterns without Generating Tree

“…Pervious works: FP-growth (Han et al, 2000) is a well-known algorithm that uses the FP-tree data structure to achieve a condensed representation of the database transactions and employs a divide-and-conquer approach to decompose the mining problem into a set of smaller problems. In essence, it mines all the frequent itemsets by recursively finding all frequent itemsets in the conditional pattern base which is efficiently constructed with the help of a node link structure.…”