Incremental Frequent Itemsets Mining With FCFP Tree

One of the most challenging tasks in association rule mining is that when a new incremental database is added to an original database, some existing frequent itemsets may become infrequent itemsets and vice versa. As a result, some previous association rules may become invalid and some new association rules may emerge. We designed a new, more efficient approach for incremental association rule mining using a Fast Incremental Updating Frequent Pattern growth algorithm (FIUFP-Growth), a new Incremental Conditional Pattern tree (ICP-tree), and a compact sub-tree suitable for incremental mining of frequent itemsets. This algorithm retrieves previous frequent itemsets that have already been mined from the original database and their support counts then use them to efficiently mine frequent itemsets from the updated database and ICP-tree, reducing the number of rescans of the original database. Our algorithm reduced usages of resource and time for unnecessary sub-tree construction compared to individual FP-Growth, FUFP-tree maintenance, Pre-FUFP, and FCFPIM algorithms. From the results, at 3% minimum support threshold, the average execution time for pattern growth mining of our algorithm performs 46% faster than FP-Growth, FUFP-tree, Pre-FUFP, and FCFPIM. This approach to incremental association rule mining and our experimental findings may directly benefit designers and developers of computer business intelligence methods.INDEX TERMS Association rule mining, data mining, FP-tree, FP-growth, FPISC-tree, frequent itemset mining, incremental association rule mining.

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Incremental Association Rule Mining With a Fast Incremental Updating Frequent Pattern Growth Algorithm

Thurachon

Kreesuradej

2021

“…And the characteristics are sparse and diverse relative to the dimensions in a spectrum. The existing searching methods are applied to find some required data, including classification methods [3], clustering algorithms [4], outlier detection algorithms [5], association rules mining [6], etc. These algorithms exhibit good performance in various fields, including image classification [3], spectral clustering [7], credit card theft [8], and so on.…”

Section: A Motivationsmentioning

confidence: 99%

SVM-Lattice: A Recognition and Evaluation Frame for Double-Peaked Profiles

Yang

Cai

et al. 2020

In big data era, the special data with rare characteristics may be of great significations. However, it is very difficult to automatically search these samples from the massive and high-dimensional datasets and systematically evaluate them. The DoPS, our previous work [1], provided a search method of rare spectra with double-peaked profiles from massive and high-dimensional data of LAMOST survey. The identification of the results is mainly depended on visually inspection by astronomers. In this paper, as a follow-up study, a new lattice structure named SVM-Lattice is designed based on SVM(Support Vector Machine) and FCL(Formal Concept Lattice) and particularly applied in the recognition and evaluation of rare spectra with double-peaked profiles. First, each node in the SVM-Lattice structure contains two components: the intents are defined by the support vectors trained by the spectral samples with the specific characteristics, and the relevant extents are all the positive samples classified by the support vectors. The hyperplanes can be extracted from every lattice node and used as classifiers to search targets by categories. A generalization and specialization relationship is expressed between the layers, and higher layers indicate higher confidence of targets. Then, including a SVM-Lattice building algorithm, a pruning algorithm based on association rules, and an evaluation algorithm, the supporting algorithms are provided and analysed. Finally, for the recognition and evaluation of spectra with double-peaked profiles, several data sets from LAMOST survey are used as experimental dataset. The results exhibit good consistency with traditional methods, more detailed and accurate evaluations of classification results, and higher searching efficiency than other similar methods. INDEX TERMS SVM-Lattice, double-peaked profiles, support vector machine, formal concept lattice.

“…Several of these algorithms maintain itemsets that are almost-frequent in memory to avoid rescanning the database when it is updated. Second, some stream mining algorithms [3]- [5], [32], [34] have been proposed to update frequent itemsets for potentially infinite transaction streams, that is when the database cannot be read more than once. Popular algorithms of this type include estDec [3] and estDec+ [32], which use structures to maintain patterns and calculate upper-bounds on calculation errors of itemset support values.…”

Section: Related Workmentioning

confidence: 99%

Mining Productive Itemsets in Dynamic Databases

Fournier-Viger

et al. 2020

Discovering frequent itemsets is a data analysis task used in numerous domains. It consists of finding sets of items (itemsets) that frequently appear in a set of database records (also called transactions). Though discovering frequent itemsets is useful, it can produce a large amount of spurious patterns. As a result, the user may spend a great amount of time to analyze the itemsets found by a frequent itemset mining algorithm to find truly interesting patterns. Hence, in recent years, a key research topic has emerged which is to discover statistically significant patterns in databases. The most popular model for identifying itemsets that are statistically significant is to discover non-redundant productive itemsets. The state-ofthe-art algorithm to extract this set of patterns is OPUS-Miner. A key drawback of that algorithm is that it is designed to be applied to a static database. Moreover, a second drawback of OPUS-Miner is that it discovers all patterns in a database. In other words, the user cannot search for itemsets containing some specific items. This paper addresses these issues by defining the novel problem of discovering targeted non redundant productive itemsets in dynamic databases. An algorithm named IDPI+ (Interactive Discovery of Productive Itemsets) is presented, storing transactions in a tree structure, which can then be interactively queried to identify productive and non redundant itemsets containing specific items. A structure named Query-Tree is also introduced to process many queries at the same time. Moreover, to handle dynamic databases, efficient transaction insertion and deletion algorithms are provided to update the tree. It was observed in an experimental evaluation on benchmark datasets containing various types of data that IDPI+ can handle thousands of queries per second on a desktop computer. Moreover, it was found that IPDI+ is more than an order of magnitude faster than a baseline algorithm.