A fast high utility itemsets mining algorithm

Abstract

Association rule mining (ARM) identifies frequent itemsets from databases and generates association rules by considering each item in equal value. However, items are actually different in many aspects in a number of real applications, such as retail marketing, network log, etc. The difference between items makes a strong impact on the decision making in these applications. Therefore, traditional ARM cannot meet the demands arising from these applications. By considering the different values of individual items as utilities, utility mining focuses on identifying the itemsets with high utilities. As "downward closure property" doesn't apply to utility mining, the generation of candidate itemsets is the most costly in terms of time and memory space. In this paper, we present a Two-Phase algorithm to efficiently prune down the number of candidates and can precisely obtain the complete set of high utility itemsets. In the first phase, we propose a model that applies the "transaction-weighted downward closure property" on the search space to expedite the identification of candidates. In the second phase, one extra database scan is performed to identify the high utility itemsets. We also parallelize our algorithm on shared memory multi-process architecture using Common Count Partitioned Database (CCPD) strategy. We verify our algorithm by applying it to both synthetic and real databases. It performs very efficiently in terms of speed and memory cost, and shows good scalability on multiple processors, even on large databases that are difficult for existing algorithms to handle.

Isolated items discarding strategy for discovering high utility itemsets

“…However, such a method is too time-consuming for a large dataset environment. Several heuristic methods have been proposed to accelerate the discovery of high utility (or SH-frequent) itemsets, such as the MEU (UMining_H) [27,28,34,35], SIP, CAC, and IAB [4,6] methods. Nevertheless, these predictive methods may not discover some high utility itemsets.…”

“…Recently, Li et al first developed some efficient approaches, including the FSM, SuFSM, ShFSM, and DCG methods, to identify all SH-frequent itemsets [22][23][24]. In the meanwhile, Liu et al also presented a Two-Phase (TP) method to discover all high utility itemsets [27,28].…”

“…Yao et al [34,35] generalize the share-confidence model [6] to develop the conventional utility mining model. This model can be used to measure the utility of an itemset in terms of net profit, total cost, or time spent [27,28,34,35].…”

“…Recently, Yao et al [36] attempted to build a unified framework for utilitybased measures [11,27,28,32,[34][35][36] that allows the user to select a suitable utility mining tool for a specific application; however, this framework only employs existing tools. Thus, to effectively discover high utility itemsets, the need for efficient algorithms remains urgent.…”

Handling Dynamic Weights in Weighted Frequent Pattern Mining

“…Therefore, the weight value of each item was heuristically chosen to be between 0.1 and 0.9, and randomly generated using a log-normal distribution. Some other pattern mining research [26], [27] has adopted the same technique. Figure 4 shows the weight distribution of 2000 distinct items using the log-normal distribution.…”