MRQAR: A generic MapReduce framework to discover quantitative association rules in big data problems

Martín, Daniel; Martínez-Ballesteros, M.; García-Gil, Diego; Alcalá‐Fdez, Jesús; Herrera, Francisco; Santos, José Cristóbal Riquelme

doi:10.1016/j.knosys.2018.04.037

Cited by 38 publications

(15 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As to the two mining algorithms, we can know that the most time-consuming part is to derive fuzzy large itemsets. To deal with this problem, MapReduce-based algorithms can be employed to improve the efficiency [25], [31]. For example, Martín et al presented a generic MapReduce framework for rule discovery [25], and Singh et al proposed a MapReduce-based Apriori algorithm for performance optimization on a Hadoop cluster [31].…”

Section: E Discussionmentioning

confidence: 99%

Post-Analysis Framework for Mining Actionable Patterns Using Clustering and Genetic Algorithms

Chen

Hong

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Mining association rules is an important technique in data analysis. Many approaches for rule analysis have been designed to address different problems. Among them, some works developed from multiobjective genetic algorithms derive a set of Pareto solutions, each of which contains a set of membership functions for fuzzy data mining from quantitative transactions with taxonomy. However, because more than one solution exists in a Pareto set, finding a method to determine the appropriate membership functions and combine them with useful knowledge for mining actionable patterns (such as fuzzy generalized association rules and fuzzy utility itemsets) is a useful research problem. Hence, this paper presents a post-analysisbased genetic-fuzzy mining (PA-GFM) framework for mining actionable patterns that involves two phases: membership-function mining and actionable pattern mining. In the first phase, an existing approach is utilized to derive the Pareto solutions with objective functions. In the second phase, a clustering technique using clustering attributes selected by the users is employed to group the Pareto solutions. The representative solution from each group is then exploited to mine actionable patterns based on the users' requirements. Experiments were conducted on both a simulated dataset and a real one to investigate the performance of the PA-GFM framework. INDEX TERMS Clustering algorithms, domain-driven data mining, fuzzy generalized association rules, fuzzy utility itemsets, multiobjective genetic algorithms.

show abstract

Section: E Discussionmentioning

confidence: 99%

Post-Analysis Framework for Mining Actionable Patterns Using Clustering and Genetic Algorithms

Chen

Hong

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Based on the MapReduce framework, there are some algorithms revised from Apriori that can mine frequent itemsets from big databases [19], [20]. Further, there are frequent itemset mining algorithms adopting MapReduce and genetic programming [21], [22].…”

Section: ) Set-enumeration Treementioning

confidence: 99%

Efficient Mining of Frequent Itemsets Using Only One Dynamic Prefix Tree

Hang

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Frequent itemset mining is a fundamental problem in data mining area because frequent itemsets have been extensively used in reasoning, classifying, clustering, and so on. To mine frequent itemsets, previous algorithms based on a prefix tree structure have to construct many prefix trees, which is very time-consuming. In this paper, we propose a novel frequent itemset mining algorithm called DPT (Dynamic Prefix Tree) which uses only one prefix tree. We first introduce the concept of the postconditional database of an itemset, and analyze the distribution of an itemset's post-conditional database in a prefix tree representing a database. Subsequently, we illuminate how DPT adjusts the prefix tree to mine frequent itemsets and give three optimization techniques. An interesting advantage of DPT is that the algorithm can directly output a prefix tree representing all frequent itemsets after slight modifications. Using only one dynamic prefix tree, DPT avoids the high cost of constructing many prefix trees and thus gains significant performance improvement. Experimental results show that DPT remarkably outperforms previous algorithms with respect to running time and memory usage, and that a prefix tree representing all frequent itemsets DPT outputs can be used more efficient than a list representing them previous algorithms output.

show abstract

“…MapReduce solutions based on Apriori (Lin et al, ), FP‐Growth (H. Li et al, ), and ECLAT (Moens et al, ) can be found in the literature. Similarly, some nonexhaustive search methods have also been proposed (Martín et al, ). It is our understanding that even when extremely efficient solutions based on MapReduce have been proposed, none of them propose a completely new methodology (solutions are mainly based on applications of existing algorithms to the MapReduce framework).…”

Section: Lesson Learnedmentioning

confidence: 99%

Frequent itemset mining: A 25 years review

Luna

Fournier-Viger

Ventura

2019

WIREs Data Min & Knowl

175

View full text Add to dashboard Cite

Frequent itemset mining (FIM) is an essential task within data analysis since it is responsible for extracting frequently occurring events, patterns, or items in data. Insights from such pattern analysis offer important benefits in decision‐making processes. However, algorithmic solutions for mining such kind of patterns are not straightforward since the computational complexity exponentially increases with the number of items in data. This issue, together with the significant memory consumption that is present in the mining process, makes it necessary to propose extremely efficient solutions. Since the FIM problem was first described in the early 1990s, multiple solutions have been proposed by considering centralized systems as well as parallel (shared or nonshared memory) architectures. Solutions can also be divided into exhaustive search and nonexhaustive search models. Many of such approaches are extensions of other solutions and it is therefore necessary to analyze how this task has been considered during the last decades. This article is categorized under: Algorithmic Development > Association Rules Technologies > Association Rules

show abstract

MRQAR: A generic MapReduce framework to discover quantitative association rules in big data problems

Cited by 38 publications

References 36 publications

Post-Analysis Framework for Mining Actionable Patterns Using Clustering and Genetic Algorithms

Post-Analysis Framework for Mining Actionable Patterns Using Clustering and Genetic Algorithms

Efficient Mining of Frequent Itemsets Using Only One Dynamic Prefix Tree

Frequent itemset mining: A 25 years review

Contact Info

Product

Resources

About