Improvement and Research of FP-Growth Algorithm Based on Distributed Spark

Deng, Lingling; Lou, Yuansheng

doi:10.1109/ccbd.2015.15

Cited by 19 publications

(8 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parallelized FP-growth work on distributed machines [23]. Its partitioning task is done in such a way that each machine executes an independent group of mining tasks.…”

Section: The Parallel Fp-growthmentioning

confidence: 99%

Geographical queries reformulation using a parallel association rules generator to build spatial taxonomies

Midaoui

Ghali

Qadi

et al. 2021

IJECE

View full text Add to dashboard Cite

Geographical queries need a special process of reformulation by information retrieval systems (IRS) due to their specificities and hierarchical structure. This fact is ignored by most of web search engines. In this paper, we propose an automatic approach for building a spatial taxonomy, that models’ the notion of adjacency that will be used in the reformulation of the spatial part of a geographical query. This approach exploits the documents that are in top of the retrieved list when submitting a spatial entity, which is composed of a spatial relation and a noun of a city. Then, a transactional database is constructed, considering each document extracted as a transaction that contains the nouns of the cities sharing the country of the submitted query’s city. The algorithm frequent pattern growth (FP-growth) is applied to this database in his parallel version (parallel FP-growth: PFP) in order to generate association rules, that will form the country’s taxonomy in a Big Data context. Experiments has been conducted on Spark and their results show that query reformulation using the taxonomy constructed based on our proposed approach improves the precision and the effectiveness of the IRS.

show abstract

“…The parallelized FP-growth work on distributed machines [23]. Its partitioning task is done in such a way that each machine executes an independent group of mining tasks.…”

Section: The Parallel Fp-growthmentioning

confidence: 99%

Geographical queries reformulation using a parallel association rules generator to build spatial taxonomies

Midaoui

Ghali

Qadi

et al. 2021

IJECE

View full text Add to dashboard Cite

show abstract

“…In paper [7] author presents a distributed FP-Growth algorithm by using the spark. The results proved that compare the MapReduce on spark.…”

Section: Related Workmentioning

confidence: 99%

“…The spark is a open source distributed framework designed by the Brekely Lab. Which can applied for Machine Learning, Data Mining and any iterative algorithms [7].…”

Section: Introductionmentioning

confidence: 99%

Distributed FP Growth Algorithm for Cloud Platform without Exposing the Individual Transaction Data

Byreddi*¹,

Reddy²

2019

IJRTE

View full text Add to dashboard Cite

Data mining is a concept of extracting the required patterns to take appropriate decisions. One of the major challenges in data mining is to extract hidden patterns with the secure and privacy from the huge databases. Privacy preserving is a method used to extract hidden patterns with privacy. In this paper Mining Association rules with privacy preserving mechanism in the cloud platform is proposed. It is a powerful technique to find the hidden pattern in the distributed database. For now many mechanisms has proposed but it has many drawback, not proven and not specific. In cloud the data is stored in the servers. The data is distributed in different servers in cloud platform. Each server has one of the transaction data. The current paper proposed the distributed FP growth algorithm for cloud platform without exposing the individual transaction data. The results proved that the proposed algorithm is best to extract hidden pattern from Cloud platform in terms of efficiency.

show abstract

“…Consequently, several methods have been suggested to improve efficiency of the algorithm. Some of such methods are; implementation of parallel FP-Growth ( [15], [14] and [16]), mining only topk frequent itemsets (Lee and Clifton [17] and Wang et al [3], and the use of distributed computing for frequent patterns mining (Deng and Low [18] and Itkar and Kulkarni [19]). Although all of these, and other similar approaches, try to tackle the challenge of the algorithm especially with the increasing datasets, they ignore the mining for infrequent items.…”

Section: Studies To Improve Fp-growth Based On Single Minimum Supportmentioning

confidence: 99%

Mining Frequent Patterns of Crime using FP-Growth with Multiple Minimum Supports based on Shannon Entropy

Matto¹,

Mwangoka²

2018

IJCA

View full text Add to dashboard Cite

FP-Growth is one of the most effective and widely used association rules mining algorithm for discovering interesting relations between items in large datasets. Unfortunately, classical FP-Growth mines frequent patterns by using single user-defined minimum support threshold. This is not adequate for real life applications such as crime patterns mining. On one side, if minimum support is set too low, huge amount of crime patterns (including uninteresting patterns) may be generated, and on the other side, if it is set too high lots of interesting patterns (including seasonal patterns) may be lost. This paper proposes the use of Multiple Item Support (MIS) thresholds instead of single minimum support to tackle the challenge. We employ Shannon entropy method to develop an algorithm that obtains MIS values from crime datasets. The proposed approach is tested on different sizes of input data via a developed working prototype. Experimental results show that our suggested approach outperforms classical FP-Growth in terms of running time and memory use.

show abstract

Improvement and Research of FP-Growth Algorithm Based on Distributed Spark

Cited by 19 publications

References 1 publication

Geographical queries reformulation using a parallel association rules generator to build spatial taxonomies

Geographical queries reformulation using a parallel association rules generator to build spatial taxonomies

Distributed FP Growth Algorithm for Cloud Platform without Exposing the Individual Transaction Data

Mining Frequent Patterns of Crime using FP-Growth with Multiple Minimum Supports based on Shannon Entropy

Contact Info

Product

Resources

About