Grid implementation of the Apriori algorithm

Aflori, Cristian; Craus, Mitică

doi:10.1016/j.advengsoft.2006.08.011

“…The algorithm proposed in [4] cannot be efficient for large data size. Many parallel and distributed variants of sequential apriori algorithm have been discussed in other resources [6,7] . In [5] , grid implementation of frequent item sets in a grid environment dealt with sales transaction of a company.…”

Section: Methodsmentioning

confidence: 99%

“…If size of the moving logs is very large, the overhead in integrating the data source will be too high. To overcome this problem data mining algorithm is executed on conventional distributed environment [6,14] . During the mining process size of the data set transferred between nodes was reduced by local and global mining [15] .…”

Section: Introductionmentioning

confidence: 99%

Incremental DataGrid Mining Algorithm for Mobility Prediction of Mobile Users

Sakthi¹,

Bhuvaneswaran²

2009

Journal of Computer Science

View full text Add to dashboard Cite

Problem statement: Mobility prediction is the important issue in Personal Communication Systems (PCS). Mobile users moving logs are stored in data grid located in different locations. Distributed data mining algorithm is applied on this moving logs to generate the mobility pattern of mobile users. As new moving logs are added to the data grid, existing mobility pattern becomes invalid and it should be updated. One of the existing work to derive the new mobility pattern is re-executing the algorithm from scratch results in excessive computation. Approach: We had designed new incremental algorithm by maintaining infrequent mobility patterns, which avoids unnecessary scan of full database. Incremental data mining algorithm taken lesser time to compute new mobility patterns. The discovered location patterns can be used to provide various location based services to the mobile user by the application server in mobile computing environment. Data grid provided geographically distributed database for computational grid which implements incremental data mining algorithm. We built data grid system on a cluster of workstation using open source globus toolkit 4.0 and Message Passing Interface extended with Grid Services (MPICH-G2). Results: The experiments were conducted on original data sets and data were added incrementally and the computation time was recorded for each data sets. The performance improvement for increment size of 100 K was about 55% for 0.20% support count and it is increased to 60% for 0.25% support count. The performance is increased about 65% for the support count 0.30%. Conclusion: We analyzed our results with various sizes of data sets and the proof shows the time taken to generate mobility pattern by incremental mining algorithm is less than re-computing approach. In future the execution time can further be reduced by balancing the workload of grid nodes.

show abstract

“…In [5] , grid implementation of frequent item sets in a grid environment dealt with sales transaction of a company. These algorithms [5][6][7] cannot be used directly in our domain, because this algorithm does not take into account the network topology while generating the candidate patterns. The services and methods for distributed data mining algorithm were discussed in [5] .…”

Section: Methodsmentioning

confidence: 99%

“…Existing research work applied data mining technique on mobile data for path mining in a single database server [2] . The sequential apriori algorithm implemented on single grid was proposed in [13] . If size of the moving logs is very large, the overhead in integrating the data source will be too high.…”

Section: Introductionmentioning

confidence: 99%

Incremental DataGrid Mining Algorithm for Mobility Prediction of Mobile Users

Sakthi¹

2009

J. of Computer Science

0

View full text Add to dashboard Cite

Problem statement: Mobility prediction is the important issue in Personal Communication Systems (PCS). Mobile users moving logs are stored in data grid located in different locations. Distributed data mining algorithm is applied on this moving logs to generate the mobility pattern of mobile users. As new moving logs are added to the data grid, existing mobility pattern becomes invalid and it should be updated. One of the existing work to derive the new mobility pattern is re-executing the algorithm from scratch results in excessive computation. Approach: We had designed new incremental algorithm by maintaining infrequent mobility patterns, which avoids unnecessary scan of full database. Incremental data mining algorithm taken lesser time to compute new mobility patterns. The discovered location patterns can be used to provide various location based services to the mobile user by the application server in mobile computing environment. Data grid provided geographically distributed database for computational grid which implements incremental data mining algorithm. We built data grid system on a cluster of workstation using open source globus toolkit 4.0 and Message Passing Interface extended with Grid Services (MPICH-G2). Results: The experiments were conducted on original data sets and data were added incrementally and the computation time was recorded for each data sets. The performance improvement for increment size of 100 K was about 55% for 0.20% support count and it is increased to 60% for 0.25% support count. The performance is increased about 65% for the support count 0.30%. Conclusion: We analyzed our results with various sizes of data sets and the proof shows the time taken to generate mobility pattern by incremental mining algorithm is less than re-computing approach. In future the execution time can further be reduced by balancing the workload of grid nodes.

show abstract

“…Apriori needs to scan the database many times, its efficiency is limited by the number of candidate itemsets. According to the shortcoming of Apriori, many people proposed their improved algorithms [3][4][5][6], such as DHP, proposed by Pork et al [7], DIC, proposed by Brin et al [8], MFI-TransSW, proposed by H. F. Li et al [9], and so on. Different from Apriori, FP-growth algorithm is based on the strategy of depth-first search, it does not need to generate candidate itemsets; instead, it compresses datasets into a FPtree and obtains frequent patterns using an FP-tree-based pattern fragment growth mining method.…”

Section: Introductionmentioning

confidence: 99%