Verification and validation of MapReduce program model for parallel K-means algorithm on Hadoop cluster

Abstract

With the development of information technology, a large volume of data is growing and getting stored electronically. Thus, the data volumes processing by many applications will routinely cross the petabyte threshold range, in that case it would increase the computational requirements. Efficient processing algorithms and implementation techniques are the key in meeting the scalability and performance requirements in such scientific data analyses. So for the same here, it has been analyzed with the various MapReduce Programs and a parallel clustering algorithm (PKMeans) on Hadoop cluster, using the Concept of MapReduce. Here, in this experiment we have verified and validated various MapReduce applications like wordcount, grep, terasort and parallel K-Means Clustering Algorithm. It has been found that as the number of nodes increases the execution time decreases, but also some of the interesting cases has been found during the experiment and recorded the various performance change and drawn different performance graphs. This experiment is basically a research study of above MapReduce applications and also to verify and validate the MapReduce Program model for Parallel KMeans algorithm on Hadoop Cluster having four nodes.

On Chain Folding Problems of Chain Mapper and Chain Reducer Meta Expressions

“…Expressions are perceived as regular structures behind the Chain Foldingdata sources, i.e., "the underlying deep regular structures are descriptions of the source, which are hidden via the sensing path" [3]. ExpressionIntention combines algebra, geometry and statistics to explain the nature of Chain Foldingdata sources and, thereby, depicts a generative Chain Foldingmodeling perspective on expression analysis.…”

On Job Merging MapReduce Meta Expressions for Multiple Decomposition Mapping and Reducing

“…Expressions are perceived as regular structures behind the Job Merging data sources, i.e., "the underlying deep regular structures are descriptions of the source, which are hidden via the sensing path" [3]. Expression Intention combines algebra, geometry and statistics to explain the nature of Job Merging data sources and, thereby, depicts a generative Job Merging modeling perspective on expression analysis.…”

How Replicated Join Expressions Equal Map Phase or Reduce Phase in a MapReduce Structure?

“…Expressions are perceived as regular structures behind the Replicated Join data sources, i.e., "the underlying deep regular structures are descriptions of the source, which are hidden via the sensing path" [3]. Expression Intention combines algebra, geometry and statistics to explain the nature of Replicated Join data sources and, thereby, depicts a generative Replicated Join modeling perspective on expression analysis.…”