Rule Based Classification on a Multi Node Scalable Hadoop Cluster

Gugnani, Shashank; Khanolkar, Devavrat; Bihany, Tushar; Khadilkar, Nikhil

doi:10.1007/978-3-319-11692-1_15

Cited by 7 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 9a shows the workflow execution time for three different Hadoop applications using the Single Node Method. The applications used were: WordCount -the standard example that comes with Hadoop, Rule Based Classification [17]-a classification algorithm adapted for MapReduce, and Prefix Span [16] -MapReduce version of the popular sequential pattern mining algorithm. This experiment was executed to show that the method developed is generic and could be used for a wide range of Hadoop applications.…”

Section: Results and Evaluationmentioning

confidence: 99%

Extending Science Gateway Frameworks to Support Big Data Applications in the Cloud

et al. 2016

Self Cite

View full text Add to dashboard Cite

Cloud computing offers massive scalability and elasticity required by many scientific and commercial applications. Combining the computational and data handling capabilities of clouds with parallel processing also has the potential to tackle Big Data problems efficiently. Science gateway frameworks and workflow systems enable application developers to implement complex applications and make these available for end-users via simple graphical user interfaces. The integration of such frameworks with Big Data processing tools on the cloud opens new opportunities for application developers. This paper investigates how workflow systems and science gateways can be extended with Big Data processing capabilities. A generic approach based on infrastructure aware workflows is suggested and a proof of concept is implemented based on the WS-PGRADE/gUSE science gateway framework and its integration with the Hadoop parallel data processing solution based on the MapReduce paradigm in the cloud. The provided analysis demonstrates that the methods described to

show abstract

Section: Results and Evaluationmentioning

confidence: 99%

Extending Science Gateway Frameworks to Support Big Data Applications in the Cloud

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…MapReduce is a Java-based software framework that lets programmers run the same computation on multiple machines at the same time to process data faster, reliable, fault-tolerant manner and more efficiently. Apache Hadoop is widely used in science because it works well with the MapReduce programming language and is free to use [11,12]. With the help of the Google File System (GFS), the information is broken up into smaller pieces and sent to many computers.…”

Section: Mapreduce Frameworkmentioning

confidence: 99%

“…If you want to cluster similar textures of images or regions, you may use the "mean μmn" shown in (11) and "standard deviation σmn" given in (12) to represent the region's homogeneous texture feature:…”

Section: Texture Featurementioning

confidence: 99%

Image mining technique using Hadoop map reduce over distributed multi-node computers connections

Shamki,

Rabee

2022

AJEST

View full text Add to dashboard Cite

The amount of today's image collections has exploded to petabytes of data. A fair amount of time cannot be allotted for the analysis of such massive datasets using a personal computer. As a result, distributed computing is required for current image collection mining. This work used multi-nodes-computers for image mining in order to improve reliability, fall tolerance, and time efficiency. Because of this, the data was divided up across the nodes in the Hadoop multi-node cluster, and the results were then compiled to create an image clustering algorithm. One master node and two slave nodes were used to test our technique on a huge dataset. Using multi-node Hadoop, we found that we could get a speed-up in implementing as high as a single-node Hadoop implementation.

show abstract

“…The MapReduce framework can be used to distribute the feature selection task [14], [15]. The MapReduce framework [16] was successfully tested on very large datasets from different domains [17], [18], [19]. The most known implementations of MapReduce are Hadoop and MongoDB.…”

Section: The Computational Time Problemmentioning

confidence: 99%

A distributed graph based approach for rough classifications considering dominance relations between overlapping classes

Laghmari

Ramdani

Marsala

2015

2015 10th International Conference on Intelligent Systems: Theories and Applications (SITA)

View full text Add to dashboard Cite

Several data from real world applications involves overlapping classes. Data is allowed to belong to multiple classes with different membership degrees. In this paper, we explore a different concept characterizing social networks, documents, and most of biological and chemical datasets: data could have multiple classes, but dominant classes are better noticed than dominated classes. For example, a document could discuss economy and politics, but it would be more focused on politics. A molecule could have multiple odors, but experts could notice some odors better than others. We are interested in this type of data, where a dominance relation exists between classes. Experts could easily make mistakes because dominated classes are hardly noticed. Data incoherence is a serious problem but not the only one. There is too much irrelevant and redundant attributes. Unfortunately this increases the computational time of generating classifiers. Our first challenge is to find an adapted model to overlapping classes considering dominance relations. The second challenge is to find the most relevant attributes. Finally the third challenge is to ensure that the approach gives results in an acceptable time. We address those challenges by taking advantage of the rough set theory, which is suited for incoherent data and allows multiple classes and attributes selection. The proposed approach works in a parallel and decentralized way to reduce the computational time. We tested it on real chemical data and the collected results are very promising.

show abstract

Rule Based Classification on a Multi Node Scalable Hadoop Cluster

Cited by 7 publications

References 7 publications

Extending Science Gateway Frameworks to Support Big Data Applications in the Cloud

Extending Science Gateway Frameworks to Support Big Data Applications in the Cloud

Image mining technique using Hadoop map reduce over distributed multi-node computers connections

A distributed graph based approach for rough classifications considering dominance relations between overlapping classes

Contact Info

Product

Resources

About