Investigating Apache Hama: a bulk synchronous parallel computing framework

Siddique, Kamran; Akhtar, Zahid; Kim, Yangwoo; Jeong, Young‐Sik; Yoon, Edward

doi:10.1007/s11227-017-1987-9

Cited by 11 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hama [28] is a pure bulk synchronous parallel model which can do vast scientific computations, e.g. matrix, graph, and network algorithms.…”

Section: Hamamentioning

confidence: 99%

A Taxonomy on Big Data: Survey

Patgiri

2018

Preprint

View full text Add to dashboard Cite

The Big Data is the most popular paradigm nowadays and it has almost no untouched area. For instance, science, engineering, economics, business, social science, and government. The Big Data are used to boost up the organization performance using massive amount of dataset. The Data are assets of the organization, and these data gives revenue to the organizations. Therefore, the Big Data is spawning everywhere to enhance the organizations' revenue. Thus, many new technologies emerging based on Big Data. In this paper, we present the taxonomy of Big Data. Besides, we present in-depth insight on the Big Data paradigm.

show abstract

“…Hama [28] is a pure bulk synchronous parallel model which can do vast scientific computations, e.g. matrix, graph, and network algorithms.…”

Section: Hamamentioning

confidence: 99%

A Taxonomy on Big Data: Survey

Patgiri

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Fegaras et al [26] proposed MRQL as SQL-like query language for large-scale data analysis. MRQL is a HLQL based on BSP that can evaluate their queries in four modes [27]: MR mode using Apache Hadoop [1], BSP mode using Apache Hama [28], Spark mode using Apache Spark [24], and in Flink mode using Apache Flink [29]. In MRQL-to-MR mode, MRQL translates MRQL queries in physical MR algebraic operators, which optimizes and translates the algebraic form to a physical plan consisting of the physical MR operators and not into native MR jobs [26].…”

Section: Related Workmentioning

confidence: 99%

“…In MRQL-to-MR mode, MRQL translates MRQL queries in physical MR algebraic operators, which optimizes and translates the algebraic form to a physical plan consisting of the physical MR operators and not into native MR jobs [26]. In [28], the authors presented Apache Hama, BSP-based model processing, which provides not only pure BSP programming model but it is also constructed on the top of HDFS [30]. Further, the work implemented in [31] presents Apache Drill as an interactive ad-hoc analysis.…”

Section: Related Workmentioning

confidence: 99%

Evaluation of high-level query languages based on MapReduce in Big Data

2018

View full text Add to dashboard Cite

Since it was presented by Google in 2004, MapReduce (MR) [1] has been emerged as a popular framework for Big Data processing model in cluster environment and cloud computing [2]. It has become a key of success for processing, analyzing and managing large data sets with some number of implementations including the open-source Hadoop framework [3, 4]. MR has many interesting qualities, highly noticed in its design and plainness in writing programs. It has only two functions, known as Map and Reduce, written by developer to process key-value data pairs. Even though, MR is very simple to understand its principle and basic concepts but it is hard to develop, optimize, and maintain its functions especially in large-scale projects [5]. MR requires approaching any problem in terms of key-value pairs where each pair can be independently computed. Also, coding efficient MapReduce programs, mainly in Java, was non-trivial for those who interested to build large-scale projects even though their programming level. This meant that many operations need multiple inputs/outputs, both simple and complex, that were very hard to achieve without wasting programming efforts and time.

show abstract

“…Among big data processing frameworks used in practice, Hama follows the original BSP model relatively closely [40,41]. It supports diverse massive computational tasks in several application domains such as graphs, matrices, machine learning, deep learning, and network algorithms.…”

Section: Utilizing Machine Learning and Bulk Synchronous Parallel Commentioning

confidence: 99%

Toward Bulk Synchronous Parallel-Based Machine Learning Techniques for Anomaly Detection in High-Speed Big Data Networks

et al. 2017

Self Cite

View full text Add to dashboard Cite

Anomaly detection systems, also known as intrusion detection systems (IDSs), continuously monitor network traffic aiming to identify malicious actions. Extensive research has been conducted to build efficient IDSs emphasizing two essential characteristics. The first is concerned with finding optimal feature selection, while another deals with employing robust classification schemes. However, the advent of big data concepts in anomaly detection domain and the appearance of sophisticated network attacks in the modern era require some fundamental methodological revisions to develop IDSs. Therefore, we first identify two more significant characteristics in addition to the ones mentioned above. These refer to the need for employing specialized big data processing frameworks and utilizing appropriate datasets for validating system's performance, which is largely overlooked in existing studies. Afterwards, we set out to develop an anomaly detection system that comprehensively follows these four identified characteristics, i.e., the proposed system (i) performs feature ranking and selection using information gain and automated branch-and-bound algorithms respectively; (ii) employs logistic regression and extreme gradient boosting techniques for classification; (iii) introduces bulk synchronous parallel processing to cater computational requirements of high-speed big data networks; and; (iv) uses the Infromation Security Centre of Excellence, of the University of Brunswick real-time contemporary dataset for performance evaluation. We present experimental results that verify the efficacy of the proposed system.

show abstract

Investigating Apache Hama: a bulk synchronous parallel computing framework

Cited by 11 publications

References 22 publications

A Taxonomy on Big Data: Survey

A Taxonomy on Big Data: Survey

Evaluation of high-level query languages based on MapReduce in Big Data

Toward Bulk Synchronous Parallel-Based Machine Learning Techniques for Anomaly Detection in High-Speed Big Data Networks

Contact Info

Product

Resources

About