A Comprehensive Survey for Hadoop Distributed File System

Abstract

In the last few days, data and the internet have become increasingly growing, occurring in big data. For these problems, there are many software frameworks used to increase the performance of the distributed system. This software is used for available ample data storage. One of the most beneficial software frameworks used to utilize data in distributed systems is Hadoop. This software creates machine clustering and formatting the work between them. Hadoop consists of two major components: Hadoop Distributed File System (HDFS) and Map Reduce (MR). By Hadoop, we can process, count, and distribute each word in a large file and know the number of affecting for each of them. The HDFS is designed to effectively store and transmit colossal data sets to high-bandwidth user applications. The differences between this and other file systems provided are relevant. HDFS is intended for low-cost hardware and is exceptionally tolerant to defects. Thousands of computers in a vast cluster both have directly associated storage functions and user programmers. The resource scales with demand while being cost-effective in all sizes by distributing storage and calculation through numerous servers. Depending on the above characteristics of the HDFS, many researchers worked in this field trying to enhance the performance and efficiency of the addressed file system to be one of the most active cloud systems. This paper offers an adequate study to review the essential investigations as a trend beneficial for researchers wishing to operate in such a system. The basic ideas and features of the investigated experiments were taken into account to have a robust comparison, which simplifies the selection for future researchers in this subject.
 According to many authors, this paper will explain what Hadoop is and its architectures, how it works, and its performance analysis in a distributed systems. In addition, assessing each Writing and compare with each other.

Survey of Distributed Computing Frameworks for Supporting Big Data Analysis

“…Hadoop is a quasi-standard platform for distributed big data applications [88] . In the Hadoop MapReduce computing framework, an iterative algorithm is decomposed into a sequence of pairs of mapper and reducer operations, that are executed sequentially in a pipeline manner [89] . One pair of mapper and reducer operations is taken as an example.…”

Construction of a rural tourism information service management system for multi-source heterogeneous data processing

“…In addition to MongoDB distributed storage tools, Hadoop distributed file system is also used to construct data storage subsystem ( Merceedi & Sabry, 2021 ). This is because with the operation of the rural tourism cloud platform, various types of data in the forum will become larger and larger as time goes on, and the efficiency of data retrieval by MongoDB NoSQL database will be lower and lower.…”

“…HDFS adopts master/slave architecture. There is one Namenode node and multiple Datanode nodes in the HDFS cluster, while Namenode is used to manage Datanode, and Datanode is used to store actual data ( Merceedi & Sabry, 2021 ).…”

Big Data Sentiment Analysis of Twitter Data

“…Using HADOOP, the author [11] of the research accomplished similar work in her paper by analyzing the data from Twitter, which is also known as big data. The analytics tools and methods that the author of the study utilised are insufficient to manage huge data, however they are used in the paper [12]. Because of this, there is a prerequisite of making use of cloud storage for the kind of applications being discussed here.…”