An experimental and comparative benchmark study examining resource utilization in managed Hadoop context

Özdil, Uluer Emre; Ayvaz, Serkan

doi:10.1007/s10586-022-03728-7

Cited by 2 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This not only reduces data access latency, but also provides load balancing of data access requests. This motivated the authors of [18], [19], [20] to investigate the performance of HDFS in remote data access.…”

Section: Related Workmentioning

confidence: 99%

Experimental Analysis of WebHDFS API Throughput

Kalmukov¹,

Marinov²

2023

IJACSA

View full text Add to dashboard Cite

Data analysis is very important for the success of any business today. It helps to optimize business processes, analyze users' behavior, demands etc. There are powerful data analytics tools, such as the ones of the Hadoop ecosystem, but they require multiple high-performance servers to run and highqualified experts to install, configure and support them. In most cases, small companies and start-ups could not afford such expenses. However, they can use them as web services, on demand, and pay much lower fees per request. To do that, companies should somehow share their data with an existing, already deployed, Hadoop cluster. The most common way of uploading their files to the Hadoop's Distributed File System (HDFS) is through the WebHDFS API (Application Programming Interface) that allows remote access to HDFS. For that reason, the API's throughput is very important for the efficient integration of a company's data to the Hadoop cluster. This paper performs a series of experimental analyses aiming to determine the WebHDFS API's throughput, if it is a bottleneck in integration of a company's data to existing Hadoop infrastructure and to detect all possible factors that influence the speed of data transmission between the clients' software and the Hadoop' file system.

show abstract

Section: Related Workmentioning

confidence: 99%