An Improved Approach for Analysis of Hadoop Data for All Files

Jain, Heena; Goyal, Ajay

doi:10.5120/ijca2017912663

Cited by 6 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Um dos princípios do Hadoopé mover a aplicação até os dados, evitando mover os dados em si. Isso possibilita aumentar a eficiência de processamento geral da plataforma [Jain and Goyal 2017]. O MapReduce foi projetado para possibilitar tal comportamento enquanto executa sobre o HDFS [Achari 2015].…”

Section: Apache Hadoopunclassified

Política Customizada de Balanceamento de Réplicas para o HDFS Balancer do Apache Hadoop

Fazul

Barcelos

2019

Anais Do Workshop De Testes E Tolerância a Falhas (WTF)

View full text Add to dashboard Cite

Data replication is a fundamental mechanism of the Hadoop Distributed File System (HDFS). However, the way data is spread across the cluster directly affects the replication balancing. The HDFS Balancer is a Hadoop integrated tool which can balance the storage load on each machine by moving data between nodes, although its operation does not address the specific needs of applications while performing block rearrangement. This paper proposes a customized balancing policy for HDFS Balancer based on a system of priorities, which can be adapted and configured according to usage demands. The priorities define whether HDFS parameters, or whether cluster topology should be considered during the operation, thus making the balancing more flexible.

show abstract

Section: Apache Hadoopunclassified