A Metadata Management Mechanism Based on HDFS

Abstract

A metadata management mechanism on the basis of HDFS was proposed in this paper, which is applied for cloud storage. The basic unit of consistent hashing is DHU in the scheme. Metadata is distributed in multiple servers in this solution, and the position of metadata is recorded in a mapping table. In addition, prefetching strategy and local cache technology are used to improve the performance of the system. Operation logs and Multi-Paxos algorithm are being used to ensure system’s reliability and consistency. Improved system with a lower latency in reading and writing than the original system was seen in the experiments. Good performance for cloud storage system can be shown by this system.