zFS - a scalable distributed file system using object disks

Rodeh, Ohad; Teperman, Avi

doi:10.1109/mass.2003.1194858

Cited by 89 publications

(50 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vesta [13], zFS [14] and Lustre [15], [16] all hashed the file pathname and/or some other unique identifier to determine the location of metadata. However, distributing metadata by hashing eliminates all hierarchical localities, and with them, many of the typical locality benefits of local file systems, for instance, the POSIX directory access semantics.…”

Section: Alternative Techniquesmentioning

confidence: 99%

NDCouplingHDFS: A Coupling Architecture for a Power-Proportional Hadoop Distributed File System

Hikida

Yokota

2014

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYEnergy-aware distributed file systems are increasingly moving toward power-proportional designs. However, current works have not considered the cost of updating data sets that were modified in a lowpower mode, where a subset of nodes were powered off. In detail, when the system moves to a high-power mode, it must internally replicate the updated data to the reactivated nodes. Effectively reflecting the updated data is vital in making a distributed file system, such as the Hadoop Distributed File System (HDFS), power proportional. In the current HDFS design, when the system changes power mode, the block replication process is ineffectively restrained by a single NameNode because of access congestion of the metadata information of blocks. This paper presents a novel architecture, a NameNode and DataNode Coupling Hadoop Distributed File System (NDCouplingHDFS), which effectively reflects the updated blocks when the system goes into high-power mode. This is achieved by coupling metadata management and data management at each node to efficiently localize the range of blocks maintained by the metadata. Experiments using actual machines show that NDCouplingHDFS is able to significantly reduce the execution time required to move updated blocks by 46% relative to the normal HDFS. Moreover, NDCouplingHDFS is capable of increasing the throughput of the system supporting MapReduce by applying an index in metadata management. key words: power-proportionality, HDFS, metadata management

show abstract

Section: Alternative Techniquesmentioning

confidence: 99%

NDCouplingHDFS: A Coupling Architecture for a Power-Proportional Hadoop Distributed File System

Hikida

Yokota

2014

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…ZFS is known for its ability to support high capacity, integrating concepts from file management and partitioning management, innovative disk structures, and a simple storage management system. ZFS is an open source project (Rodeh & Teperman, 2003).…”

Section: Compressed Read-write File Systemsmentioning

confidence: 99%

Smaller Flight Data Recorders

Wiseman¹,

Barkai²

2013

Journal of Aviation Technology and Engineering

View full text Add to dashboard Cite

Data captured by flight data recorders are generally stored on the system's embedded hard disk. A common problem is the lack of storage space on the disk. This lack of space for data storage leads to either a constant effort to reduce the space used by data, or to increased costs due to acquisition of additional space, which is not always possible. File compression can solve the problem, but carries with it the potential drawback of the increased overhead required when writing the data to the disk, putting an excessive load on the system and degrading system performance. The author suggests the use of an efficient compressed file system that both compresses data in real time and ensures that there will be minimal impact on the performance of other tasks.

show abstract

“…It has a good expandability, but the directory is distributed in several metadata servers, and it requires directory traversal among metadata servers to locate the metadata, which may result in the low efficiency access. Hash method [8][9] will locate the storage position of this file according to the file identifier (such as the full path name of the file). It breaks through the limitations of single metadata server, but it requires migrating related metadata when renaming a directory.…”

Section: Introductionmentioning

confidence: 99%