Richer File System Metadata Using Links and Attributes

Ames, Alexander K.; Bobb, Nikhil; Brandt, Scott; Hiatt, A.; Maltzahn, Carlos; Miller, Ethan L.; Neeman, A; Tuteja, Deepa

doi:10.1109/msst.2005.28

Cited by 40 publications

(33 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proposals that have attempted to replace hierarchical directory structures altogether [3,5,7,10,12,17,19,24,26], in favour of relying on rich collections of metadata, can also be mapped in terms of their particular choices.…”

Section: Mapping Existing Workmentioning

confidence: 99%

See 1 more Smart Citation

Exploring the design space of metadata-focused file management systems

Watson

Dekeyser

Albadri

2017

Proceedings of the Australasian Computer Science Week Multiconference

View full text Add to dashboard Cite

Operating systems both old and new are reliant on the venerable hierarchical file system. For some time now, however, attempts have been made to either define new file systems or to bolt on applications that offer much improved functionality to attach and use metadata. This is because researchers have shown that traditional file systems are not able to meet users' needs in terms of organising large numbers of files effectively, and to support expeditious retrieval of those files when they are needed at a later time.Numerous proposals for post-hierarchical file management systems have been described in the literature; researchers focus on different dimensions of such systems in order to solve or reduce identified limitations. In some cases this leads to significantly different file system architectures, while in other cases new functionality is added on top of a traditional system through special purpose userspace applications. Orthogonally, some proposals focus on tags while others favour named attribute-value pairs. Still other choices are, seemingly, made in an ad hoc and often implicit manner. This paper investigates the different dimensions and associated choices that participate in the proposal of new approaches and that affect their ability to improve on current systems. The Cartesian product of those dimensions and options forms a large design space; we map some of the existing literature onto that design space and discuss approaches to evaluate new proposals.

show abstract

Section: Mapping Existing Workmentioning

confidence: 99%

“…The table discounts, among others, the "relationships" dimension; however, several proposals (LIFS [3,4], MDFS [10], and QMDS [5]) are focused precisely on that dimension.…”

Section: Mapping Existing Workmentioning

confidence: 99%

Exploring the design space of metadata-focused file management systems

Watson

Dekeyser

Albadri

2017

Proceedings of the Australasian Computer Science Week Multiconference

View full text Add to dashboard Cite

show abstract

“…The distinction between memory regions is similar to that between the protected and non-protected regions used in the NVRAM store of our work. LiFS [1,2] is a relational link-based file system that stores all of its metadata structures in MRAM. While all of these systems promise higher performance, none of these systems include the combination of page protection via memory protection and online consistency checking that we describe.…”

Section: Nvram In File Systemsmentioning

confidence: 99%

“…Mechanisms for protected regions and logging were incorporated into an experimental file system called LiFS [1]. The consistency checking code currently resides outside of LiFS, but runs against a log generated by LiFS.…”

Section: Prototype Implementationmentioning

confidence: 99%

Reliability mechanisms for file systems using non-volatile memory as a metadata store

Greenan

Miller

2006

Proceedings of the 6th ACM &Amp; IEEE International Conference on Embedded Software - EMSOFT '06

View full text Add to dashboard Cite

Portable systems such as cell phones and portable media players commonly use non-volatile RAM (NVRAM) to hold all of their data and metadata, and larger systems can store metadata in NVRAM to increase file system performance by reducing synchronization and transfer overhead between disk and memory data structures. Unfortunately, wayward writes from buggy software and random bit flips may result in an unreliable persistent store. We introduce two orthogonal and complementary approaches to reliably storing file system structures in NVRAM. First, we reinforce hardware and operating system memory consistency by employing page-level write protection and error correcting codes. Second, we perform on-line consistency checking of the filesystem structures by replaying logged file system transactions on copied data structures; a structure is consistent if the replayed copy matches its live counterpart. Our experiments show that the protection mechanisms can increase fault tolerance by six orders of magnitude while incurring an acceptable amount of overhead on writes to NVRAM. Since NVRAM is much faster and consumes far less power than disk-based storage, the added overhead of error checking leaves an NVRAM-based system both faster and more reliable than a disk-based system. Additionally, our techniques can be implemented on systems lacking hardware support for memory management, allowing them to be used on lowend and embedded systems without an MMU.

show abstract

“…Prior experience in attempting to use a relational database for file metadata with a graph structure [10] has led us to our approach of implementing our own store for such metadata [11], and we have continued to pursue that approach in this work. Our performance results presented in section 6 show a validation of this approach, exhibiting superior performance of our system against that of a relational database.…”

Section: Introductionmentioning

confidence: 99%

QMDS: a file system metadata management service supporting a graph data model-based query language

Ames

Gokhale

Maltzahn

2013

International Journal of Parallel, Emergent and Distributed Sys

View full text Add to dashboard Cite

File system metadata management has become a bottleneck for many data-intensive applications that rely on high-performance file systems. Part of the bottleneck is due to the limitations of an almost 50 year old interface standard with metadata abstractions that were designed at a time when high-end file systems managed less than 100MB. Today's highperformance file systems store 7 to 9 orders of magnitude more data, resulting in numbers of data items for which these metadata abstractions are inadequate, such as directory hierarchies unable to handle complex relationships among data. Users of file systems have attempted to work around these inadequacies by moving application-specific metadata management to relational databases to make metadata searchable. Splitting file system metadata management into two separate systems introduces inefficiencies and systems management problems.To address this problem, we propose QMDS: a file system metadata management service that integrates all file system metadata and uses a graph data model with attributes on nodes and edges. Our service uses a query language interface for file identification and attribute retrieval. We present our metadata management service design and architecture and study its performance using a text analysis benchmark application. Results from our QMDS prototype show the effectiveness of this approach. Compared to the use of a file system and relational database, the QMDS prototype shows superior performance for both ingest and query workloads.

show abstract

Richer File System Metadata Using Links and Attributes

Cited by 40 publications

References 12 publications

Exploring the design space of metadata-focused file management systems

Exploring the design space of metadata-focused file management systems

Reliability mechanisms for file systems using non-volatile memory as a metadata store

QMDS: a file system metadata management service supporting a graph data model-based query language

Contact Info

Product

Resources

About