QMDS: a file system metadata management service supporting a graph data model-based query language

Ames, Sasha; Gokhale, Maya; Maltzahn, Carlos

doi:10.1080/17445760.2012.658802

Cited by 12 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternative post-hierarchical file system architectures have been proposed (Gi ord et al, 1991;Dekeyser et al, 2008;A. Ames et al, 2005;S. Ames et al, 2013;Seltzer & Murphy, 2009;Rizzo, 2004;Padioleau, Sigonneau, & Ridoux, 2006) to avoid the problems posed by traditional file systems.…”

Section: Replacing Hierarchal File Systemsmentioning

confidence: 99%

See 1 more Smart Citation

iConference Proceedings 2017 Volume 1

Sterzer¹

2017

View full text Add to dashboard Cite

File systems (FS) are an essential part of operating systems in that they are responsible for storing and organising files and then retrieving those when needed. Because of the high capacity of modern storage devices and the growing number of files stored, the traditional FS model is no longer able to meet modern users' needs in terms of storing and retrieving files. So using metadata emerges as an e cacy solution for the limitations of file systems. In this paper we propose a new model dubbed VennTags to solve the FS problems. We do this by utilising the idea of overlapping the sets as in Venn diagram, and adopting DAG structure (instead of tree) to achieve that we have used tagging capability and exposed a query language at the level of the API. We evaluate the expressive power of VennTags model that shows its ability to resolve the FS limitations compared to other solutions.

show abstract

Section: Replacing Hierarchal File Systemsmentioning

confidence: 99%

“…Some of these proposals rely on a rich collection of file metadata rather than the hierarchical directory structure (A. Ames et al, 2005;S. Ames, Gokhale, & Maltzahn, 2013;Dekeyser, Watson, & Motrøn, 2008;Gi ord, Jouvelot, Sheldon, et al, 1991;Rizzo, 2004;Seltzer & Murphy, 2009).…”

Section: Introductionmentioning

confidence: 99%

iConference Proceedings 2017 Volume 1

Sterzer¹

2017

View full text Add to dashboard Cite

show abstract

“…A number of query languages either directly represent or can be used for property graphs [32]. These languages include SPARQL [33] for RDF data [34], GraphGrep for regular expression queries in graphs [35], Cypher for Neo4j graph databases [36], Gremlin from the Thinkerpop project [37], specific query language for provenance [38], Quasar for QMDS [39], and SQL (SemiJoin) for relational databases [40,41,42]. Among these languages, previous research [43] suggests that a low-level language such as Gremlin provides better performance because it allows users to manually control each traversal step in detail.…”

Section: Traversal Languagementioning

confidence: 99%

An asynchronous traversal engine for graph-based rich metadata management

et al. 2016

View full text Add to dashboard Cite

Rich metadata in high-performance computing (HPC) systems contains extended information about users, jobs, data files, and their relationships. Property graphs are a promising data model to represent heterogeneous rich metadata flexibly. Specifically, a property graph can use vertices to represent different entities and edges to record the relationships between vertices with unique annotations. The high-volume HPC use case, with millions of entities and relationships, naturally requires an out-of-core distributed property graph database, which must support live updates (to ingest production information in real time), low-latency point queries (for frequent metadata operations such as permission checking), and large-scale traversals (for provenance data mining). Among these needs, large-scale property graph traversals are particularly challenging for distributed graph storage systems. Most existing graph systems implement a "level-synchronous" breadth-first search algorithm that relies on global synchronization in each traversal step. This performs well in many problem domains; but a rich metadata management system is characterized by imbalanced graphs, long traversal lengths, and concurrent workloads, each of which has the potential to introduce or exacerbate stragglers (i.e., abnormally slow steps or servers in a graph traversal) that lead to low overall throughput for synchronous traversal algorithms. Previous research indicated that the straggler problem can be mitigated by using asynchronous traversal algorithms, and many graph-processing frameworks have successfully demonstrated this approach. Such systems require the graph to be loaded into a separate batch-processing framework instead of being iteratively accessed, however. In this work, we investigate a general asynchronous graph traversal engine that can operate atop a rich metadata graph in its native format. We outline a traversal-aware query language and key optimizations (traversal-affiliate caching and execution merging) necessary for efficient performance. We further explore the effect of different graph partitioning strategies on the traversal performance for both synchronous and asynchronous traversal engines. Our experiments show that the asynchronous graph traversal engine is more efficient than its synchronous counterpart in the case of HPC rich metadata processing, where more servers are involved and larger traversals are needed. Moreover, the asynchronous traversal engine is more adaptive to different graph partitioning strategies.

show abstract

“…The table discounts, among others, the "relationships" dimension; however, several proposals (LIFS [3,4], MDFS [10], and QMDS [5]) are focused precisely on that dimension.…”

Section: Mapping Existing Workmentioning

confidence: 99%

“…Proposals that have attempted to replace hierarchical directory structures altogether [3,5,7,10,12,17,19,24,26], in favour of relying on rich collections of metadata, can also be mapped in terms of their particular choices.…”

Section: Mapping Existing Workmentioning

confidence: 99%

Exploring the design space of metadata-focused file management systems

Watson

Dekeyser

Albadri

2017

Proceedings of the Australasian Computer Science Week Multiconference

View full text Add to dashboard Cite

Operating systems both old and new are reliant on the venerable hierarchical file system. For some time now, however, attempts have been made to either define new file systems or to bolt on applications that offer much improved functionality to attach and use metadata. This is because researchers have shown that traditional file systems are not able to meet users' needs in terms of organising large numbers of files effectively, and to support expeditious retrieval of those files when they are needed at a later time.Numerous proposals for post-hierarchical file management systems have been described in the literature; researchers focus on different dimensions of such systems in order to solve or reduce identified limitations. In some cases this leads to significantly different file system architectures, while in other cases new functionality is added on top of a traditional system through special purpose userspace applications. Orthogonally, some proposals focus on tags while others favour named attribute-value pairs. Still other choices are, seemingly, made in an ad hoc and often implicit manner. This paper investigates the different dimensions and associated choices that participate in the proposal of new approaches and that affect their ability to improve on current systems. The Cartesian product of those dimensions and options forms a large design space; we map some of the existing literature onto that design space and discuss approaches to evaluate new proposals.

show abstract

QMDS: a file system metadata management service supporting a graph data model-based query language

Cited by 12 publications

References 19 publications

iConference Proceedings 2017 Volume 1

iConference Proceedings 2017 Volume 1

An asynchronous traversal engine for graph-based rich metadata management

Exploring the design space of metadata-focused file management systems

Contact Info

Product

Resources

About