Improving the performance of log-structured file systems with adaptive methods

Matthews, Jeanna; Roselli, Drew; Costello, Adam M.; Wang, Randolph Y.; Anderson, Thomas E.

doi:10.1145/268998.266700

Cited by 69 publications

(31 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since its inception, LFS has been evaluated [3,27,35,38] and used [1,7,12,17] by many different groups. Much of the work done to improve both LFS and LFS cleaners is directly applicable to CVFS.…”

Section: Related Workmentioning

confidence: 99%

Metadata Efficiency in a Comprehensive Versioning File System

Soules¹,

Goodson²,

Strunk³

et al. 2002

View full text Add to dashboard Cite

A comprehensive versioning file system creates and retains a new file version for every WRITE Multiversion b-trees extend the per-entry key with a timestamp and keep current and historical entries in a single tree; CVFS uses this structure for directories, reducing the associated space requirements by 99%. Experiments with CVFS verify that its current-version performance is similar to that of non-versioning file systems. Although access to historical versions is slower than conventional versioning systems, checkpointing is shown to mitigate this effect.We thank the members and companies of the PDL Consortium (including EMC, Hewlett-Packard, Hitachi, IBM, Intel, Network Appliance, Panasas, Seagate, Sun, and Veritas) for their interest, insights, feedback, and support. We thank IBM and Intel for hardware grants supporting our research efforts. This material is based on research sponsored by the Air Force Research Laboratory, under agreement number F49620-01-1-0433. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory or the U.S. Government. This work is also partially funded by the DARPA/ITO OASIS program (Air Force contract number F30602-99-2-0539-AFRL). Craig Soules is supported by a USENIX Fellowship. Garth Goodson is supported by an IBM Fellowship. Report Documentation PageForm Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

show abstract

Section: Related Workmentioning

confidence: 99%

Metadata Efficiency in a Comprehensive Versioning File System

Soules¹,

Goodson²,

Strunk³

et al. 2002

View full text Add to dashboard Cite

show abstract

“…Matthews et al demonstrated how to overcome the segment cleaning problem at higher disk utilizations [6]. They present an adaptive cleaning mechanism that chooses either full segment cleaning, as in LFS, or their technique of hole-plugging.…”

Section: A Log-structured File Systems and Their Successorsmentioning

confidence: 99%

Classifying data to reduce long term data movement in shingled write disks

Jones

Amer

Miller

et al. 2015

2015 31st Symposium on Mass Storage Systems and Technologies (MSST)

View full text Add to dashboard Cite

Abstract-Shingled Magnetic Recording (SMR) is a means of increasing the density of hard drives that brings a new set of challenges. Due to the nature of SMR disks, updating in place is not an option. Holes left by invalidated data can only be filled if the entire band is reclaimed, and a poor band compaction algorithm could result in spending a lot of time moving blocks over the lifetime of the device. We propose using write frequency to separate blocks to reduce data movement and develop a band compaction algorithm that implements this heuristic. We demonstrate how our algorithm results in improved data management, resulting in an up to 47% reduction in required data movements when compared to naive approaches to band management.

show abstract

“…It offers bad performance if there are random writes, high utilization, and little idle time. Matthews et al [1997] investigate three techniques to improve the write performance of an LFS and one technique to improve the read performance. A simulation predicts the results of using the techniques.…”

Section: Cleaning (Reclamation Of Space) In a Log-structured File Systemmentioning

confidence: 99%

Online reorganization of databases

Sockut

Iyer

2009

ACM Comput. Surv.

View full text Add to dashboard Cite

In practice, any database management system sometimes needs reorganization, that is, a change in some aspect of the logical and/or physical arrangement of a database. In traditional practice, many types of reorganization have required denying access to a database (taking the database offline) during reorganization. Taking a database offline can be unacceptable for a highly available (24-hour) database, for example, a database serving electronic commerce or armed forces, or for a very large database. A solution is to reorganize online (concurrently with usage of the database, incrementally during users' activities, or interpretively). This article is a tutorial and survey on requirements, issues, and strategies for online reorganization. It analyzes the issues and then presents the strategies, which use the issues. The issues, most of which involve design trade-offs, include use of partitions, the locus of control for the process that reorganizes (a background process or users' activities), reorganization by copying to newly allocated storage (as opposed to reorganizing in place), use of differential files, references to data that has moved, performance, and activation of reorganization. The article surveys online strategies in three categories of reorganization. The first category, maintenance, involves restoring the physical arrangement of data instances without changing the database definition. This category includes restoration of clustering, reorganization of an index, rebalancing of parallel or distributed data, garbage collection for persistent storage, and cleaning (reclamation of space) in a logstructured file system. The second category involves changing the physical database definition; topics include construction of indexes, conversion between B + -trees and linear hash files, and redefinition (e.g., splitting) of partitions. The third category involves changing the logical database definition. Some examples are changing a column's data type, changing the inheritance hierarchy of object classes, and changing a relationship from one-to-many to many-to-many. The survey encompasses both research and commercial implementations, and this article points out several open research topics. As highly available or very large databases continue to become more common and more important in the world economy, the importance of online reorganization is likely to continue growing.

show abstract

Improving the performance of log-structured file systems with adaptive methods

Cited by 69 publications

References 21 publications

Metadata Efficiency in a Comprehensive Versioning File System

Metadata Efficiency in a Comprehensive Versioning File System

Classifying data to reduce long term data movement in shingled write disks

Online reorganization of databases

Contact Info

Product

Resources

About