The case for efficient file access pattern modeling

Abstract

Most modern I/O systems treat each file access independently. However, events in a computer system are driven by programs. Thus, accesses to files occur in consistent patterns and are by no means independent. The result is that modern I/O systems ignore useful information.Using traces of file system activity we show that file accesses are strongly correlated with preceding accesses. In fact, a simple last-successor model (one that predicts each file access will be followed by the same file that followed the last time it was accessed) successfully predicted the next file 72% of the time. We examine the ability of two previously proposed models for file access prediction in comparison to this baseline model and see a stark contrast in accuracy and high overheads in state space. We then enhance one of these models to address the issues of model space requirements. This new model is able to improve an additional 10% on the accuracy of the last-successor model, while working within a state space that is within a constant factor (relative to the number of files) of the lastsuccessor model. While this work was motivated by the use of file relationships for I/O prefetching, information regarding the likelihood of file access patterns has several other uses such as disk layout and file clustering for disconnected operation.

Omnisc'IO: A Grammar-Based Approach to Spatial and Temporal I/O Patterns Prediction

“…Kroeger and Long [30] study several spatial access pattern modelings techniques, some of which are inspired by text compression algorithms such as variants of PPM (prediction by partial matching). The contexts (or symbols) used in these models are parameters of system-level I/O calls (i.e., file name, offset, size, etc.).…”

Mining hidden mixture context with ADIOS-P to improve predictive pre-fetcher accuracy

“…Second, a dataset collected directly from the visualization software, VisIt, integrated with the ADIOS provenance module, ADIOS-P. The DFSTrace consists of 4 different datasets collected from the different machines, barber, ives, dvorak, and mozart, each of which has unique file access characteristics [19]; barber has the highest rate of system calls per second, dvorak has the highest percentage of write activity, ives the largest number of users, and mozart a typical desktop work-station.…”

“…In the file system ares, a series of Partitioned Context Modeling (PCM) based schemes [19][20][21] have been studied for sequence file prediction for IO prefetching. AMP [22] and TaP [23] have been proposed for sequence prediction.…”

A Novel Weighted-Graph-Based Grouping Algorithm for Metadata Prefetching

“…Based on the previous research, Long and coworkers developed a serial of successor-based predictive prefetching algorithms in their efforts to advance the prefetching accuracy while maintaining a reasonable performance gain [24], [25], [26]. The features of these predictors are summarized below as they are state of the art and are most relevant to our design.…”