LPCC

Qian, Yingjin; Li, Xi; Ihara, Shuichi; Dilger, Andreas; Thomaz, Carlos Eduardo; Wang, Shilong; Cheng, Wu; Li, Chunyan; Zeng, Lingfang; Wang, Fang; Feng, Dan; Süß, Tim; Brinkmann, André

doi:10.1145/3295500.3356139

Cited by 19 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…File prefetching also has been researched in the multi-tiered storage of high performance computing (HPC) systems [27,28]. Alturkestani et al [27] proposed multilayered buffer storage (MLBS), which is a data management method in three layered storages of super computers for HPC.…”

Section: Multi-tiered Storage Systemsmentioning

confidence: 99%

“…Selection of prefetched files is based on whether the application processes in LIFO or FIFO. In the research of Qian et al [28], a HPC client of hierarchical storage management (HSM) architecture uses its SSD as cache. Files to be cached are selected using file access information such as open/close events, client NID, and job ID.…”

Section: Multi-tiered Storage Systemsmentioning

confidence: 99%

“…Some commercial cloud file synchronization services [8][9][10][11] provide the functionality to change the online and local modes of files, but this function only allows users to manually change the mode of selected files on an on-demand basis. There are also cloud storage and distributed file systems [12][13][14][15][16][17][18][19][20] and other multi-tiered storage systems [21][22][23][24][25][26][27][28][29][30] that use similar approaches to the proposed file mode transition mechanism by prefetching or caching files among storages. However, as they focus more on the performance of file access in costly high-layered storages, cache hit ratio or prediction accuracy are the most important performance metric.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization Service

Lim

2023

Applied Sciences

View full text Add to dashboard Cite

In this paper, we propose an active file mode change mechanism in which the file synchronization system of cloud storage automatically changes files in a directory of a client to the online or local mode by considering tradeoff between local storage usage and file access time according to directory activation ratio. When the directory activation ratio rises above a certain threshold, the proposed active file mode change mechanism selects online mode files in this directory based on file access delay time and local storage usage and changes them to the local mode to reduce file access delay of active IoT clients. When the directory activation ratio falls below the threshold, the active file mode change mechanism selects the local mode files based on the last access time and local storage usage and changes them to the online mode to increase available local storage. Experimental results show that the proposed active file mode change mechanism can control when and how much the client can reduce and increase the local storage usage and the file access delay by changing file mode parameters according to the requirements of various IoT devices.

show abstract

Section: Multi-tiered Storage Systemsmentioning

confidence: 99%

Section: Multi-tiered Storage Systemsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization Service

Lim

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Burst buffer file systems furthermore use node-local file systems to store data which is accessed in the context of a single application run or for longer-running workflows [10,65,70,71]. Also parallel file system like Lustre can directly benefit from performance advantages of local file system, both for object storage devices and for node-local client caching [53].…”

Section: Related Work and Motivationmentioning

confidence: 99%

Simurgh

Moti

Schimmelpfennig

Salkhordeh

et al. 2021

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

The availability of non-volatile main memory (NVMM) has started a new era for storage systems and NVMM specific file systems can support extremely high data and metadata rates, which are required by many HPC and data-intensive applications. Scaling metadata performance within NVMM file systems is nevertheless often restricted by the Linux kernel storage stack, while simply moving metadata management to the user space can compromise security or flexibility.This paper introduces Simurgh, a hardware-assisted user space file system with decentralized metadata management that allows secure metadata updates from within user space. Simurgh guarantees consistency, durability, and ordering of updates without sacrificing scalability. Security is enforced by only allowing NVMM access from protected user space functions, which can be implemented through two proposed instructions. Comparisons with other NVMM file systems show that Simurgh improves metadata performance up to 18x and application performance up to 89% compared to the second-fastest file system. CCS CONCEPTS• Information systems → Storage class memory; Phase change memory; • Software and its engineering → File systems management.

show abstract

Ad Hoc File Systems for High-Performance Computing

Brinkmann

Mohror

et al. 2020

J. Comput. Sci. Technol.

Self Cite

View full text Add to dashboard Cite

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include node-local SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 "Challenges and Opportunities of User-Level File Systems for HPC" and discusses application scenarios as well as design strategies for ad hoc file systems using nodelocal storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

show abstract

LPCC

Cited by 19 publications

References 20 publications

An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization Service

An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization Service

Simurgh

Ad Hoc File Systems for High-Performance Computing

Contact Info

Product

Resources

About