uMMAP-IO: User-Level Memory-Mapped I/O for HPC

Rivas-Gomez, Sergio; Fanfarillo, Alessandro; Valat, S.; Laferriere, Christophe; Couvée, Philippe; Narasimhamurthy, Sai

doi:10.1109/hipc.2019.00051

Cited by 3 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, memorymapped I/O, i.e., mmap(), suffers from lack of scalability for multithreaded applications [4], and lacks the flexibility for application-specific optimizations [2]. These limitations motivated the design of userspace memory-mapped I/O libraries [3], [6], [9]. Userspace memory-mapped I/O transfers page fault signals to the userspace, enabling applicationspecific optimizations while avoiding kernel-space bottlenecks.…”

Section: A Userspace Memory-mapped I/omentioning

confidence: 99%

“…Userspace paging was proposed to overcome the performance limitations incurred by system-level mmap() [2], [4]. The most recent userspace memory-mapped I/O tools include Userland CO-PAGER [5], uMMAP-IO [3], and UMap [2]. The advantages of UMap include higher flexibility for applicationspecific tuning by exposing a larger number of paging parameters.…”

Section: Related Workmentioning

confidence: 99%

“…Fast storage technology can significantly improve the I/O performance of data-intensive applications, but at the expense of programming complexity to handle a complex HPC storage hierarchy [2], [3]. To address this challenge, data processing tools leverage memory-mapped I/O [4].…”

Section: Introductionmentioning

confidence: 99%

“…To address these limitations, past research has studied handling paging management in userspace [2], [3], [5], [6]. § Authors have equally contributed to this work.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

AutoPager: Auto-tuning Memory-Mapped I/O Parameters in Userspace

Youssef

Shah

Gokhale

et al. 2022

2022 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

The exponential growth in dataset sizes has shifted the bottleneck of high-performance data analytics from the compute subsystem to the memory and storage subsystems. This bottleneck has led to the proliferation of non-volatile memory (NVM). To bridge the performance gap between the Linux I/O subsystem and NVM, userspace memory-mapped I/O enables application-specific I/O optimizations. Specifically, UMap, an open-source userspace memory-mapping tool, exposes tunable paging parameters to application users, such as page size and degree of paging concurrency. Tuning these parameters is computationally intractable due to the vast search space and the cost of evaluating each parameter combination. To address this challenge, we present AUTOPAGER, a tool for autotuning userspace paging parameters. Our evaluation, using five data-intensive applications with UMap, shows that AUTOPAGER automatically achieves comparable performance to exhaustive tuning with 10ˆless tuning overhead. and 16.3ˆand 1.52ŝ peedup over UMap with default parameters and UMap with page-size only tuning, respectively.

show abstract

Section: A Userspace Memory-mapped I/omentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…To address these limitations, past research has studied handling paging management in userspace [2], [3], [5], [6]. § Authors have equally contributed to this work.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AutoPager: Auto-tuning Memory-Mapped I/O Parameters in Userspace

Youssef

Shah

Gokhale

et al. 2022

2022 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

show abstract

“…These tools have either used the system's mmap without changes or optimized versions of it using different techniques. These techniques include optimized kernel-level mmap path [2], [6], userspace memory-mapped I/O [7], [18], [19], or a gradual step-away from using mmap into implementing userspace buffers to avoid mmap's bottlenecks [11], [20]. Kernel-space mmap optimizations require changing or updating operating system modules.…”

Section: Related Workmentioning

confidence: 99%

Optimizing Performance and Storage of Memory-Mapped Persistent Data Structures

Youssef

Islam

Iwabuchi

et al. 2022

2022 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

Persistent data structures represent a core component of high-performance data analytics. Multiple data processing systems persist data structures using memory-mapped files. Memory-mapped file I/O provides a productive and unified programming interface to different types of storage systems. However, it suffers from multiple limitations, including performance bottlenecks caused by system-wide configurations and a lack of support for efficient incremental versioning. Therefore, many such systems only support versioning via full-copy snapshots, resulting in poor performance and storage capacity bottlenecks. To address these limitations, we present Privateer 2.0, a virtual memory and storage interface that optimizes performance and storage capacity for versioned persistent data structures. Privateer 2.0 improves over the previous version by supporting userspace virtual memory management and block compression. We integrated Privateer 2.0 into Metall, a C++ persistent data structure allocator, and LMDB, a widely-used key-value store database. Privateer 2.0 yielded up to 7.5× speedup and up to 300× storage space reduction for Metall incremental snapshots and 1.25× speedup with 11.7× storage space reduction for LMDB incremental snapshots.

show abstract