LITE Kernel RDMA Support for Datacenter Applications

Tsai, Shin-Yeh; Zhang, Yiying

doi:10.1145/3132747.3132762

Cited by 110 publications

(37 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of them rely on the native network stack of TCP/IP [16][17][18], [22], [25][26], whereas others either exploit such features as RDMA [11][12][13][14][15], [19][20][21] or adopt new network protocols [53]. In particular, RDMA and its variants [11][12][13][14][15] expose remote memory to users directly via a collection of programming APIs, allowing users to manipulate remote memory by means of considerable application modifications. However, users have to deal with an RDMA low-level abstraction [12][13], [46], often involving steep learning and high programming skills in order to unleash the full potential of RDMA.…”

Section: Programming Remote Memorymentioning

confidence: 99%

“…In particular, RDMA and its variants [11][12][13][14][15] expose remote memory to users directly via a collection of programming APIs, allowing users to manipulate remote memory by means of considerable application modifications. However, users have to deal with an RDMA low-level abstraction [12][13], [46], often involving steep learning and high programming skills in order to unleash the full potential of RDMA. In addition, RDMA library is available only in a few limited programming languages.…”

Section: Programming Remote Memorymentioning

confidence: 99%

“…Swapping moves (1) cold memory pages from local memory to 'swap-out partitions' in storage and (2) miss pages from 'swapout partitions' back to local memory, in order to expand address space beyond system's actual physical memory. However, it relies on asynchronous (for swapping pages out) and read-ahead (for swapping miss pages in) procedures to hide lengthy disk seek and write/read times [12][13], [44][45], [67][68][69]. Under memory paging to/from remote memory (of a connected machine), however, care must be taken to fully benefit from much faster DRAM for a low latency [41][42][43][44][45].…”

Section: Remote Memory Pagingmentioning

confidence: 99%

See 2 more Smart Citations

Cooperative Memory Expansion via OS Kernel Support for Networked Computing Systems

Srinuan

Yuan

Tzeng

2020

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

The growing popularity of in-memory computing for bigdata analytics often causes performance bottlenecks to memory subsystem resided in operating systems (OS). This article purposes cooperative memory expansion (COMEX), an OS kernel extension. COMEX establishes a stable pool of memory collectively across nodes in a cluster and enhances OS's memory subsystem for memory aggregation from connected machines by allowing process's page table to track remote memory page frames without programmer effort or modifications to application codes. COMEX employs Remote Direct Memory Access (RDMA) for low-latency data transfer with destination kernel bypassed and does not rely on an old design of the I/O block subsystem usually adopted by all known remote paging. COMEX fits soundly in the emerging system design approach of resource disaggregation which breaks hard walls between server-centric machines into a new design paradigm of separated resource pools. The new architecture facilitates both system scaling-up and scaling-out, also eliminates imbalance resources existing in datacenters. We have implemented COMEX based on Linux kernel 3.10.87 and deployed on our 32 networked servers. Performance evaluation results under ten applications from two benchmark suites reveal the speedup of up to 170 times when application execution footprints are 10 times larger than available system memory. Index Terms-I/O block devices, memory management, networked computer systems, operating systems (OS), page tables, remote direct memory access (RDMA).

show abstract

Section: Programming Remote Memorymentioning

confidence: 99%

Section: Programming Remote Memorymentioning

confidence: 99%

Section: Remote Memory Pagingmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative Memory Expansion via OS Kernel Support for Networked Computing Systems

Srinuan

Yuan

Tzeng

2020

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…RackMem DVS supports several remote and local storage backends. The RDMA backend is based on LITE [52] and provides a high-performance and easy-to-use networking stack for remote memorybacked storage. RackMem's networking module implements abstractions for remote memory, state sharing, and coordinated I/O polling to provide programmability, scalability for multiple connections, and efficient I/O completion.…”

Section: Rackmem Dvs Modulementioning

confidence: 99%

RackMem

Changyeon

Kim

Geng

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

High-performance computing (HPC) clusters suffer from an overall low memory utilization that is caused by the node-centric memory allocation combined with the variable memory requirements of HPC workloads. The recent provisioning of nodes with terabytes of memory to accommodate workloads with extreme peak memory requirements further exacerbates the problem. Memory disaggregation is viewed as a promising remedy to increase overall resource utilization and enable cost-effective up-scaling and efficient operation of HPC clusters, however, the overhead of demand paging in virtual memory management has so far hindered performant implementations. To overcome these limitations, this work presents RackMem, an efficient implementation of disaggregated memory for rack scale computing. RackMem addresses the shortcomings of Linux's demand paging algorithm and automatically adapts to the memory access patterns of individual processes to minimize the inherent overhead of remote memory accesses. Evaluated on a cluster with an Infiniband interconnect, RackMem outperforms the state-of-the-art RDMA implementation and Linux's virtual memory paging by a significant margin. RackMem's custom demand paging implementation achieves a tail latency that is two orders of magnitude better than that of the Linux kernel. Compared to the state-of-the-art remote paging solution, RackMem achieves a 28% higher throughput and a 44% lower tail latency for a wide variety of real-world workloads. CCS CONCEPTS • Computer systems organization → Cloud computing; • Software and its engineering → Virtual memory; Distributed memory; Cloud computing.

show abstract

“…We are starting to see RDMA deployments [19], but it requires loss-less network fabric and individual systems closely tied with low-level hardware details [15,31,40]. LITE [65] remedies the latter problem by kernel-level abstraction. This could help PASTE support RDMA, providing a higher-level interface to integrate with NVMMs and to be transparent to TCP/IP networking.…”

Section: Kernel-bypass Networkingmentioning

confidence: 99%

Paste

Honda

Eggert

Santry

2016

Proceedings of the 15th ACM Workshop on Hot Topics in Networks

View full text Add to dashboard Cite

Non-Volatile Main Memory (NVMM) devices have been integrated into general-purpose operating systems through familiar file-based interfaces, providing efficient bytegranularity access by bypassing page caches. To leverage the unique advantages of these high-performance media, the storage stack is migrating from the kernel into user-space. However, application performance remains fundamentally limited unless network stacks explicitly integrate these new storage media and follow the migration of storage stacks into user-space. Moreover, we argue that the storage and the network stacks must be considered together when being designed for NVMM. This requires a thoroughly new network stack design, including low-level buffer management and APIs. We propose PASTE, a new network programming interface for NVMM. It supports familiar abstractionsincluding busy-polling, blocking, protection, and run-tocompletion-with standard network protocols such as TCP and UDP. By operating directly on NVMM, it can be closely integrated with the persistence layer of applications. Once data is DMA'ed from a network interface card to host memory (NVMM), it never needs to be copied again-even for persistence. We demonstrate the general applicability of PASTE by implementing two popular persistent data structures: a write-ahead log and a B+ tree. We further apply PASTE to three applications: Redis, a popular persistent key-value store, pKVS, our HTTP-based key value store and the logging component of a software switch, demonstrating that PASTE not only accelerates networked storage but also enables conventional networking functions to support new features.

show abstract

LITE Kernel RDMA Support for Datacenter Applications

Cited by 110 publications

References 40 publications

Cooperative Memory Expansion via OS Kernel Support for Networked Computing Systems

Cooperative Memory Expansion via OS Kernel Support for Networked Computing Systems

RackMem

Paste

Contact Info

Product

Resources

About