NFS over RDMA

Callaghan, Brent; Lingutla-Raj, Theresa; Chiu, Alex; Staubach, Peter; Asad, Omer

doi:10.1145/944747.944753

Cited by 31 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RDMA-based MPI implementations ( [27], [19], [20]) show a significant improvement in performance and has been widely used in the HPC community. Other projects have proposed design and implementation of HBase [13], NFS [4], memcached [15] and HDFS [14] replacing the communication layer with RDMA.…”

Section: Related Workmentioning

confidence: 99%

Exploring Low-Latency Interconnect for Scaling Out Software Routers

Kim

Moon

2016

2016 2nd IEEE International Workshop on High-Performance Interconnection Networks in the Exascale and Big-Data Era (HiPINEB)

View full text Add to dashboard Cite

We propose and evaluate RoCE (RDMA over Converged Ethernet) as a low-latency back-plane for horizontally scaled software router nodes. By exploring combinations of design choices in developing internal fabric for software routers, we select a set of parameters and packet I/O APIs that yield the lowest latency and highest throughput. Using the optimal settings derived, we measure and compare latency and throughput of an RoCE interconnect against Ethernet using a high-performance userspace network driver (Intel DPDK). Our comparison shows that RoCE keeps low latency in all packet sizes while it has throughput penalties for network workloads (e.g., small packet sizes). To mitigate throughput penalties imposed by guaranteeing low latency, we suggest a hardware-assisted, batched forwarding scheme based on scatter-and-gather functionality of RDMAcapable NICs. When forwarding ingress network packets, our scheme achieves comparable to or higher throughput then Ethernet at the cost of several microseconds of latency.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploring Low-Latency Interconnect for Scaling Out Software Routers

Kim

Moon

2016

2016 2nd IEEE International Workshop on High-Performance Interconnection Networks in the Exascale and Big-Data Era (HiPINEB)

View full text Add to dashboard Cite

show abstract

“…al. designed an initial implementation of RPC over RDMA based on RDMA read [1] for NFS, as shown in Fig. 1 (a).…”

Section: B Nfs Over Rdma Overviewmentioning

confidence: 99%

“…Two basic designs of sunRPC over RDMA, namely Read-Read design and Read-Write design, are proposed by [1] and [2] respectively. In both designs, for the RPC Call and Reply, inline protocol using RDMA Send is proposed; for bulk data transfer, RDMA protocol, in which the server RDMA read (or write) data from (or to) the client, is proposed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Early Experiences with Write-Write Design of NFS over RDMA

Zhang

Huo

et al. 2009

2009 IEEE International Conference on Networking, Architecture, and Storage

View full text Add to dashboard Cite

The Network File System (NFS) protocol, as the de facto standard for sharing files in a distributed environment, has deployed Infiniband as the underlying transport of sunRPC, namely NFS over RDMA. In the current Read-Write design of NFS over RDMA, NFS write performance is limited for not fully utilizing the features of Infiniband. In this paper, we take on the challenge of enhancing the write performance of NFS. We propose and evaluate a new design of sunRPC over RDMA, namely Write-Write design. To guarantee the security of our design, we propose an HCA-based memory protection extension of Infiniband. Evaluations show that our WriteWrite design increases the kernel-to-kernel RPC bandwidth by 15~27%. In real disk test, our Write-Write design gains 15%~22% in multi-client benchmarks compared with the Read-Write design.

show abstract

“…al. [3] then provided an initial implementation NFS over RDMA (NFSoRDMA) on Solaris. A team at the Ohio State University further completed the design and development of NFSoRDMA on Open Solaris for performance enhancement, compliant to IETF specification [4].…”

Section: Related Workmentioning

confidence: 99%

Performance of RDMA-capable storage protocols on wide-area network

Rao

Wyckoff

et al. 2008

2008 3rd Petascale Data Storage Workshop

View full text Add to dashboard Cite

Because of its high throughput, low CPU utilization, and direct data placement, RDMA (Remote Direct Memory Access) has been adopted for transport in a number of storage protocols, such as NFS and iSCSI. In this presentation, we provide a performance evaluation of RDMA-based NFS and iSCSI on Wide-Area Network (WAN). We show that these protocols, though benefit from RDMA on Local Area Network (LAN) and on WAN of short distance, are faced with a number of challenges to achieve good performance on long distance WAN. This is because of (a) the low performance of RDMA reads on WAN, (b) the small 4KB chunks used in NFS over RDMA, and(c)the lack of RDMA capability in handling discontinuous data. Our experimental results document the performance behavior of these RDMA-based storage protocols on WAN.

show abstract

NFS over RDMA

Cited by 31 publications

References 6 publications

Exploring Low-Latency Interconnect for Scaling Out Software Routers

Exploring Low-Latency Interconnect for Scaling Out Software Routers

Early Experiences with Write-Write Design of NFS over RDMA

Performance of RDMA-capable storage protocols on wide-area network

Contact Info

Product

Resources

About