Virtual device passthrough for high speed VM networking

Abstract

Supporting network I/O at high packet rates in virtual machines is fundamental for the deployment of Cloud data centers and Network Function Virtualization. Historically, SR-IOV and hardware passthrough were thought as the only viable solution to reduce the high cost of virtualization. In previous work [15] we showed how even plain device emulation can achieve VM-to-VM speeds of millions of packets per second (Mpps), though still at least 3 times slower than bare metal. In this paper, to fill this gap, we present ptnetmap, a virtual passthrough network device (based on the netmap framework). ptnetmap allows VMs to connect to any netmap port (physical devices, software switches, netmap pipes), conserving the speed and isolation of the native netmap system, and removing the constraints of hardware passthrough. Our work includes two key features not present in previous proposals: we provide a high speed path also to untrusted VMs, and do not require dedicated polling cores/threads, which is fundamental to achieve an efficient use of resources. Besides these features, our speed is also beyond previously published values. Running on top of ptnetmap, VMs can saturate a 10 Gbit link at 14.88 Mpps, talk at over 20 Mpps to untrusted VMs, and over 70 Mpps to trusted VMs. ptnetmap extends the netmap framework, and currently supports Linux and FreeBSD guests, and QEMU/KVM host. Support for bhyve/FreeBSD host is under development

Flexible virtual machine networking using netmap passthrough

“…To overcome these limitations, netmap passthrough [7] (ptnetmap) has been introduced as a technique to completely avoid hypervisor device emulation in the packet datapath, unblocking the full potential of netmap also for VM environments.…”

“…The original ptnetmap implementation [7] uses a slightly modified version of e1000/virtio-net guest drivers and hypervisor device emulators, where only the notification part of those devices is used, while the datapath (e.g. e1000 rings or virtio-net VQs) is completely bypassed.…”

“…In the past we have worked extensively in this space, addressing high packet rates, device independence and isolation. Our netmap framework [9], high performance VALE switch [11], hypervisor enhancements for high packet rates [12] and virtual passthrough solutions [7] led to excellent performance in terms of packet rates, complete device independence and extreme simplicity in terms of configuration. However, they do not match state of the art solutions such as VirtIO when dealing with bulk TCP traffic.…”

“…In this work we present ptnet, a solution for VM networking that uses the netmap data format as the underlying virtual device model. This lets us build very simple device drivers that natively support multi queue operation, and leverage our previous netmap passthrough work [7], to communicate with hypervisors at rates of tens of millions of packets per second. The contributions of this work, compared to [7], are (i) a new device model for paravirtualized devices, (ii) efficient access to passthrough ports even for legacy applications, (iii) a performance evaluation of our solution comparing it to that of VirtIO, and finally (iv) an actual implementation of both guest OS and hypervisor support of ptnet.…”

Accelerating Packet Processing in a Xen Environment with OpenDataPlane

“…Recently, Garzella et al presented pnetmap [17], a virtual passthrough network device that allows Virtual Machines to connect to any netmap port by removing the constraints of hardware passthrough. HAN et al designed a new hybrid software/hardware architecture called SoftNIC [18].…”