A Hardware/Software Approach for Mitigating Performance Interference Effects in Virtualized Environments Using SR-IOV

Abstract

Abstract-Single Root I/O Virtualization (SR-IOV) is an extension to the PCI Express (PCIe) standard that allows virtual machines (VMs) to directly access shared I/O devices without host involvement. This enabled SR-IOV to become the best-performing solution for virtual I/O to date, which lead to its commercial adoption, e.g., in the Amazon EC2. On the downside, a malicious VM can exploit the direct access to an SR-IOV device by flooding it with PCIe packets. This results in a congestion on the PCIe interconnect, which leads to performance interference effects between the malicious VM, concurrent VMs and even the host.In this paper, we present a hardware/software approach that detects and mitigates such Denial-of-Service (DoS) attacks. On the hardware side, we propose monitoring extensions within SR-IOV devices that distinguish legal device use from malicious device use by observing the rate of incoming PCIe transactions at VM granularity. Malicious VMs are reported to the host via interrupts. On the software side, performance interference effects can then be mitigated by dynamically adjusting the host's scheduling of the malicious VM or even shutting it down.We implement a prototype with a commercial off-theshelf SR-IOV Ethernet controller and an FPGA board. On it, we demonstrate that appropriate scheduling of malicious VMs successfully mitigates interference effects for three cloudrelevant benchmarks. For example, Memcached is restored to 99.4% of baseline performance (compared to 61.8% without our extensions). In contrast to QoS features proposed in the PCIe 3.0 standard, our solution is more flexible. Additionally, it can be realized as an add-on to existing misuse detection hardware like the Intel Malicious Driver Detection (MDD).

An I/O Virtualization Framework With I/O-Related Memory Contention Control for Real-Time Systems

“…As in our proposal, [37] uses the QoS-400 regulators, but with a different objective, i.e., regulating the memory interference generated by DMA-based inter-VM data transfers. Richter et al [38] implemented a scheduling mechanism for mitigating the interference generated by malicious VMs using the SRIO-V technology (not present in the Ultrascale+), and without considering real-time requirements and I/O-related memory contention. Pu et al [39] presented a thorough experimental comparison of average-case performance metrics of different solutions to run CPU-bound and I/O-bound workloads in different VMs for cloud computing.…”

FlexVF: Adaptive network device services in a virtualized environment

“…virtual disk instance), and adapts its performance by adjusting its I/O rate and throughput as the utilization reaches a certain threshold. Richter et al [25] describe an anti-DoS mechanism for preventing packet flooding on SR-IOV-enabled guests by counting PCI Express operations and throttling any malicious guest.…”

Pacer: Comprehensive Network Side-Channel Mitigation in the Cloud

“…Some prior work [51,45,17] uses techniques similar to those used for network side channel mitigation to performance isolate co-located tenants. Richter et al [52] propose to performance-isolate co-located tenants by modifying the NIC firmware. Pacer's traffic shaping can be similarly implemented in the NIC.…”