Efficient memory virtualization for Cross-ISA system mode emulation

Abstract

Cross-ISA system-mode emulation has many important applications. For example, Cross-ISA system-mode emulation helps computer architects and OS developers trace and debug kernel execution-flow efficiently by emulating a slower platform (such as ARM) on a more powerful plat-form (such as an x86 machine). Cross-ISA system-mode emulation also enables workload consolidation in data centers with platforms of different instruction-set architectures (ISAs). However, system-mode emulation is much slower. One major overhead in system-mode emulation is the multi-level memory address translation that maps guest virtual address to host physical address. Shadow page tables (SPT) have been used to reduce such overheads, but primarily for same-ISA virtualization. In this paper we propose a novel approach called embedded shadow page tables (ESPT). EPST embeds a shadow page table into the address space of a cross-ISA dynamic binary translation (DBT) and uses hardware memory management unit in the CPU to translate memory addresses, instead of software translation in a current DBT emulator like QEMU. We also use the larger address space on modern 64-bit CPUs to accommodate our DBT emulator so that it will not interfere with the guest operating system. We incorporate our new scheme into QEMU, a popular, retargetable cross-ISA system emulator. SPEC CINT2006 benchmark results indicate that our technique achieves an average speedup of 1.51 times in system mode when emulating ARM on x86, and a 1.59 times speedup for emulating IA32 on x86_64.

HSPT

“…So frequently switching back and forth between SPT and HPT is needed. This excessive overhead of page table switching is intolerable, therefore, ESPT proposes to embed the SPT into the HPT to avoid such table switching [14]. In ESPT, LKMs are used to create and manage embedded shadow page entries in HPT (as shown in Figure 1(c)).…”

“…SPEC CINT2006 benchmark results indicate that ESPT achieves an average speed up of 1.51X in system mode when emulating ARM-32 on x86-64 and a 1.59X for emulating IA32 on x86-64 against the original software MMU [14].…”

“…One challenge in system virtualization is the large memory virtualization overhead where the virtual address of the guest must be mapped to the guest physical address, then the guest physical address be mapped to the host physical address during program execution (as shown in Figure 1(a)). For example, on average QEMU spends 23%∼43% of the total execution time in memory translation for SPEC CINT2006 benchmarks when running in system mode emulation [14]. So optimizations to minimize such memory virtualization overhead are the key to enhance the performance of the system-level emulator.…”

“…However, software MMU is one major contributor to sluggish simulation. Some recent approaches, such as the Embedded Shadow Page Table (ESPT) [14] exploits a combination of software MMU and hardware MMU to significantly cut down the memory virtualization cost. ESPT utilizes the larger address space on modern 64-bit processors and creates a loadable kernel module (LKM) to embed the shadow page entries into the host page table.…”

“…This table can be used by the hardware walker to resolve TLB misses. In [14], ESPT has achieved significant speed up (>50%).…”