Second Level Address Translation
Second Level Address Translation (SLAT), also known as nested paging, is a hardware-assisted virtualization technology which makes it possible to avoid the overhead associated with software-managed shadow page tables.
AMD has supported SLAT through the Rapid Virtualization Indexing (RVI) technology since the introduction of its third-generation Opteron processors (code name Barcelona). Intel's implementation of SLAT, known as Extended Page Table (EPT), was introduced in the Nehalem microarchitecture found in certain Core i7, Core i5, and Core i3 processors.
ARM's virtualization extensions support SLAT, known as Stage-2 page-tables provided by a Stage-2 MMU. The guest uses the Stage-1 MMU. Support was added as optional in the ARMv7ve architecture and is also supported in the ARMv8 (32-bit and 64-bit) architectures.
Modern processors use the concepts of physical memory and virtual memory; running processes use virtual addresses and when an instruction requests access to memory, the processor translates the virtual address to a physical address using a page table or translation lookaside buffer (TLB). When running a virtual system, it has allocated virtual memory of the host system that serves as a physical memory for the guest system, and the same process of address translation goes on also within the guest system. This increases the cost of memory access since the address translation needs to be performed twice – once inside the guest system (using software-emulated shadow page table), and once inside the host system (using hardware page table).
In order to make this translation more efficient, processor vendors implemented technologies commonly called SLAT. By treating each guest-physical address as a host-virtual address, a slight extension of the hardware used to walk a non-virtualized page table (now the guest page table) can walk the host page table. With multilevel page tables the host page table can be viewed conceptually as nested within the guest page table. A hardware page table walker can treat the additional translation layer almost like adding levels to the page table.
Using SLAT and multilevel page tables, the number of levels needed to be walked to find the translation doubles when the guest-physical address is the same size as the guest-virtual address and the same size pages are used. This increases the importance of caching values from intermediate levels of the host and guest page tables. It is also helpful to use large pages in the host page tables to reduce the number of levels (e.g., in x86-64, using 2 MB pages removes one level in the page table). Since memory is typically allocated to virtual machines at coarse granularity, using large pages for guest-physical translation is an obvious optimization, reducing the depth of look-ups and the memory required for host page tables.
Rapid Virtualization Indexing
Rapid Virtualization Indexing (RVI), known as Nested Page Tables (NPT) during its development, is an AMD second generation hardware-assisted virtualization technology for the processor memory management unit (MMU).
A VMware research paper found that RVI offers up to 42% gains in performance compared with software-only (shadow page table) implementation. Tests conducted by Red Hat showed a doubling in performance for OLTP benchmarks.
Extended Page Tables
Extended Page Tables (EPT) is an Intel second-generation x86 virtualization technology for the memory management unit (MMU). EPT support is found in Intel's Core i3, Core i5, Core i7 and Core i9 CPUs, among others. It is also found in some newer VIA CPUs.
According to a VMware evaluation paper: "EPT provides performance gains of up to 48% for MMU-intensive benchmarks and up to 600% for MMU-intensive microbenchmarks", although it can actually cause code to run slower than a software implementation in some corner cases.
Stage-2 page-table support is present in ARM processors that implement exception level 2 (EL2).
Mode Based Execution Control
Mode Based Execution Control (MBE) is an extension to x86 SLAT implementations first available in Intel Kaby Lake and AMD Zen 2 CPUs. The extension extends the execute bit in the extended page table (guest page table) into 2 bits - one for user execute, and one for supervisor execute.
MBE was introduced to speed up guest usermode unsigned code execution with kernelmode code integrity enforcement. Under this configuration, unsigned code pages can be marked as execute under usermode, but must be marked as no-execute under kernelmode. To maintain integrity by ensuring all guest kernelmode executable code are signed even when the guest kernel is compromised, the guest kernel does not have permission to modify the execute bit of any memory pages. Modification of the execute bit, or switching of the guest page table which contains the execute bit, is delegated to a higher privileged entity, in this case the host hypervisor. Without MBE, each entrance from unsigned usermode execution to signed kernelmode execution must be accompanied by a VM exit to the hypervisor to perform a switch to the kernelmode page table. On the reverse operation, an exit from signed kernelmode to unsigned usermode must be accompanied by a VM exit to perform another page table switch. VM exits significantly impact code execution performance. With MBE, the same page table can be shared between unsigned usermode code and signed kernelmode code, with two sets of execute permission depending on the execution context. VM exits are no longer necessary when execution context switches between unsigned usermode and signed kernel mode.
Support in software
Hypervisors that support SLAT include the following:
- Hyper-V for Windows Server 2008 R2, Windows 8 and later. The Windows 8 (and later Microsoft Windows) Hyper-V actually requires SLAT.
- Hypervisor.framework, a native macOS hypervisor, available since macOS 10.10
- KVM, since version 2.6.26 of the Linux kernel mainline
- Parallels Desktop for Mac, since version 5
- VirtualBox, since version 2.0.0
- VMware ESX, since version 3.5
- VMware Workstation. The VMware Workstation 14 (and later VMware Workstation) actually requires SLAT.
- Xen, since version 3.2.0
- Qubes OS — SLAT mandatory
- bhyve — SLAT mandatory and slated to remain mandatory
- vmm, a native hypervisor on OpenBSD — SLAT mandatory
- ACRN, an open-source lightweight hypervisor, built with real-time and safety-criticality in mind, optimized for IoT and Edge usages.
Some of the above hypervisors actually require SLAT in order to work at all (not just faster) as they do not implement a software shadow page table; the list is not fully updated to reflect that.
- AMD-V (codename Pacifica) – the first-generation AMD hardware virtualization support
- Page table
- "Rapid Virtualization Indexing with Windows Server 2008 R2 Hyper-V | The Virtualization Blog". Blogs.amd.com. 2009-03-23. Retrieved 2010-05-16.
- "AMD-V Nested Paging" (PDF). July 2008. Archived from the original (PDF) on 2012-09-05. Retrieved 2013-12-11.
- "Performance Evaluation of AMD RVI Hardware Assist" (PDF). Retrieved 2010-05-16.
- "Red Hat Magazine | Red Hat Enterprise Linux 5.1 utilizes nested paging on AMD Barcelona Processor to improve performance of virtualized guests". Magazine.redhat.com. 2007-11-20. Retrieved 2010-05-16.
- "VMware engineer praises AMD's Nested Page Tables". Searchservervirtualization.techtarget.com. 2008-07-21. Retrieved 2010-05-16.
- "Intel Virtualization Technology List". Ark.intel.com. Retrieved 2014-02-17.
- "Intel added unrestricted guest mode on Westmere micro-architecture and later Intel CPUs, it uses EPT to translate guest physical address access to host physical address. With this mode, VMEnter without enable paging is allowed."
- "Intel 64 and IA-32 Architectures Developer's Manual, Vol. 3C" (PDF). Intel. Retrieved 13 December 2015.
If the 'unrestricted guest' VM-execution control is 1, the 'enable EPT' VM-execution control must also be 1.
- Performance Evaluation of Intel EPT Hardware Assist
- Mulnix, David L. "Intel Xeon Processor Scalable Family Technical Overview". intel. Retrieved 3 September 2021.
- Analysis of the Attack Surface of Windows 10 Virtualization-based Security
- Arkley, Brent. "The potential performance Impact of Device Guard (HVCI)". Borec's Legacy meets Modern Device Management Blog. Retrieved 3 September 2021.
- "AMD-V Rapid Virtualization Indexing and Windows Server 2008 R2 Hyper-V Second Level Address Translation". Doing IT Virtual. Retrieved 2010-05-16.
- Bott, Ed (2011-12-08). "Does your PC have what it takes to run Windows 8's Hyper-V?". ZDNet. Retrieved 2014-02-17.
- "Support & Drivers". Retrieved 13 December 2015.
- "Hypervisor | Apple Developer Documentation".
- "Kernel Newbies: Linux 2 6 26".
- Sheng Yang (2008-06-12). "Extending KVM with new Intel Virtualization technology" (PDF). linux-kvm.org. KVM Forum. Archived from the original (PDF) on 2014-03-27. Retrieved 2013-03-17.
- Inc, Parallels. "KB Parallels: What's new in Parallels Desktop 5 for Mac". kb.parallels.com. Retrieved 2016-04-12.
- "Changelog for VirtualBox 2.0". Archived from the original on 2014-10-22.
- liz. "VMware Workstation 14 Pro Release Notes". docs.vmware.com. Retrieved 2020-11-19.
- "Benchmarks: Xen 3.2.0 on AMD Quad-Core Opteron with RVI". 2008-06-15. Retrieved 2011-05-13.
- "Hardware Compatibility List (HCL)". Qubes OS. Retrieved 2020-01-06.
- Implementation of a BIOS emulation support for BHyVe: A BSD Hypervisor
- "21.7. FreeBSD as a Host with bhyve". Retrieved 13 December 2015.
- Coming Soon to OpenBSD/amd64: A Native Hypervisor
- vmm(4) — virtual machine monitor
- ACRN Memory Management High-Level Design