![Page 1: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/1.jpg)
The Impact of Virtualization on Computer Architecture and
Operating Systems
Mendel Rosenblum
![Page 2: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/2.jpg)
Talk Outline
• Virtualization– What is virtualization? Why is it so compelling?
• Implications for computer architecture– Known techniques and current challenges
• Implications for system software– Implications for operating systems & OS researchers
• Conclusions
![Page 3: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/3.jpg)
App
Operating System
What is Virtualization?
• A level of indirection between hardware and software.
Hardware
App App App
App AppOperating SystemVirtualization Layer
App App App
Operating System OS
• Virtual Machine abstraction– Run all software written for physical machine.
VM-1 VM-2
![Page 4: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/4.jpg)
Virtualization Layer - Optimize HW utilization, power, etc.
User’s view of virtualization
MAIL SERVICE WEB STORE
COMPUTE STORAGE INTERCONNECT
SANs/NAS
CRM
VM VM
PHYS
ICA
L VI
EWLO
GIC
AL
VIEW
![Page 5: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/5.jpg)
Virtualization Layer - Optimize HW utilization, power, etc.
User’s view of virtualization
MAIL SERVICE WEB STORE
COMPUTE STORAGE INTERCONNECT
SANs/NAS
CRM
VM VM
PHYS
ICA
L VI
EWLO
GIC
AL
VIEW
VM - Exchange Server
CPU 2x1 GHZMemory 2 GBDisk 100 GBNetwork 1 GBFault toleranceDisaster recoverySecurity
VM - Exchange Server
CPU 2x1 GHZMemory 2 GBDisk 100 GBNetwork 1 GBFault toleranceDisaster recoverySecurity
![Page 6: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/6.jpg)
Key low-level VMM operations
Hardware
VMM
App
OS
Hardware
VMM
App
OS
Storage
•Multiplex
![Page 7: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/7.jpg)
Key low-level VMM operations
Hardware
VMM
App
OS
Hardware
VMM
App
OS
StorageApp
OS
•Suspend
•Multiplex
![Page 8: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/8.jpg)
App
OS
Key low-level VMM operations
Hardware
VMM
App
OS
Hardware
VMM
App
OS
StorageApp
OS
•Suspend
•Resume (Provision)
•Multiplex
![Page 9: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/9.jpg)
App
OS
Key low-level VMM operations
Hardware
VMM
App
OS
Hardware
VMM
App
OS
App
OS
Storage
•Suspend
•Resume (Provision)
•Migration
•Multiplex
![Page 10: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/10.jpg)
VMM Implementation
• Safely and efficiently multiplex the virtual hardware on the physical hardware– Virtual CPUs on Physical CPUs– VM’s Physical Memory on Machine’s Memory– VM’s I/O Devices on Real I/O Devices
• Norm is time sharing rather than space sharing.
![Page 11: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/11.jpg)
Hardware support for virtualization
• Goals:– Reduce virtualization overheads
• Goal: Run software same speed as without VMM.– Reduce the complexity of VMM software
• Goal: Trusted code base small ~ 10K lines
• Old hat in mainframe world.• Current status in the x86 world:
– CPU -> First generation shipping now. – Mem -> First generation shipping soon.– I/O -> Still a work in progress. Big challenges.
![Page 12: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/12.jpg)
CPU Virtualization Today
• Classic VMM trick: Directly execute VM in less privileged mode on real CPU.– Trap and emulate privileged instructions.
• Popular x86 VMMs use binary translation to detect and emulate privileged inst.– Works well because of high trap overheads.
![Page 13: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/13.jpg)
Virtual CPU architecture support
• From Mainframes: Microcode assist– Fewer traps
• x86 support: Intel’s VT, AMD-V– New mode for running VMs
• Trap and emulate style. – Fewer and faster traps
• Right direction but challenges remain– See next talk for details.
![Page 14: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/14.jpg)
Virtual Physical Memory
• Virtual Memory like features:– Non-contiguous layout– COW sharing of identical pages– Demand paging allowing memory over-commit.
• Classic VMM: Shadow Page Tables– VMM uses page table with VA->MA
VM1 VM2
MachineMemory 0
![Page 15: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/15.jpg)
Memory Architecture Support
• Cost of shadow page tables can be high– Workloads with many start/exit process – OSes that “flip pages” to avoid copies & COW.
• Classic mainframe:– Hardware support for the PA->MA map
• x86 Support: AMD’s NPT, Intel’s EPT– Another “page table” for mapping
![Page 16: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/16.jpg)
Modern I/O different from 1970’s I/O
• Can’t just read old papers to get solutions• Large device diversity
– Not everything is a channel architecture• High performance I/O devices
– 10Gig ethernet– 3D graphics
![Page 17: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/17.jpg)
Current virtual I/O devices• Guest device driver• Virtual device• Virtualization layer
– emulates the virtual device– remaps guest and real I/O addresses– multiplexes and drives the physical device– I/O features, e.g., COW disks,
• Real device– may be different from virtual device
Device Driver
Device Driver
I/O Stack
DeviceEmulation
Guest OS
Vir
tuali
zati
on
Layer
![Page 18: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/18.jpg)
Much functionality in I/O stack
– De-multiplexing I/Os – Converting formats (e.g. SCSI disk -> SAN)– Resource management (e.g. traffic shaping)– Fault tolerance– Enforce security policy... and much more.
• Difficult for hardware to accelerate and maintain rich functionality
![Page 19: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/19.jpg)
Passthrough I/O - Fast but inflexible
I/O MMU
DeviceManager
VF VF VF
PFI/O Device
Guest OS
Device Driver
Guest OS
Device Driver
Guest OS
Device Driver
VirtualizationLayer
I/O devices with:•Multiple personalities
Interface per VM
•I/O MMU for DMA
Remap PA to MA
Validate addresses
•Manageable by VMM
![Page 20: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/20.jpg)
Passthrough I/O Challenges
• DMA - Can the I/O MMU handle: – Discontinuous physical memory?– Read-only physical memory (COW)? (Disable?)– Paged out physical memory? (Disable?)
• VM Mobility– Can I migrate a VM? – Can I migrate to a system with a different I/O device?
• Interrupt routing– How does the interrupts get to the right VM?
![Page 21: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/21.jpg)
Virtualization of 3D graphics
• UI devices such as GPUs are challenging.– Time sensitive and high performance
• How to multiplex screen?– Windows vs. full screen
• Virtualization leading to interest in remote display technology– Host many PCs on a server
![Page 22: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/22.jpg)
Arch Support for Virtual I/O
• Challenge: Get acceleration and flexibility– Most hardware all or nothing. – Designers need to understand VMM
functionality.• Mobility support:
– Standardized virtual interfaces for devices– Ability to load and store virtual device state
![Page 23: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/23.jpg)
Summary of Hardware Support
• Current CPU trends are positive– Multicore, etc.
• Virtualization support should give:– Lower virtualization overheads– Simplify VMM implementation– Ubiquitous deployment
• Support should accelerate not replace – Give software the ability to use VMM layer.
![Page 24: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/24.jpg)
Operating Systems & Virtualization
Traditional major roles for an OS: 1. Manage hardware resources of machine.2. Export abstractions and functionality to
support application programs.Virtualization influences:
VM-optimized operating systems (today)Operating systems for virtual appliances (future)
![Page 25: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/25.jpg)
Paravirtualization
• Old idea, new term: – Modify OS to run better inside a virtual machine.
• Examples likely obsoleted by hardware support:– Remove trapping instructions from OS– Reduce shadow page table overheads– Reduce I/O device emulation overheads
• Resource management seems key:– GuestOS <-> VMM about resource management– CPU and memory resources – Get good inter-VM resource managment
![Page 26: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/26.jpg)
More Interesting: Virtual Appliances
• Trend to use virtual machine as software distribution mechanism.– Applications and OS bundled together.– Like Appliance Computing without hardware
• Many benefits for software vendor and customer:– Choose OS based on application needs, not what
customer has. • Functionality, performance, reliability, security, manageability.
– Simplify testing and support. – Offload much from customer.
• Example: CRM system
![Page 27: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/27.jpg)
Application-selected operating systems
For Virtual Appliances:• Don’t need hardware management in OS.
– Current OSes manage hardware.• Only services for one application needed
– Current OSes try to support broad range of applications.
• Look at hardware appliance operating systems for examples
![Page 28: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/28.jpg)
Desirable properties for VA OSes
• Highly customizable– Include only what application needs.
• Supports common VA functionality– “Firmware” update– Browser-based interfaces
• Interfaces to VMM and IT infrastructure– Authentication, policies, etc.
![Page 29: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/29.jpg)
Implications for operating systems
For Modern Operating Systems:• Address the needs of applications or fade awayFor Operating System Researchers:• Now a much lower bar for OS adoption
– In past need both drivers and application support.• Opportunity for new OSes
– Target the needs of particular application area– Be better in an important area:
• Security, reliability, performance, manageability
![Page 30: The Impact of Virtualization on Computer Architecture and ......Virtual CPU architecture support • From Mainframes: Microcode assist – Fewer traps • x86 support: Intel’s VT,](https://reader034.vdocuments.mx/reader034/viewer/2022050302/5f6b0d2e06059a5ab65726e2/html5/thumbnails/30.jpg)
Conclusions
• Virtualization is here and will be everywhere in near future.– Cannot handle future multicore without it.
• Large impact on how computing is now– Opportunities for architecture help.– Opportunities for new system software stacks.