PAC097

VMware Workstation and GSX Server Architecture and

Internals: The Evolution

Scott Devine

Founder, Principal Engineer

VMware

This presentation may contain VMware confidential information.

Copyright © 2005 VMware, Inc. All rights reserved. All other marks and names mentioned herein may be trademarks of their respective

companies.

Outline

�Basic Architecture

�Virtual Machine State

�Networking

Conventional Computer System

�Operating Systems provide

� Hardware management / device drivers

� Application programmers interface (APIs)

Hardware

Operating System

Application Application Application

Traditional Virtual Machine Monitor

� Virtual machines provide

� Encapsulation

� Isolation

� Hardware Independence

Hardware

Virtual Machine Monitor

Virtual Machine

Operating System

Application Application

Virtual Machine

Operating System

Application Application

VMware Workstation Goals

�Provide the traditional benefits of virtual machines

�Virtualize x86 PCs

�Run like an application

�Run with good performance

VMware as an Application

� Installs like an application on an existing computer system

�All virtual machine data stored on existing file system

�No disk partitioning

�Native applications can run simultaneously to virtual machines

�Use existing device drivers

VMware as an Application

Hardware

Host OS

NativeApplication VMware Workstation

Virtual Machine

Guest OS

App App

VMware Virtual Machine Monitor�Virtualization techniques require

� Access to privileged CPU state

� Complete control of the Memory Management Unit (MMU)

�The Host OS gets in our way

Need to both run as an application and

as a privileged virtual machine monitor

3-Component Model

�Switch to privileged VMM to run virtual machine

� VMM takes complete control of CPU and MMU

� Host OS state is saved / restored on “World” switch

Hardware

Host OS

NativeApplication Virtual Machine

Guest OS

App App

VMware Workstation

VMM

Resources�The application only switches to the VMM when it’s running

� VMM switches back on interrupts

� Host OS sees the application as running during this time

�The application allocates all memory used by the virtual machine

�All resources used by the virtual machine are charged to the application

Devices

�Device I/O is routed through the application

�Use standard OS system call interfaces

Hardware

Host OS

NativeApplication Virtual Machine

Guest OS

App App

VMware Workstation

VMM

CPU / Memory Virtualization

�Traditional virtualization technique

Direct Execution

�Run OS at an unprivileged CPU level

�CPU traps to VMM on privileged instructions

�VMM emulates privileged instruction

�However, x86 is not strictly virtualizable

Binary Translation

� Inspects each instruction before its executed

�Replaces “dangerous” instructions with calls to emulation code

�Stores sequences of translated instructions in a translation cache

�Fast, but slower than direct execution

Dual Virtualization Method

�Use traditional direct execution when possible

�Well behaved user-level programs

�Use binary translation when not

�Operating system

�Real mode programs (old 16 bit DOS apps)

�User-level programs with special privileges (X server on Linux)

Dual Virtualization Method

Direct

Exec.

OK?

Return to

VM

Direct Execution

Binary Translation

Emulation

Privileged

Instruction

CPU

State

Goals Met� 3-Component model

� Acts like an application

� Use raw hardware for best performance

�Device I/O through application

� Reuse device drivers on host OS

�Dual virtualization

� Virtual x86 with binary translation

� Get good performance with direct execution

Outline

�Basic Architecture

�Virtual Machine State

�Networking

Virtual Machine State

�Virtual machine encapsulation: All the state of the virtual machine is stored on a few files on the host file system

Disks

�All disk state is stored in host files

�Disk read/write commands from the guest are translated into file read/write system calls

Host

File

VM

Guest OS

App App

Disk

Delta Disks

�Only stores differences between original and child

�Dependent on original disk not changing

VM A

Disk

Delta

Disk

Base

Disk

Checkpoints

�Virtual machine’s running state captured to a host file

�CPU state

�Memory

�Device state

�Time

VM

Guest OS

App App

Check-

point

File

Undoable Disks – Workstation 1.0

�Put disk in Undoable mode

VM’s

Disk

Disk

File

Undoable Disks – Workstation 1.0

�Put disk in Undoable mode

�All writes would go to a .REDO delta disk

VM’s

Disk.REDO

Disk

File

Undoable Disks – Workstation 1.0

�Put disk in Undoable mode

�All writes would go to a .REDO delta disk

�Commit or discard at power off

VM’s

Disk

.REDO

Disk

File

Undoable Disks – Workstation 1.0

�Put disk in Undoable mode

�All writes would go to a .REDO delta disk

�Commit or discard at power off

VM’s

Disk

.REDO

Disk

File

Suspend/Resume – Workstation 2.0

�Use a checkpoint to suspend execution of virtual machine to disk

�All running state of the virtual machine is saved in checkpoint

�Disk state is already on in a file

�Resume will continue execution

�Checkpoint file is removed

�Can only resume from a suspend once

Snapshots – Workstation 4.0

�“Undoableness” for the whole virtual machine

�Take a snapshot at any point

�Revert to a snapshot many times

VM

Guest OS

App App

Base

Disk

Snapshots – Workstation 4.0

�Undoableness for the whole virtual machine

�Take a snapshot at any point

�Revert to a snapshot many times

VM

Guest OS

App AppSnapshot

Check-

point

Base

Disk

Current

Disk

Multiple Snapshots –Workstation 5.0

�Creates a timeline of snapshots

�Going back in time does not delete future snapshots

�Creates a tree of snapshots

�All snapshots are kept with the virtual machine

Clones – Workstation 5.0

�Creates a copy of the virtual machine

VM A

Disk

Delta

Disk

Base

Disk

VM B

Disk

Delta

Disk

Outline

�Basic Architecture

�Virtual Machine State

�Networking

Basic Networking

�Vmnets acts as virtual switches

Host OS

NativeApplication

Vmnet 1

Virtual Machine

Guest OS

App App

Virtual Machine

Guest OS

App App

Bridged – Workstation 1.0

� Makes Vmnet look like an extension of the real Ethernet

Host

Vmnet 0

Virtual Machine

Guest OS

App App

Bridge

Network

Stack

Host File Access – Workstation 2.0

� Extra setup on Linux to share host file system on new network interface

Host

Vmnet 1

Virtual Machine

Guest OS

App App

DHCPServer

Network Stack

HostIf

NAT – Workstation 3.0

� NAT process does the address translation for all virtual machines on Vmnet 8

Host

Vmnet 8

Virtual Machine

Guest OS

App App

NATProcess

Host

Network Stack

Teams – Workstation 5.0

�Team: A group of networked virtual machines with the network topology

Host

Team Private Vmnet B

Server VM

Guest OS

App App

Client VM

Guest OS

App App

Team Private Vmnet A

Firewall VM

Guest OS

App App

Team Network Properties

�Team Private Vmnets have knobs

�Bandwidth

�Packet Loss Rate

�Can simulate modems, DSL, T1, LANs

VMware GSX Server Goals

�Enhance the VMware virtualization platform for enterprise servers

�Remote management

�Scripting

Remote Management in GSX Server

�Web-based remote management

�Serverd manages all running virtual machines

Hardware

Host OS

Virtual Machine

Guest OS

App App

vmware

VMM

serverdvmwarevmware

web

server

Remote Console

�Remote Console connections go direct to the virtual machines

Hardware

Host OS

Virtual Machine

Guest OS

App App

vmware

VMM

serverdvmwarevmware

web

server

VirtualCenter Management

�GSX Server can be managed by VirutalCenter

�VirtualCenter connects direct to serverd

Hardware

Host OS

Virtual Machine

Guest OS

App App

vmware

VMM

serverdvmwarevmware

web

server

Conclusion

�Basic virtualization techniques have stood the test of time

�Extended I/O functionality

�Extended remote management capabilities