virtualization - os3 · virtualization what is virtualization "a technique for hiding the...
TRANSCRIPT
LIA
Large Installation Administration
Virtualization
Virtualization
What is Virtualization"a technique for hiding the physical characteristics of
computing resources from the way in which other systems, applications, or end users interact with those resources."
Virtualization is the process of making things more abstract in order to make them easier to use.
Examples?
Storage virtualization
Files Linear sequence of bytes Instead of blocks on a disk (or magnetic particles)
Disk partitioning RAID - redundant array of independent disks Logical Volume management
Combines disks and partitions into logical disks.
Network virtualization
VLAN Multiple logical networks on same physical wires
Channel bonding multiple links combined offered a single, higher-
bandwidth link Computer clusters
multiple discrete computers into larger metacomputers
e.g. Hadoop Virtual NICs and bridges for VM communication
Resource virtualization
Multiprogramming Each process thinks it has CPU to itself
Virtual memory Present linear address space composed of non-
consecutive blocks of: Physical memory Disk space
Virtual MachinesVirtual Machines
Credits
Slides largely based on: “Virtual Machines: Versatile Platforms for Systems
and Processes” James E. Smith Ravi Nair
Morgan Kaufmann Publishers 2005
Why Virtual Machines?
Isolate applications in separate VMs Sandbox applications for security Support different OSes concurrently Legacy applications on legacy OSes Application testing using VMs with known state Testing OS upgrades, training OS development
Why Virtual Machines?
LIA context: Resource utilization
“Server consolidation” Facilitate maintenance Basis for cloud computing
Computer Architecture Computer Architecture RecapRecap
Instruction Set Architecture
Software
Hardware
ISA
E.g. x86_64
User + System ISA
Application Programs
Hardware
Operating System
UserISA
SystemISA
User ISA
= For doing computations Simple Memory Instructions
Move data from memory to registers and v.v. Integer Instructions Floating-Point Instructions Branch instructions
Jump to address Jump to address if ...
Part of SHA1 in assembly:
addl %esi, %e;movl %c, %esi;xorl %d, %esi;andl %b, %esi;xorl %d, %esi;
System ISA (1/2)
= Management of system resources System Resources:
Main memory Storage Other I/O devices
Management: Fair allocation between user programs Prevent concurrent/unauthorized access
Role of the Operating System
User Mode
System ISA (2/2)
OS requires special privileges over user programs OS runs in CPU Kernel mode Apps run in CPU User Mode
x86: Implemented via 2 privilege levels / rings
Kernel Mode
Ring 0 1 2 3
System ISA Instructions
Processor Management “Change to user mode + run application” Timer interrupt gives control back to OS
Memory Management Manage page table, TLB: virtual memory
I/O Management load and store to/from device
Traps “Change to kernel mode from application” On purpose (system call) or on exception
System Call Instruction
Application Programs
Hardware
Operating System
System callHandler
System call: read(file)
I/O instructions
Disk
E.g. syscall on x86 or svc on ARM.
System Call
= Mechanism for User Mode code to request services from Kernel Mode
What services? Read/write to files and devices Create processes
i.e. use Operating System abstractions: File abstraction for storing blocks on disk Process abstraction for running different code in
parallel
System Call Interface (1/2)
Application Programs
Hardware
Operating System
ISA
System calls
System Call Interface (2/2)
Read/write files or devices: open(filename, …) CreateFile(...) read(fd,data, …) ReadFile(...) write(fd,data, …) WriteFile(...) ioctl() SetConsoleMode(...)
Manipulate processes fork() CreateProcess(…) exit() ExitProcess(…)
More...
System calls via libraries
Application Programs
Hardware
Libraries
Operating System
System calls
E.g. libc
Architecture Model
Application Programs
Hardware
Libraries
Operating System
System calls
Run
Interrupts, Traps, faults
Privileged instructions
User Mode
Kernel Mode
Virtual MachinesVirtual Machines
Virtualize the machine?
What is the machine? Machine is defined by an interface 3 interfaces that can be virtualized:1. Instruction Set Architecture (ISA)2. Application Binary Interface (ABI)3. Application Programming Interface (API)
Interface 1: ISA
Application Programs
Hardware
Libraries
Operating System
System calls
ISA
Virtualize a complete machine, running an OS supporting multiple processes
= System VM
Interface 2: ABI
Application Programs
Hardware
Libraries
Operating System
ABI =System calls + user ISA
System calls
Virtualize the environment of a single process
= Process VM
Interface 3: API
Application Programs
Hardware
Libraries
Operating System
API =Libraries + user ISA
System calls
Example Virtualizing ISA
Support a machine's complete ISA VM/370 Xen* KVM*
Example Virtualizing ABI
Run binaries unmodified on different platform Sun WABI
Run Win32-x86 binaries on Solaris-SPARC Digital FX!32
Run Win32-x86 binaries on Win32-Alpha
Example Virtualizing API
Recompile applications from source Runs on any platform with same API
E.g. Linux-x86 and Linux-ARM (Assuming platform-independent code)
VM Implementations
Virtual Machine Monitor
Application Programs
Hardware
Libraries
Guest OS
Application Programs
(a) System VM (b) Process VM
Virtual Machine Monitor
Hardware
Host OS
What ISA? Same or different
Same: Run Win32-x86 on Linux-x86 Diff: Run Linux-ARM on Win32-x86
Virtual Machine Monitor
Application Programs
Hardware
Libraries
OS
Target ISA
Source ISA
Taxonomy
Same ISA
MultiprogrammedSystems
Different ISA
Emulators/Translators
High-levelLanguage VMs
Same ISA
Classic-SystemVMs
Hosted VMs
Different ISA
Whole-SystemVMs
Codesigned VMs
Process VMs System VMs
Win32native
Example: Windows Multiprogramming
Process
Win32native
Windows OS
X86 Hardware
Win32 API
One CPU, illusion of processes running in parallel
Example: Android
Java
Java VM + Libs
ARM Hardware
Linux OS
High-level Language VM (HLL-VM)
Different ISA: Java vs. ARM
Example: Android Emulation
Process
Java
Java VM
ARM VMRuntime
Linux OS Win32native
Windows OS
X86 Hardware
Win32 API
Example: Android Emulation on Xen
Java Java VM
ARM VMRuntime
Linux OSWin32Native
Windows OS
LinuxNative
LinuxNative
Linux OS
Xen Hypervisor
X86 Hardware
Xen domain
Example: VMWare
VMApp
Host OS
VMMonitor
X86 Hardware
VMDriver
Applications
Guest OS
Example: AS/400
Virtual Machine Monitor
Application Programs
Hardware
OS
Target ISA
Source ISA
Higher level ISA
Allow evolution of hardware ISA
Taxonomy Examples
Same ISA
MultiprogrammedSystems
Different ISA
Emulators/Translators
High-levelLanguage VMs
Same ISA
Classic-SystemVMs
Hosted VMs
Different ISA
Whole-SystemVMs
Codesigned VMs
Process VMs System VMs
ARM VM runtime
Java VM, MS CLR
VM/370
VMware, Xen,Docker*
UNIX
AS/400
FX!32
Implementing Virtual Machines Implementing Virtual Machines with Different ISAswith Different ISAs
VM implementation: Emulation
Emulation = implement interface of one system on another system with a different interface
Example: x86 instruction addl %edx,4(%eax)
Emulated via PowerPC instructions: lwz r4,0(r1) lddi r5,r4,4 lwzx r5,r2,r5 lwz r4,12(r1) add r5,r4,r5 stw r5,12(r1)
Emulation Model
Sa S
b
e
Source
Executing instruction e changes state of Source machine from S
a to S
b
Emulation Model
Sa
S'a
Sb
S'b
e
e'
Source
Target
Real machine has corresponding state S'a
Performs e by means of instruction(s) e'
Recall: Registers + Memory
CPU
r0
r1
r2
rN
PC MEM
Example: Change registers
e
e'
CPU MEM CPU MEM
Source
TargetCPU MEM CPU MEM
Example: Change registers
e
e'
CPU MEM CPU MEM
Source
TargetCPU MEM CPU MEM
CPU state of Source kept in Target memory, not registers!
Emulation Performance + Methods
Can be slow because of mapping Source to Target! Range of emulation methods:
Interpretation Binary translation
Interpretation: Decode a single source instruction and execute
using target instructions Binary translation:
Translate a block of source instructions once and reuse
Interpretation
Source instruction is a series of bytes Different formats
RISC: clean and simple CISC: complex with legacy Non-hardware: Java bytecodes
Complexity of format influences interpretation performance!
Example Formats
x86:
Java:
Prefixes Opcode Opcode ModR/M SIB Displace-ment Immediate
0-4 bytes optional optional optional 0,1,2,4 0,1,2,4 bytes
Opcode Index
Opcode Index1 Index2Opcode Index1
Opcode Data1 Data2Opcode
Index1
Software developer's manual: 3796 pages!
Java VM Specification: 604 pages
x86 Format
Prefixes: Repetition for strings, overrides for address and operand sizes
ModR/M: addressing mode and which register SIB: base register, index register, index scale Displacement: offset to be added to address Immediate: variable length operand
Prefixes Opcode Opcode ModR/M SIB Displace-ment Immediate
0-4 bytes optional optional optional 0,1,2,4 0,1,2,4 bytes
Binary Translation
Per-instruction interpretation slow Especially when complex
Alternative: Translate blocks of source instructions once Reuse
cf. Just-in-Time compilers Hard
Performance Tradeoff
E(n) = time needed to execute an instruction n times
Formula: E(n) = S + n*T S = startup time T = time required per emulation of the instruction
Interpretation: S low, T high
Binary translation S high, T low
Performance Tradeoff
E
n
Interpretation
Binary Translation
In practise: automatic profiling, often used code is binary translated
OS Emulation
For Process VMs have to emulate whole ABI User ISA System call interface
System call instructions (e.g. sysenter) emulated Translate from Source OS to Target OS Same OS: straightforward to hard Diff OS: straightfoward to impossible
No guarantees that Target OS has same features as Source!
E.g. fsync()