practicum presentation: study of resource isolation within virtual enviroments of misbehaving...
TRANSCRIPT
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
1/56
Study of secure isolation of virtual machines
and
their exposure to hosts in a virtual
environment.
Gavin Fitzpatrick
School of Computing
Dublin City University
Dublin, Ireland
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
2/56
Virtualization Concepts
Type 1 Hypervisor used Type 2 Hypervisor used
Testing tools
Experiments
Results
Conclusions
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
3/56
Virtualization Concepts
x86 architectures are designed based on 4 rings of privilege:
Ring 3: executes user mode - has no direct access to the underling hardware
Ring 2: not used by modern operating systems.
Ring 1: not used by modern operating systems.
Ring 0: has full access to underlying hardware within the host system
Popek & Goldberg define an x86 Virtual Machine Monitor(VMM) with the
following characteristics:
Fidelity:V
MM must provide computing environment identical to a physical machine Performance: Programs should only have minimum performance impact when using a
VMM
Safety: The VMM must have complete control of the system resources
Ref: Popek & Goldberg Formal requirements for Virtualizable 3rd Generation Architectures
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
4/56
Virtualization Isolation
As discussed in *Virtual Doppelganger paper,
Isolation within virtualization can be defined
under two different dimensions:
Resource Isolation
Namespace Isolation
*Ref: http://www.cs.princeton.edu/~mef/research/paenevirtualization.pdf
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
5/56
Namespace Isolation
Namespace Isolation:
States how a VMM limits access to its file-system,processes, memory addresses, user ids etc.
2 aspect:
1. Configuration Independence: File names ofone VM do not conflict with that of anotherV
M. All hypervisors tested were unable to use the
same name for vms or their associated config/virtual disk files (vmdk, vdi, vhd)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
6/56
Namespace Isolation
2. Security:
One VM cannot modify data belonging to
another VM stored in the same host.
Within modern enterprise level environments
VMs are stored on iSCSI or Fibre-channel
networks which are inaccessible from VMs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
7/56
Resource Isolation
Refers to a VMMs ability to isolate resource
consumption of one VM from that of another
VM using appropriate algorithms:
This Presentation looks at how resource
isolation is affected by VMs misbehaving.
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
8/56
Hypervisor (VMM)
Type 1: ESXi, XEN Server, Hyper-V
Type 2: VMWare Workstation, Oracle VirtualBox
Containers: Virtual code that runs as an application,
Allows multiple encapsulated isolated instances which point to the underlying O.Son which its executed
*diagrams from: http://www-01.ibm.com/redbooks/community/display/REDP4480/Virtualization+Strategies+Architectural+Overview
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
9/56
ESXi
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
10/56
XENServer
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
11/56
Hyper-V
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
12/56
KVM
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
13/56
Virtualbox
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
14/56
Workstation
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
15/56
Testing tools
Commercial benchmarking tools includeVMark , Passmark and
All tests executed on VM4 (Ubuntu Guest)
Ramspeed
Systester
Geekbench
FIO
Ping testing ( Look at skipping?!)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
16/56
Ramspeed
Used to test memory performance with the following operations:
Copy (A=B)
Scale (A=m*B)
Add (A = B+C)
Triad (A=m*B+C)
2 tests are performed for Integers and Floating Point Numbers
10 rounds are performed for each test and results are averaged
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
17/56
Systester
Used to benchmark CPU performance by
calculating 512,000 digits of Pi using 2 algorithms:
Borwein Quadradic Covergance: Runs for 5 rounds
Gausse-Legendre: Runs for 10 rounds
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
18/56
Geekbench
Propreitary benchmarking tool used for memory & cpuperformance
Scores tested on following factors:
1. Integer Calculations (Blowfish, Text Compress/Decompress)
2. Floating Point calculations (Primality test, Dot Product)3. Memory operations (Read/Write Sequential, Stdlib Copy/Write)
4. Stream operations similar to Ramspeed( Copy, Scale, Add, Triad (similar to ramspeed tests)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
19/56
FIO
I/O benchmark tool used to test disk subsystemwithin Linux O.S with libaio library
Test performed:
Random write: 10 x 32mb files written Max average bandwidth recorded over 10
reads/writes
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
20/56
Ping testing
Tests Network I/O within each hypervisor, pingtests run from VM4 within the host to
Gateway
Host IP (Physical IP for host) VM2 (Win2003 server DoS victim during Exp4a/b)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
21/56
Testing Script
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
22/56
Experiments
Testing Environment
Crashme O.S stress test
Fuzz Application stress test
Forkbomb Memory stress test
DoS I/O stress test
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
23/56
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
24/56
Exp1 - Crashme
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
25/56
Exp1 VM1 CPU/MEM activity
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
26/56
Crashme - Observations
Exp1:
CPU: 100% usage
MEM usage: 75%
PRNGs used: Mersenne Twist (MT) Common PRNG tool
VNSQ (Variation of Middle Sq. Method) Take any No. Sq it, and take middle digits
Rand() uses c++ library
Virtualbox- in non VT-x mode: Running Exp1 on Virtualbox in non VT-x mode causes the O.S to hang using (MT) method
Both VNSQ and Rand() cause O.S to restart
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
27/56
Exp2 - Fuzz
Exp2:
CPU: 88% + usage
MEM: 30%+ usage
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
28/56
Exp3 - Forkbomb
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
29/56
Exp3 - Forkbomb
Forkbomb code Linux (VM3): defined in a bash script as:
Forkbomb code Windows (VM1,VM2): defined in batch as :%0|%0
REF: http://www.cyberciti.biz/faq/understanding-bash-fork-bomb/
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
30/56
Exp3a-c Host MEM activity
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
31/56
Exp4 - DoS
Exp4a:
CPU: 100% usage
NIC transmit/recieve: 10,000KBps / 5,500KBps
Exp4b:
CPU: 25% usage
NIC transmit/ recieve: 0 / 13,500KBps
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
32/56
Results
Illustrated on a test by test basis for all
experiments for the following:
ESXi XEN
Hyper-V
Workstation Virtualbox / Virtualbox nonVTx
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
33/56
ESXi - Memory
Ramspeed:
Y-axis in MB persec(higher score=better result)
Geekbench:
Y-axis is score (higherscore=better result)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
34/56
ESXi - CPU
Systester:
Y-axis in time (lower time better result)
Geekbench:
Y-axis is score (higherscore=better result)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
35/56
ESXi HDD/LAN
0
200
400
600
800
1000
1200
ESX KB/s
Avg Write per Exp
KB/s
0
0.05
0.1
0.15
0.20.25
0.3
0.35
0.4
0.45
ESX
Ping Host - Secs
0
0.2
0.4
0.6
0.8
1
1.2
1.4
ESX
Ping Gateway -
Secs
0
0.5
1
1.5
22.5
3
3.5
4
4.5
5
ESX
Ping VM - Secs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
36/56
ESXi - Summary
Memory: Geekbench: 2.2% better than average
Ramspeed: 2.5% better than average
CPU: 5% better than average
Disk: 18% below average (especially in Exp3c)
Network:
Host: 55% above average VM: 22% above average
GW: 19% above average
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
37/56
XEN-Memory
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
38/56
XEN-CPU
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
39/56
XEN-Disk/Network
0
200
400
600
800
1000
1200
1400
XenServer
Avg Write per Exp
KB/s
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
XenServerPing Host - Secs
0
1
2
3
4
5
6
XenServer
Ping VM - Secs
0
0.2
0.4
0.6
0.8
1
1.2
1.4
XenServer
Ping Gateway -
Secs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
40/56
XEN - Summary
Memory: Geekbench: Follows average apart from Exp3c
Ramspeed: 4.5% below average (Exp3c, Exp4b majorfactors)
CPU: 3% better than average
Disk: 41% greater performance than average
Network: Host: 20.3% above average (Exp4a performs badly)
VM: 31% below average
GW: 16.4% above average
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
41/56
Hyper-V - Memory
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
42/56
Hyper-V - CPU
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
43/56
Hyper-V HDD/LAN
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Hyper-V
Ping Host - Secs0
1
2
3
4
5
6
Hyper-V
Ping VM - Secs
0
0.5
1
1.5
2
2.5
3
3.5
Hyper-V
Ping Gateway - Secs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
44/56
Hyper-V Summary
Memory: Geekbench: Exp1,2,3a score below average, Exp3b-4b score
above average
Ramspeed: 3.4% below average (Exp3c-4b main cause)
CPU: 2.5% below average (resulting from Borwein tests)
Gauss test follows average
Disk: 18% above average (Exp3b, 3c show major loss in performance)
Network: Host: 81% below average
VM: 31% below average
GW: 4.5% below average
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
45/56
Workstation - MEM
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
46/56
Workstation - CPU
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
47/56
Workstation HDD/LAN
0
200
400
600
800
1000
1200
WorkstationKB/s
Avg Write per Exp KB/s
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Workstation
Ping Host - Secs
0
1
2
3
4
5
6
Workstation
Ping VM - Secs
0
0.5
1
1.5
2
2.5
3
3.5
Hyper-V
Ping Gateway - Secs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
48/56
Workstation - Summary
Memory: Geekbench: 1.1% below average (Exp3c-4b main cause)
Ramspeed: Integer tests 3.3% abover average, FloatingPoint tests 6% above average
CPU: 1.2 % below score across all experiments
Disk: 19% below average keeping in line with average trend
Network: Host: 5.1% better than average
VM: 11.4% better than average
GW: 10% better than average
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
49/56
Virtualbox - Memory
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
50/56
Virtualbox - CPU
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
51/56
Virtualbox HDD/LAN
0
200
400
600
800
1000
1200
1400
1600
Virtualbox KB/s
Avg Write per Exp KB/s
0
0.1
0.2
0.3
0.4
0.5
0.6
Virtualbox
Ping Host - Secs
0
1
2
3
4
5
6
Virtualbox
Ping VM - Secs
0
0.2
0.4
0.6
0.8
1
1.2
1.41.6
Virtualbox
Ping Gateway - Secs
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
52/56
Virtualbox - Summary
Memory: Geekbench: 1.1% below average (Exp3c-4b main cause)
Ramspeed: Integer tests 3.3% above average, FloatingPoint tests 6% above average
CPU: 1.2 % below score across all experiments
Disk: 19% below average keeping in line with average trend
Network: Host: 15% below average across all experiments
VM: 5% below average across all experiments
GW: 6.1% below average across all experiments
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
53/56
Conclusions
Type 1 Baremetal (ESXi)
Outperforms all hypervisors on:
Network (utilizes NIOC) CPU/MEM ( CPU Scheduler / Shadow Page tables)
Performs poorly for:
Disk access (SIOC doesnt enforce isolation)
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
54/56
Conclusions
Type 1 Para Virtualization (XEN & Hyper-V)
Disk I/O performs well on both platforms Due to ability of Guests to utilize the Domain0 hardware
device drivers
Network I/O per poorly XEN & Hyper-V both perform poorly for Network I/O
isolation
Mixed CPU/MEM results: XEN: Average Memory performance, better than average
CPU performance
Hyper-V Poor CPU/MEM performance
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
55/56
Conclusions
Type 2 Hosted (Virtualbox & Workstation)
Disk I/O performs poorly on both platforms
Network I/O mixed results: Virtualbox performs poorly across Network tests
Workstation performs well across Network tests
CPU performs poorly:
Both platforms report lower than average CPU results, this is due toHost CPU treating each Guest as a separate process, which has the
same CPU time slice allocation as other Host Context Ring 3 processes Memory performs well:
Both platforms perform well for memory access, VMM allocatephysical memory directly to each Guest, host in unaware of this.
-
8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.
56/56