heterogeneous live migration of virtual machines pengcheng liu, ziye yang, xiang song, yixun zhou,...
TRANSCRIPT
Heterogeneous Live Migration of Virtual Machines
Pengcheng Liu, Ziye Yang, Xiang Song, Yixun Zhou, Haibo Chen, and Binyu Zang
Parallel Processing Institute, Fudan Universityhttp://ppi.fudan.edu.cn
The market of Virtualization is booming
• Virtual data-center, grid computing, and cloud computing
• “By 2010, at least 30%of non-desktop IT infrastructure will be virtualized (up from less than 5 percent in 2007).”– www.cio.com/article/170653
• What can virtualization do?– Sever consolidation– Mobility
• Virtual machine live migration– …
VM Live Migration
• Benefits– Load balancing– Online maintenance – Fault tolerance
23/4/18 PPI, Fudan University 3
VirtualMachine
Virtual Machine
VMMHardware Hardware
VMM
War between different VMM venders
• The battle in virtualization area is and will be continuing – Vmware, Microsoft, kvm, …
– They will likely co-exisit in a long time• Linux kernel 2.6.23 embeds different vmm support
– Xen, lguest, kvm
• Cloud Computing• More resources brings inevitably heterogeneity
Virtual Machine
VirtualMachine
Motivation
• Heterogeneity of different VMMs– Break uniformity and mobility– Downgrade the applicability of live migration
23/4/18 PPI, Fudan University 5
VMM 2VMM 1Hardware Hardware
Contribution: Vagrant
• A preliminary study aim at– Bridging the heterogeneity of different VMMs and
support live migration
• Goal– Provide common migration protocol– Use common format of VM abstraction– Provide a pool of memory migration algorithms
• Preliminary results– Support VM migration across Xen and KVM– Accept able performance
Outline
• Heterogeneity among VMMs
• Vagrant Design and Implementation
• Evaluation
• Conclusion
23/4/18 PPI, Fudan University 7
Heterogeneity in Abstracting Resources
• CPU– Internal virtual CPU format
• Memory– Memory management information
• I/O Devices– Virtual I/O Device format
23/4/18 PPI, Fudan University 8
Heterogeneity in Migration Algorithms
• Stop-and-copy
• Pre-copy
• Push and pull
23/4/18 PPI, Fudan University 9
Heterogeneity in Migration Protocols
• Different set of control commands
• Example – Xen VMM initiates migration by sending a string
”receive” to the target– KVM sends a special integer to start migration
23/4/18 PPI, Fudan University 10
Outline
• Heterogeneity among VMMs
• Vagrant Design and Implementation
• Evaluation
• Conclusion
23/4/18 PPI, Fudan University 11
Architecture of Vagrant
• Vagrant acts as a middle man between different VMMs
23/4/18 PPI, Fudan University 12
VMM A
VM
Management VM
VMM B
Vagrant Agency
StateTransformer
Algorithm pool
Management VM
Vagrant Agency
StateTransformer
Algorithm pool
Bridging the Heterogeneity
• CPU State– Intercept the state from local virtual CPU module
– Essential CPU state is transformed into Vagrant format
– Requirements :• VMMs can expose same CPU features
23/4/18 PPI, Fudan University 13
Bridging the Heterogeneity (2/2)
• I/O Devices– Intercept the state from local device module• H/W based virtualization relies on device emulation
– Eg. QEMU used by Xen, KVM and VirtualBox
– Requirements:• VMMs can support all I/O devices existing in the
migrated VM
23/4/18 PPI, Fudan University 14
Migration Protocol
• Initiate and Control the migration process– Common migration protocol and commands– Negotiation before the migration• Ensure the same capability of pre- and post- migration
environment– CPU features, supported I/O devices
23/4/18 PPI, Fudan University 15
VMM A VMM B
Hey,I want a RTL8169 NIC,
Can you support?
Stop and Copy
23/4/18 PPI, Fudan University 17
VirtualMachine
Virtual Machine
VMMHardware Hardware
VMM
Simply stop the VM and send
VM state
May cause very long
downtime, but total
time is short
Pre-copy Based Algorithm
• Sending memory pages while the VM is still running
23/4/18 PPI, Fudan University 18
VirtualMachine
Virtual Machine
VMMHardware Hardware
VMM
Sending memory pages
VM is still running
Stop the VM
Send whole VM state to destination
Pre-copy Reduces
downtime
Algorithm Pool
• Algorithm pool– At least one common algorithm is needed– Choose the algorithm by negotiation
• Ideally, support a set of different algorithms– Choose appropriate algorithm in different
scenarios– eg, the user wants to finish the migration as soon
as possible, stop and copy is better
23/4/18 PPI, Fudan University 19
Implementation
• A prototype based on– X86 with VT-x extensions– Xen 3.1.0 and KVM-48
– CPU• Both expose most CPU features
– I/O Device• Both rely on QEMU• VMM support for some devices, such as PIC/APIC/IOAPIC
23/4/18 PPI, Fudan University 20
Outline• Heterogeneity among VMMs
• Vagrant Design and Implementation
• Evaluation– Migration time– Application performance
• Conclusion23/4/18 PPI, Fudan University 21
Experimental Setup(1/2)
• Machine setup:– Migration between pair of PCs with
• 2.33 GHz Intel Core Duo CPU, 2GB RAM• RTL8169 NIC• 250 GB 7200 RPM SATA disk
– NFS server• 3.0 GHz Pentium IV with 1GB RAM• Intel Pro 100/1000 Ethernet NIC• 250 GB 7200 RPM SATA disk
– Machines were connected via switched Gigabit Ethernet
23/4/18 PPI, Fudan University 22
Experimental Setup (2/2)
• VM configuration– 128 MB of RAM– 5GB disk image with installation of Fedora Core 6
• Migration time under different workload– Idle, kernel-compile, memtest86, apache benchmark (ab)
• Application performance– Migrating a running Web Server
• Continuously serves a 256 KB file to a remote client
23/4/18 PPI, Fudan University 23
Questions for Evaluation
• Whether Vagrant has an acceptable total migration and minimal downtime?
• Whether applications during the migration can still provide acceptable performance?
• Is Vagrant’s performance comparable to other migration framework, eg. Xen and KVM?
23/4/18 PPI, Fudan University 24
Total Migration Time
23/4/18 PPI, Fudan University 25
Vagrant adopts batched memory transfer while KVM uses one-by-one manner
Migration Downtime
23/4/18 PPI, Fudan University 26
Downtime in Vagrant in comparable to homogeneous live migration
Migrating a Running Web Server
23/4/18 PPI, Fudan University 27
Memory Pre-copy, about 10% downgrade
Less than 1 second
Related Work
• Homogenous live migration over LAN– Vmotion, Xen (Clark et al. ’05) and KVM
• Live wide-area migration– Bradford et al. (vee’07)– “VM Turntable” (FGCS’06)
• Tons of virtualization related work
23/4/18 PPI, Fudan University 28
Conclusion
• Heterogeneous live VM migration framework– General abstraction of VMs and migration protocol
– No changes to hosted OS and application
– Minor changes to VMM abstraction
– Comparable downtime and end-to-end time
23/4/18 PPI, Fudan University 29
Heterogeneous Live Migration of Virtual Machines
Pengcheng Liu, Ziye Yang, Xiang Song, Yixun Zhou, Haibo Chen, and Binyu Zang
Parallel Processing Institute, Fudan Universityhttp://ppi.fudan.edu.cn
Backup Slides
• Control commands in Vagrant
• CPU state in Vagrant
• I/O devices configured for the VM
• Memory migrating algorithm
23/4/18 PPI, Fudan University 32
Control Commands in Vagrant
• Common set of control commands– Each has a 3 seconds timeout
23/4/18 PPI, Fudan University 33
CPU State in Vagrant
• The necessary information to be transferred includes– GPR, control regs, segment regs, debug regs, msr,
fpu regs, sysenter registers– Time stamp counter– Pending events
23/4/18 PPI, Fudan University 34
I/O devices configured for the VM
• The migrated VM is configured with– Keyboard and mouse– Rtc– PIC/APIC/IOAPIC– PIT– IDE disk– NIC– Serial port– VGA
23/4/18 PPI, Fudan University 35