Live Migration(LM) Benchmark ResearchCollege of Computer ScienceZhejiang UniversityChina

dfdsfsdfsdfsdfsdf

OutlineBackground and MotivesVirt-LM Benchmark OverviewFurther Issues and Possible SolutionsConclusionOur Possible Work under the Cloud WG

dfdsfsdfsdfsdfsdf

Background and Motives

dfdsfsdfsdfsdfsdf

Significance of Live MigrationConcept: Migration: Move VM between different physical machinesLive: Without disconnecting client or application (invisible)Relation to Cloud Computing and Data Centers:Cloud Infrastructures and data centers have to efficiently use their huge scales of hardware resources. Virtualization Technology provides two approaches:Server ConsolidationLive MigrationRoles in a Data Center:Flexibly remap hardware among VMs.Balance workloadSave energyEnhance service availability and fault tolerance

dfdsfsdfsdfsdfsdf

Motives of the LM BenchmarkScale and frequency leads to a significant LM cost (TC):

S(Scale): How many servers?Google: Estimated 200,000 to 500,000 servers, included in 36 data centers in 2008MS: Added 10,000 servers per month in 2008FaceBook: More than 30,000 servers in its data center in 2008F(Frequency):How often it happens?Load balancingOnline maintainance and proactive fault tolerancePower managementC(Cost of Live Migration):Hardware and network bandwidthsave and transfer VM stateWorkload performance: share hardwareService availability: downtime

dfdsfsdfsdfsdfsdf

Motives of the LM BenchmarkA LM benchmark is in need.LM benchmark helps make right decisions to reduce costDesign better LM strategiesChoose better platformEvaluation of a data center should include its LM performanceVMware released VMmark 2.0 for multi-server performance in DEC, 2010Existing evaluation methodologies have their limitations.VMmark 2.xDedicated to the VMwares platformsA macro benchmark -- no spefic metrics about LM performanceExisting research on LM ([Vee09 Hines], [HPDC09 Liu], [Cluster09 Jin], [IWVT08 Liu], [NSDI05 Clark], )All dedicated to design LM strategies No unified metrics and workloads. Results are not comparable to each other.Some critical issues are not mentioned.Still lack of a formal and qualified LM benchmark

dfdsfsdfsdfsdfsdf

Virt-LM Benchmark Overview

dfdsfsdfsdfsdfsdf

Goal and CriteriasGoal of Virt-LM Benchmark: Compare LM performance among different hardware and software platform, especially in data center scenarios

Design Criteria:Metric SufficientObservableConciseWorkloadTypicalScalableScoring methodology ImpartialStability Produce repeatable resultsCompatibilityUsability

dfdsfsdfsdfsdfsdf

System Under TestSystem Under TestSUT:Evaluation TargetHardware and software platformIncluding its VMM and the LM strategies it used

dfdsfsdfsdfsdfsdf

MetricsMetrics and Measurement: DowntimeDef: how long the VM is suspendedMeasure: pingTotal migration timeDef: how long a LM lastsMeasure: timing the LM commandAmount of migrated dataDef: how many data is transferredMeasure: transferred data on its exclusive TCP port Migration overheadDef: How much LM impaires performance of the workloadMeasure: Declined percentage of the workloadss score

*Metrics Sufficiency:Cost : migration overhead, amount of migrated data (burden on network)QoS: downtime, total migration timemigration overhead,

dfdsfsdfsdfsdfsdf

WorkloadsRepresentative to real scenariosWhere: Data centersWhen: Load balancing power management,service enhancement and fault tolerate

dfdsfsdfsdfsdfsdf

WorkloadsDuring a live migration,VM could run different servicesMail ServerApplication ServerFile ServerWeb ServerDatabase ServerStandby ServerOther VMs exist on the same platformHeavy during load balancingLight during power managementRandom during service enhancement and fault toleranceHappens at any moments (Migrations Points)*

dfdsfsdfsdfsdfsdf

Workload ImplementationInternal workload typesMail Server: SPECmail2008App Server: SPECjAppServer2004File Server: DbenchWeb Server: SPECweb2005Database Server: SysbenchStandby Server: Idle VM

External workload typesHeavy: more VMs to fully utilize the machineIncreasing VMs until workload performances are underminedLight: single VM on the platform

Platform (HW and VMM)VMVMVMmigrateOSInternalWorkloadExternal workload

dfdsfsdfsdfsdfsdf

Migration Points ProblemDuring the run of a workloadLM happens at random timePerformance varies at different points

workload: 483xalancbmk of SPECcpu2006

How to fully represent a workloads performance varietyTest as many migration pointsspreading the whole run of a workload

dfdsfsdfsdfsdfsdf

Migration Points ProblemProblemtoo many points prolong the test significantlySoutionMore sample results in each runOnly a few runs

ImplementationDivide a workloads runtime into many time sectors Each time sector is longer than total migration timeMigrate at the startpoint of each sector

dfdsfsdfsdfsdfsdf

Scoring MethodGoal: compute an overall scoreEach metric icompute a final score SiNormalize each result (Pij) using reference system(Rij)

Sum up results of all workloads:

Si of reference system is always 1000:Lower Score indicates higher performanceOpen Problem: merge the 4 metrics SiDifferent propertydifferent variationSimply adding up is not appropriateCurrent implementation in Virt-LM: Final result have 4 scores

dfdsfsdfsdfsdfsdf

Other CriteriasUsabilityEasy to configureVM images ProvidedWorkloads pre-installedEasy to runAutomatically managed after launch

CompatibilitySuccessful on Xen and KVM

Scalable workload: Fully utilize the hardwareHeavy enough macro workloadLive migration lasts a long time.(Multiple live migration)more than one are migrated concurrently

dfdsfsdfsdfsdfsdf

Benchmark ComponentsLogical componentsSystem Under TestMigration Target PlatformVM Image StorageManagement Agent

Benchmark componentsWorkload VM imagesDistributed on VM Image StorageRunning ScriptsInstalled on Management Agent

dfdsfsdfsdfsdfsdf

Internal Running ProcessFor every class of workloadInitialize the environmentRun the workloadMigrate the VM at different migration points Fetch the metrics resultsCollect all results and Compute an overall scoreManagement Agent automatically control the whole process

*

dfdsfsdfsdfsdfsdf

Experiments on Xen and KVMExperiment SetupSUT-XENVMMXen 3.3 on Linux 2.6.27HardwareDELL OPTIPLEX 755, 2.4GHz Intel Core Quad Q66002GB memory, sata disk, 100Mbit networkSUT-KVMVMMKVM-84 on Linux 2.6.27HardwareSame as SUT-XENVMLinux 2.6.27, 512MB mem, one coreWorkloadInternal: SPECjvm2008, cpu/mem intensive workloadsExternal: Light: single VMMigration Points:Spreading the whole running

dfdsfsdfsdfsdfsdf

Experiments on Xen and KVMAnalysisSUT-KVM intensively compress the dataLess migrated data and less total time

More overhead

dfdsfsdfsdfsdfsdf

Experiments on Xen and KVMAnalysisSUT-XEN strictly control the downtimeLess downtime

More migrated dataDue to more rounds of pre-copy to decrease downtime

dfdsfsdfsdfsdfsdf

Experiments on Xen and KVMAnalysis ConclusionSUT-XEN less downtimeand overhead,But more consumption of network

dfdsfsdfsdfsdfsdf

Further Issues and Possible Solutions

dfdsfsdfsdfsdfsdf

1. Workload ComplexityTotal test takes a long time

When workloads has too many combination

(I) Internal workload types:Mail Server,App Server, File Server, Web Server, DBServer , Standby Server(E) External workload types: Heavy, Light(P) Migration points quantity: Considerable due to the long run time of each workloadInternal workloadExternal workloadMultiple migrationMigration PointsTotal time = Runtime * N workload typesN = I * E * P (* M )

dfdsfsdfsdfsdfsdf

Possible SolutionsSpeed up for migration points: (Virt-LMs current implementation)More samples in a runUsing time-insensitive workloadsMicro operation: CPU, Memory, IODifferent memory r/w intensityAdvantage:Eliminate the Migration Points dimensionInternal workloads are reducedRuntime of each each workload is shortenDisadvantage:Different from real scenariosHybridTest time-insensitive micro workloadsAnalysis and predict typical workloads resultsRedefine an average workload

dfdsfsdfsdfsdfsdf

2. Multiple/Concurrent Live MigrationProblem: Define overall metricsRepresentative for platforms maxium performanceOther concerns:When average results decreased obviouslyWhen system cannot afford

Possible solutionsMaximum sum of metricsDefine different thresholds

Platform (HW and VMM)VMVMVMVMAveragedecreasedObviouslySystem cannot affordThresholds: Concurrent numbersMaximum sumSumdecreasedObviously

dfdsfsdfsdfsdfsdf

3. Other IssuesOverall score computationVirt-LM produces 4 scores as the final result

Definition of external workloadsCurrent implementation is simple

RepeatabilityNeed more experiment to examMigration points are not precisely arranged

CompatibilityShould be compatible to other VMM, besides Xen and KVM

UsabilityMore easy to configure and run

dfdsfsdfsdfsdfsdf

Conclusion

dfdsfsdfsdfsdfsdf

Current WorkInvestigation on recent work on LMSummarize the critical problemsMigration points Workload complexityScoring methodsMultiple live migrationPresent some possible solutionsImplement a benchmark prototype Virt-LM More details in Virt-LM: A Benchmark for Live Migration of Virtual Machine(ICPE2011)

dfdsfsdfsdfsdfsdf

Future workImprove and complete Virt-LMImplement and test other solutionsWorkload complexityMultiple live migrationOverall score computationOthersTest and compare their effectiveness and choose best one

*

dfdsfsdfsdfsdfsdf

Our Possible Work under the Cloud WG

dfdsfsdfsdfsdfsdf

Possible WorkRelation to the cloud benchmarkEnough migration cost in the workloadAlthough the cost maybe not a metric, we have to ensure workload could cause enough cost.How fast could a cloud reallocate resources?If implemented by live migration technology, it regards to following two factors:1. how many migrations (determined by) resource management and reallocation strategies2. how fast for each migration live migration efficiency & costPossible future work under cloud benchmarkWe may work on how to ensure the workload produce enough live migration cost*

dfdsfsdfsdfsdfsdf

Possible WorkWe hope to cooperate with other members, maybe join a sub-project related to live migration. We hope can contribute to the design of the Cloud Benchmark*

dfdsfsdfsdfsdfsdf

Team MembersProf. Dr. Qinming Hehqm@zju.edu.cn

Kejiang Ye Representative of the SPEC Research Groupyekejiang@zju.edu.cn

Assoc. Prof. Dr. Deshi Yeyedeshi@zju.edu.cn

Jianhai ChenChenjh919@zju.edu.cn

Dawei Huangtossboyhdw@zju.edu.cn.

dfdsfsdfsdfsdfsdf

Appendix: Teams Recent Work

dfdsfsdfsdfsdfsdf

Virtualization PerformanceVirtualization in Cloud Computing SystemIEEE Cloud2011, IEEE/ACM GreenCom2010Performance Evaluation & Benchmark of VMACM/SPEC ICPE2011, IWVT2008 (ISCA Workshop), EUC2008Performance Optimization of VMACM HPDC2010, IEEE HPCC2010, IEEE ISPA2009Performance Modeling of VMIEEE HPCC2010, IFIP NPC2010Performance Testing Toolkit for VMIEEE ChinaGrid2010

*

dfdsfsdfsdfsdfsdf

Publications [1] Live Migration of Multiple Virtual Machines with Resource Reservation in Cloud Computing Environments (IEEE Cloud2011, Accept)[2] Virt-LM: A Benchmark for Live Migration of Virtual Machine (ACM/SPEC ICPE2011)[3] Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective (IEEE/ACM GreenCom2010)[4] Analyzing and Modeling the Performance in Xen-based Virtual Cluster Environment, (IEEE HPCC2010 )[5] Two Optimization Mechanisms to Improve the Isolation Property of Server Consolidation in Virtualized Multi-core Server, (IEEE HPCC2010)[6] Evaluate the Performance and Scalability of Image Deployment in Virtual Data Center, (IFIP NPC2010)[7] vTestkit: A Performance Benchmarking Framework for Virtualization Environments, (IEEE ChinaGrid2010)[8] Improving Host Swapping Using Adaptive Prefetching and Paging Notifier, (ACM HPDC2010)[9] Load Balancing in Server Consolidation, (IEEE ISPA2009)[10] A Framework to Evaluate and Predict Performances in Virtual Machines Environment, (IEEE EUC2008)[11] Performance Measuring and Comparing of Virtual Machine Monitors, (IWVT2008, ISCA Workshop)

*

dfdsfsdfsdfsdfsdf

Thank you!

dfdsfsdfsdfsdfsdf

We have designed and implemented a live migration benchmark Virt-LM.*****Internal: workload run insides *Figure 1 and 2 shows performance varies at different migration points*P:target platforms results, R:reference platforms resultsi:metric ij:workload j*Easy to configure, easy to run*SUT: target platform, MTP: assist to migrate, VM Image Storage: disk, Management Agent: auto control the whole test

Configure benchmark components: figure 2*SUT-KVM compress the data, far more less than 512MB memory size*SUT-KVM compress the data, far more less than 512MB memory size**