Performance and Scalability of Microsoft® SQL Server® on VMware vSphere™ 4

W H I T E P A P E R

2

VMware white paper

Table of Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Performance Test environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4workload Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

hardware Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Server hardware Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Storage hardware and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Client hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Software Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Storage Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Benchmark Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Single Virtual Machine Scale Up tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Multiple Virtual Machine Scale Out tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

experiments with vSphere™ 4: Features and enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Performance results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Single Virtual Machine performance relative to Native . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Multiple Virtual Machine performance and Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

performance impact of individual Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Virtual Machine Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Large pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

performance impact of vSphere enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Virtual Machine Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Storage Stack enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Network Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

resource Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Disclaimers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

about the author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

3

VMware white paper

IntroductionVMware vSphere™ 4 contains numerous performance related enhancements that make it easy to virtualize a resource heavy database with minimal impact to performance. The improved resource management capabilities in vSphere facilitate more effective consolidation of multiple SQL Server virtual machines (VMs) on a single host without compromising performance or scalability. Greater consolidation can significantly reduce the cost of physical infrastructure and of licensing SQL Server, even in smaller-scale environments.

This paper describes a detailed performance analysis of Microsoft SQL Server 2008 running on vSphere. The performance test places a significant load on the CPU, memory, storage, and network subsystems. The results demonstrate efficient and highly scalable performance for an enterprise database workload running on a virtual platform.

To demonstrate the performance and scalability of the vSphere platform, the test:

• MeasuresperformanceofSQLServer2008inan8virtualCPU(vCPU),58GBvirtualmachineusingahighendOLTPworkloadderivedfrom TPC-E1.

• Scalestheworkload,database,andvirtualmachineresourcesfrom1vCPUto8vCPUs(scaleuptests).

• Consolidatesmultiple2vCPUvirtualmachinesfrom1to8virtualmachines,effectivelyovercommittingthephysicalCPUs (scale out tests).

• QuantifiestheperformancegainsfromsomeofthekeynewfeaturesinvSphere.

The following metrics are used to quantify performance:

• SinglevirtualmachineOLTPthroughputrelativetonative(physicalmachine)performanceinthesameconfiguration.

• Aggregatethroughputinaconsolidationenvironment.

HighlightsThe results show vSphere can virtualize large SQL Server deployments with performance comparable to that on a physical environment. The experiments show that:

• SQLServerrunningina2vCPUvirtualmachineperformedat92percentofaphysicalsystembootedwith2CPUs.

• An8vCPUvirtualmachineachieved86percentofphysicalmachineperformance.

The statistics in Table 1 demonstrate the resource intensive nature of the workload.

Table 1. Comparison of workload profiles for physical machine and virtual machine in 8 CPU configuration

Metric Physical Machine Virtual Machine

throughput in transactions per second* 3557 3060

average response time of all transactions** 234 ms 255 ms

Disk i/O throughput (iOpS) 29 K 25 .5 K

Disk i/O latencies 9 ms 8 ms

Network packet rate receiveNetwork packet rate send

10 K/sec16 K/sec

8 .5 K/sec8 K/sec

Network bandwidth receiveNetwork bandwidth send

11 .8 Mb/sec123 Mb/sec

10 Mb/sec105 Mb/sec

1 See disclaimer on page 13.

*Workload consists of a mix of 10 transactions. Metric reported is the aggregate of all transactions.

**Averageoftheresponsetimesofall10transactionsintheworkload.

4

VMware white paper

In the consolidation experiments, the workload was run on multiple 2 vCPU SQL Server virtual machines:

• AggregatethroughputisshowntoscalelinearlyuntilphysicalCPUsarefullyutilized.

• WhenphysicalCPUsareovercommitted,vSphereevenlydistributesresourcestovirtualmachines,whichensurespredictableper-formance under heavy load.

Performance Test environment In this section, the workload’s characteristics, hardware and software configurations, and the testing methodology are described. The same components were used for both the physical machine and virtual machine experiments.

Workload Description TheworkloadusedintheseexperimentsismodeledontheTPC-Ebenchmark.ThisworkloadwillbereferredtoastheBrokerageworkload.TheBrokerageworkloadisanon-comparableimplementationoftheTPC-Ebusinessmodel2.

• TheTPC-Ebenchmarkusesadatabasetomodelabrokeragefirmwithcustomerswhogeneratetransactionsrelatedtotrades,account inquiries, and market research. The brokerage firm in turn interacts with financial markets to execute orders on behalf of the customers and updates relevant account information.

• Thebenchmarkis“scalable,”meaningthatthenumberofcustomersdefinedforthebrokeragefirmcanbevariedtorepresenttheworkloads of different-sized businesses. The benchmark defines the required mix of transactions the benchmark must maintain. The TPC-E metric is given in transactions per second (tps). It specifically refers to the number of Trade-Result transactions the server can sustain over a period of time.

TheBrokerageworkloadconsistsoftentransactionsthathaveadefinedratioofexecution.Ofthesetransactiontypes,fourupdatethedatabase,andtheothersareread-only.TheI/Oloadisquiteheavyandconsistsofsmallaccesssizes.ThediskI/Oaccessesconsistof random reads and writes with a read- to-write ratio of 7:1.

The workload is sized by number of customers defined for the brokerage firm. To put the storage requirements in context, at the 185,000-customerscaleinthe8CPUexperiments,approximately1.5terabytesofstoragewereusedforthedatabase.

In terms of the impact on the system, this workload spends considerable execution time in the operating system kernel context which incurs more virtualization overhead than user-mode code. This workload also was observed to have a large cache resident workingsetandisverysensitivetothehardwaretranslationlookasidebuffer(TLB).Efficientvirtualizationofkernellevelactivitywithin the guest operating system code and intelligent scheduling decisions are critical to performance with this workload.

2 See disclaimer on page 13.

5

VMware white paper

Hardware ConfigurationThe experiments were run in the VMware performance laboratory. The test bed consisted of a server running the database software (in both native and virtual environments), a storage backend with sufficient spindles to support the storage bandwidth requirements, and a client machine running the benchmark driver. Figure 1 is a schematic of the test bed configuration:

Figure 1. Test bed configuration

Server Hardware Description

Table 2. Server hardware

Dell power edge 2970

Dual Socket Quad-Core Server

2 .69Ghz aMD Opteron processor 2384

64GB memory

Storage Hardware and Configuration

Table 3. Storage hardware

eMC CLariiON CX3-40 array

180 15K rpM disk drives

LUN configuration for 8 vCpU virtual machine : 32 data + 1 log LUN in both Native and eSX

LUN configuration for multiple virtual machines : 4 data + 1 log LUN in each VM

Client HardwareThesingle-virtualmachinetestswererunwithasingleclient.Anadditionalclientwasusedforthemulti-virtualmachinetestsbecausea single client benchmark driver could drive only four virtual machines. Table 4 gives the hardware configuration of the two client systems:

Table 4. Client hardware

Dell Power edge 1950 HP Proliant DL580 G5

Single-socket, quad-core server Dual-socket, quad-core server

2 .66Ghz intel Xeon X5355 processor 2 .9Ghz intel Xeon X7350 processor

16 .0GB memory 64GB memory

Software Configuration

2-socket/8 core AMD server

1Gb direct-attach4-way and 8-way Intel clients

4Gb/s Fibre Channel switch

EMC CX3-40, 180 drives

Server Hardware Description

6

VMware white paper

Software VersionsThe operating system software, Microsoft SQL Server 2008, and benchmark driver programs were identical in both native and virtual environments.

Table 5. Software Versions

Software Version

VMware eSX 4 .0 rtM build #164009

Operating system (guest and native) Microsoft® windows Server® 2008 enterprise (Build 60001: Service pack 1)

Database management software Microsoft® SQL Server® 2008 enterprise edition 10 .0 .1600 .22

Storage LayoutMeasures were taken to ensure there were no hot spots in the storage layout and that the database was uniformly laid out over all availablespindles.Multiple,five-diskwide,RAID-0LUNswerecreatedontheEMCCLARiiONCX3-40.WithinWindows,theseLUNswere convertedtodynamicdiskswithstriped“volumes”createdacrossthem.Eachofthesevolumeswasusedasadatafileforthedatabase.

The same database was used for experiments comparing native and virtual environments to ensure that the configuration of the storagesubsystemwasidenticalforbothexperiments.ThiswasachievedbyimportingthestorageLUNSasRawDiskMapping(RDM)devicesinESX.AschematicofthestoragelayoutisinFigure 2.

Figure 2. Storage layout

Benchmark Methodology

Windows Server 2008

stri

pe

stri

pe Dynamic disk 1

Dynamic disk 2

VMware ESX

Dynamic disk 3

RDM1 RDM3RDM2

LUN1 LUN3LUN2

stri

pe

stri

pe

7

VMware white paper

Single Virtual Machine Scale Up TestsThe tests include native and virtual experiments at 1, 2, 4, and 8 CPUs. For the native experiments, the bcdedit Windows utility was used to specify the number of processors to be booted. The size of the database, the memory size, and SQL Server buffer cache were carefully scaled to ensure the workload profile was unchanged, while getting the best performance from the system for each data point.Allthescaleupexperimentsranat100percentCPUutilizationinnativeandvirtualconfigurations.Forexample,at2vCPUs,inthe guest, the workload would saturate both virtual CPUs. Table6 describes the configuration used for each experiment.

Table 6. Brokerage workload configuration for scale up tests

Number of CPUs Database Scale (Number of Customers)

Number of User Connections

Virtual Machine/Native Memory

SQL Server Buffer Cache Size

1 30,000 65 9GB 7GB

2 60,000 120 16GB 14GB

4 115,000 220 32GB 28GB

8 185,000 440 58GB 53 .5GB

Multiple Virtual Machine Scale Out TestsThe experiment includes identical 2 vCPU virtual machines with the same load applied to each. In order to provide low disk latencies forthehighvolumeofIOPSfromeachvirtualmachine,thedatadisksforindividualVMsareconfiguredonexclusivespindlesandtwovirtual machines shared one log disk. Table 7 describes the configuration used for each virtual machine in the multi-virtual machine experiments.

Table 7. Brokerage workload configuration for each 2 vCPU virtual machine

Database Scale Number of User Connections

Virtual Machine Memory SQL Server Buffer Cache

20000 35 7GB 5GB

Experiments with vSphere 4: Features and EnhancementsThe 4 vCPU configuration mentioned above in Table6wasusedtoobtaintheresultswithESXfeaturesandenhancements.

8

VMware white paper

Performance resultsThesectionsbelowdescribetheperformanceresultsofexperimentswiththeBrokerageworkloadonSQLServerinanativeandvirtual environment. Single and multiple virtual machine results are examined, and then results are detailed that show improvements duetospecificESX4.0features.

Single Virtual Machine Performance Relative to NativeHow vSphere 4 performs and scales relative to native are shown in Figure 3 below. The results are normalized to the throughput observed in a 1 CPU native configuration:

Figure 3. Scale-up performance in vSphere 4 compared with native

Thegraphdemonstratesthe1and2vCPUvirtualmachinesperformingat92percentofnative.The4and8vCPUvirtualmachinesachieve88and86percentofthenon-virtualthroughput,respectively.At1,2,and4vCPUsonthe8CPUserver,ESXisabletoeffectivelyoffloadcertaintaskssuchasI/Oprocessingtoidlecores.HavingidleprocessorsalsogivesESXresourcemanagementmoreflexibilityin making virtual CPU scheduling decisions. However, even with 8 vCPUs on a fully committed system, vSphere still delivers excellent performance relative to the native system.

The scaling in the graph represents the throughput as all aspects of the system are scaled such as number of CPUs, size of the benchmark database, and SQL Server buffer cache memory. Table 8showsESXscalingcomparablytothenativeconfiguration’sability to scale performance.

Table 8. Scale up performance

Comparison Performance Gain

Native 8 CpU vs . 4 CpU 1 .71

vSphere 8 vCpU vs . 4 vCpU 1 .67

0

1

4

5

6

3

2

1 2 4 8

Thro

ughp

ut(N

orm

aliz

ed to

1 C

PU N

ativ

e Re

sult)

Number of Physical or Virtual CPUs

Native

ESX 4.0

9

VMware white paper

Multiple Virtual Machine Performance and ScalabilityThese experiments demonstrate that multiple heavy SQL Server virtual machines can be consolidated to achieve scalable aggregate throughput with minimal performance impact to individual virtual machines. Figure 4 shows the total benchmark throughput as eight2vCPUSQLServervirtualmachinesareaddedtotheBrokerageworkloadontothesingle8-wayhost:

Figure 4. Consolidation of multiple SQL Server virtual machines

Each2vCPUvirtualmachineconsumesabout15percentofthetotalphysicalCPUs,5GBofmemoryintheSQLServerbuffercache,andperformsabout3600I/Ospersecond(IOPS).

Asthegraphdemonstrates,thethroughputincreaseslinearlyasuptofourvirtualmachines(8vCPUs)areadded.AsthephysicalCPUs wereovercommittedbyincreasingthenumberofvirtualmachinesfromfourtosix(afactorof1.5),theaggregatethroughputincreasesbyafactor of 1.4 .

AddingeightvirtualmachinestothissaturatesthephysicalCPUsonthishost.ESX4.0nowschedules16vCPUsontoeightphysicalCPUs,yetthebenchmarkaggregatethroughputincreasesafurther5percentastheESXschedulerisabletodelivermorethroughputusingthefewidlecyclesleftoverinthe6vCPUconfiguration.Figure5showstheabilityofESXtofairlydistributeresourcesinthe8vCPUconfiguration:

Figure 5. Overcommit fairness for 8 virtual machines

0.0

1.0

1.5

0.5

4.0

4.5

5.0

5.5

6.0

3.0

3.5

2.0

0

20

80

100

120

60

40

2.5

1 2 4 6 8

Thro

ughp

ut(N

orm

aliz

ed to

1VM

resu

lt)

Phys

ical

CPU

Util

izat

ion

(%)

Number of 2 vCPU VMs

Linear Performance Improvement Until CPU SaturationCPU

Overcommitted

ESX 4.0

Physical CPU Utilization

CPU Overcommitted (# CPUs > # Physical CPUs)

Perc

enta

ge o

f Agg

rega

te T

hrou

ghpu

tfo

r Eac

h V

M

Throughput Distribution for 8 x 2 vCPU VMs

75.0

87.5

100.0

62.5

50.0

37.5

0.0

12.5

25.0

VM 4

VM 3

VM 2

VM 1

VM 8

VM 7

VM 6

VM 5

10

VMware white paper

Table9 highlights the resource intensive nature of the eight virtual machines that were used for the scale out experiments:

Table 9. Aggregate system metrics for eight SQL Server virtual machines

aggregate Throughput in Transactions per Second

Host CPU Utilization Disk I/O Throughput (IOPS)

Network Packet Rate Network Bandwidth

2760 100% 23K 8 K/s receive7 .5 K/s send

9 Mb/sec receive98 Mb/sec send

Performance Impact of Individual Features Inthissection,wequantifytheperformancegainsfromkeyexistingESXfeaturessuchaschoiceinvirtualmachinemonitorandlargepage support.

Virtual Machine Monitor Modernx86processorsfromAMDandIntelhavehardwaresupportforCPUinstructionsetandmemorymanagementunit(MMU)virtualization. More details on these technologies can be found in [1], [2], and [3].ESX3.5andvSphereeffectivelyleveragethesefeatures to reduce the virtualization overhead and increase performance as compared to native for most workloads. Figure6 details theimprovementsduetoAMD’shardwareMMUassupportedbyVMware.ResultsareshownascomparedtoVMware’sall-softwareapproach,binarytranslation(BT),andamixedmodeofAMD-VandvSphere’ssoftwareMMU.

Figure 6. Benefits of hardware assistance for CPU and memory virtualization

Intheseexperiments,enablingAMD-Vresultsinan18percentimprovementinperformancecomparedwithBT.Mostoftheoverhead in binary translation of this workload comes from the cost of emulating privileged, high resolution timer-related calls used by SQL Server. RVI provides another 4 percent improvement in performance. When running on processors that include support for AMD-VRVI,ESXchoosesRVIasthedefaultmonitortypeforWindowsServer2008guests.

0.80

0.88

0.92

0.84

1.12

1.16

1.20

1.24

1.28

1.04

1.08

0.96

1.00

Binary Translation (BT) Hardware-Assisted CPUVirtualization

Hardware-Assisted Memory & CPU Virtualization

Thro

ughp

ut N

orm

aliz

ed to

BT

11

VMware white paper

Large PagesHardware assist for MMU virtualization typically improves the performance for many workloads. However, it can introduce overhead arisingfromincreasedlatencyintheprocessingofTLBmisses.ThiscostcanbeeliminatedormitigatedwiththeuseoflargepagesinESX([2], [3]). Large pages can therefore reduce overhead due to the hardware MMU in addition to the performance gains they provide to many applications.

Databaseapplicationssuchasthistestenvironmentsignificantlybenefitfromlargepages.Table 10 demonstrates the performance gainsthatcanbeseenfromconfiguringlargepagesintheguestandESX.

Table 10. Performance benefits of large pages with RVI monitor

SQL Server 2008 Memory Page Size

Physical Memory Page Size on ESX

Performance Gain Relative to Small Pages in Guest and ESX

Small Small Baseline

Large Small 5%

Small* Large* 13%

Large Large 19%

*ESX3.5andvSpheredoabest-effortallocationofallguestpagesontolargephysicalpagesinordertoensuregoodperformancebydefaultforyourapplicationonthehardware-assistedMMUmonitor.

Asseeninthetable,theBrokerageworkloadgainssignificantlyfromhavinglargepagesinbothSQLServerandESX.More information on large pages and its impact to a variety of workloads is available at [4].

Performance Impact of vSphere EnhancementsInthissection,keyenhancementsinvSpherearebrieflydescribedandgainsarequantifiedfortheBrokerageworkload.Manyofthese improvements derive from changes in the storage and networking stacks and in the CPU and memory resource scheduler. Most of these enhancements are designed to deliver best performance out-of-the-box and should require little tuning.

Virtual Machine MonitorvSphereintroducedsupportforeightvCPUsinavirtualmachine.AsshowninTable9, SMP scaling for this workload in the physical and virtual environment closely matches. In both cases, approximately 70 percent improvement in throughput is observed going from four to eight CPUs.

Storage Stack EnhancementsThereareanumberofimprovementsinthevSpherestoragelayerthatsignificantlyimprovetheCPUefficiencyofperformingI/Ooperations and allow virtual machines to handle the demands of the most intensive enterprise applications. Figure 7 summarizes the performance impact from vSphere storage enhancements:

Figure 7. Performance improvement from storage stack enhancements

0.9

0.95

1.1

1.15

1.2

1.05

1

LSI Logic Adapter LSI Logic Adapter + ImprovedI/O Concurrency

PVSCSI Adapter

Thro

ughp

ut (N

orm

aliz

ed to

LSI

Log

ic A

dapt

er)

12

VMware white paper

Abriefdescriptionoftheseenhancementsandfeaturesisgivenbelow:

• PVSCSI: A New Paravirtualized SCSI Adapter and Driver

ThePVSCSIdriverisinstalledintheguestoperatingsystemaspartofVMwareToolsandsharesI/Orequestandcompletionqueueswiththehypervisor.ThetightercouplingenablesthehypervisortopollforI/Orequestsfromguestandcompleterequestsusinganadaptiveinterruptcoalescingmechanism.ThisbatchingofI/OrequestsandcompletionofinterruptssignificantlyimprovestheCPUefficiencyforhandlinglargevolumesofI/Ospersecond(IOPs).

A6percentimprovementinthroughputwiththePVSCSIdriverandadapterisobservedovertheLSILogicparalleladapterandguestdriver.

• I/O Concurrency Improvements

InpreviousreleasesofESX,I/OrequestsissuedbytheguestareroutedtotheVMkernelviathevirtualmachinemonitor(VMM).OncetherequestsreachtheVMkernel,theyexecuteasynchronously.TheexecutionmodelinESX4.0allowstheVMMto asynchronouslyhandletheI/OrequestsallowingvCPUsintheguesttoexecuteothertasksimmediatelyafterinitiatinganI/Orequest.ThisimprovedI/Oconcurrencymodelisdesignedaroundtworingstructuresperadapterwhicharesharedbythe VMM and the VMkernel.

Theworkloadbenefitedsignificantlyfromthisoptimizationandsawa9percentimprovementinthroughput.

• Virtual Interrupt Coalescing

InordertofurtherreducetheCPUcostofI/Oinhigh-IOPSworkloads,ESX4.0implementsvirtualinterruptcoalescingtoallowforbatchingofI/Ocompletionstohappenattheguestlevel.Thisissimilartothephysicalinterruptcoalescingthatisprovidedinmany Fibre Channel host bus adapters.

Withthisoptimization,thediskI/OinterruptratewithinaWindowsServer2008guestwashalfofthatwithinthenativeoperatingsystem in the 8 vCPU configuration.

• Interrupt Delivery Optimizations

InESX3.5,virtualinterruptsforWindowsguestswerealwaysdeliveredtovCPU0.vSpherepostsinterruptstothevCPUthat initiatedtheI/O,allowingforbetterscalabilityinsituationswherevCPU0couldbecomeabottleneck.

ESX4.0uniformlydistributesvirtualI/OinterruptstoallvCPUsintheguestfortheBrokerageworkload.

Network Layer NetworktransmitcoalescingbetweentheclientandserverbenefitedthisbenchmarkbytwopercentonvSphere.TransmitcoalescingwasavailablefortheenhancedvmxnetnetworkingadaptersinceESX3.5anditwasfurtherenhancedinvSphere.Dynamictransmitcoalescing was also added to the E1000g adapter.

In these experiments, the parameter Net.vmxnetThroughputWeight,availablethroughtheAdvancedSoftwareSettingsconfigurationtab in the VI Client, was changed from the default value 0 to 128, thus favoring transmit throughput over response time.

Settingthisparametercouldincreasenetworktransmitlatenciesandmustbetestedforitsimpactbeforebeingset.NoincreaseintransactionresponsetimeswereobservedfortheBrokerageworkloadduetothissetting.

Resource ManagementvSphere includes several enhancements to the CPU scheduler that help significantly in the single virtual machine as well as multiple virtual machine performance and scalability of this workload.

• Further Relaxed Co-scheduling & Removal of CellRelaxedco-schedulingofvCPUsisallowedinESX3.5buthasbeenfurtherimprovedinESX4.0.InpreviousversionsofESX,thescheduler would acquire a lock on a group of physical CPUs within which vCPUs of a virtual machine were to be scheduled.

InESX4.0thishasbeenreplacedwithfiner-grainedlocking,reducingschedulingoverheadsincaseswherefrequentscheduling decisionsareneeded.Boththeseenhancementsgreatlyhelpthescalabilityofmulti-vCPUvirtualmachines.

13

VMware white paper

• Enhanced Topology / Load Aware SchedulingAgoalofthevSphereCPUresourcescheduleristomakeCPUmigrationdecisionsinordertomaintainprocessorcacheaffinitywhile maximizing last level cache capacity.

Cache miss rates can be minimized by always scheduling a vCPU on the same core. However, this can result in delays in scheduling certaintasksaswellaslowerCPUutilization.Bymakingintelligentmigrationdecisions,theschedulerinESX4.0strikesagoodbalance between high CPU utilization and low cache miss rates.

The scheduler also takes into account the processor cache architecture. This is especially important in light of the differences between the various processor architectures in the market. Migration algorithms have also been enhanced to take into account the load on the vCPUs and physical CPUs.

ConclusionTheexperimentsinthispaperdemonstratethatvSphereisoptimizedout-of-the-boxtohandletheCPU,memory,andI/O requirements of most common SQL Server database configurations. The paper also clearly quantified the performance gains from the improvements in vSphere. These improvements can result in better performance, higher consolidation ratios, and lower total cost for each workload.

The results show that performance is not a barrier for configuring large multi-CPU SQL Server instances in virtual machines or consolidating multiple virtual machines on a single host to achieve impressive aggregate throughput. The small difference observed in performance between the equivalent deployments on physical and virtual machines—without employing sophisticated tuning at the vSphere layer—indicate that the benefits of virtualization are easily available to most SQL Server installations.

DisclaimersAlldataisbasedonin-labresultswiththeRTMreleaseofvSphere4.Ourworkloadwasafair-useimplementationoftheTPC-Ebusinessmodel;theseresultsarenotTPC-EcompliantandarenotcomparabletoofficialTPC-Eresults.TPCBenchmarkandTPC-Earetrademarks of the Transaction Processing Performance Council.

The throughput here is not meant to indicate the absolute performance of Microsoft SQL Server 2008, nor to compare its performance toanotherDBMS.SQLServerwasusedtoplaceaDBMSworkloadonESX,andobserveandoptimizetheperformanceofESX.

Thegoaloftheexperimentwastoshowtherelative-to-nativeperformanceofESX,anditsabilitytohandleaheavydatabaseworkload. It was not meant to measure the absolute performance of the hardware and software components used in the study.

The throughput of the workload used does not constitute a TPC benchmark result.

acknowledgementsTheauthorwouldliketothankAravindPavuluri,RezaTaheri,PritiMishra,ChethanKumar,JeffBuell,NikhilBhatia,SeongbeomKim, FeiGuo,WillMonin,ScottDrummonds,KaushikBanerjee,andKrishnaRajafortheirdetailedreviewsandcontributionstosectionsofthis paper.

about the authorPriya Sethuraman is a Senior Performance Engineer in VMware’s Core Performance Group where she focuses on optimizing the performance of applications in a virtualized environment. She is passionate about performance and the opportunity to make softwarerunefficientlyonalargesystem.Shepresentedherworkon”EnterpriseApplicationPerformanceandScalabilityonESX” atVMworldEurope2009.

PriyareceivedherMSinComputerSciencefromUniversityofIllinoisUrbana-Champaign.PriortojoiningVMware,shehadextensiveexperienceindatabaseperformanceonUNIXplatforms.

14

VMware white paper

references[1]AMDVirtualization(AMD-V)Technology.http://www.amd.com/us-en/0,,3715_15781_15785,00.html.RetrievedApril14,2009.

[2]PerformanceEvaluationofAMDRVIhardwareAssist.http://www.vmware.com/pdf/RVI_performance.pdf.

[3]PerformanceEvaluationofIntelEPTHardwareAssist.http://www.vmware.com/pdf/Perf_ESX_Intel-EPT-eval.pdf.

[4]LargePagePerformance.Technicalreport,VMware,2008.http://www.vmware.com/resources/techresources/1039.

[5]ImprovingPerformancewithInterruptCoalescingforVirtualMachineDiskI/OinVMwareESXServer.IrfanAhmad,AjayGulati,andAliMashtizadeh,“SecondInternationalWorkshoponVirtualizationPerformance:Analysis,Characterization,andTools(VPACT’09)”,heldwithIEEEInternationalSymposiumonPerformanceAnalysisofSystemsandSoftware(ISPASS),2009.

VMware, Inc. 3401 Hillview Ave Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.comCopyright © 2009 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents.

VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All othermarks and names mentioned herein may be trademarks of their respective companies. VMW_09Q2_WP_VSPHERE_SQL_P15_R1