whats new vmware vsphere 50 storage technical whitepaper

7/31/2019 Whats New VMware vSphere 50 Storage Technical Whitepaper

1/15

Whats New in VMwarevSphere 5.0 StorageT E C H N I C A L M A R K E T I N G D O C U M E N T AT I O N

v 1 . 0 / U p D A T E D M A y 2 0 1 1


2/15

Whs Nw in VMw vSph 5.0 Sog

T E C H N I C A L W H I T E p A p E R / 2

Table of Contents

Introduction 3vSphere VMFS-5 3

vSphere Storage DRS 4

Datastore Clusters 4

Placement Recommendations 5

Afnity Rules and Maintenance Mode 6

vSphere Storage APIs Storage Awareness 7

Prole-Driven Storage 7

Virtual Machine Compliance Status 8

Fibre Channel over Ethernet Sotware Initiator 9

iSCSI Initiator Enhancements 10

vSphere Storage I/O Control 11

vSphere Storage APIs Array Integration 11

vSphere Thin Provisioning 11

Hardware Acceleration or NAS 13

T10 Compliancy 14

vSphere Storage vMotion 14

Conclusion 15


3/15



InodcionVMware vSphere 5.0 (vSphere) brings many new capabilities to extend the benets o vSphere 4.1. These

new eatures and enhancements to core capabilities in vSphere provide more perormance optimization andeasier provisioning, monitoring and troubleshooting. This paper ocuses on the storage-specic eatures and

enhancements that are available in vSphere 5.0 and provides an overview o how they optimize storage

utilization, ease monitoring, and increase operational eciency. Wherever possible, we will also provide use

cases and requirements that might apply to these new unctions.

The topics to be covered in this paper are:

vSphereVMFS-5

vSphereStorageDistributedResourceScheduler(StorageDRS)

vSphereStorageAPIs-StorageAwareness

Prole-DrivenStorage

FibreChanneloverEthernet

vSphereStorageI/OControl(StorageI/OControl)vSphereStorageAPIsArrayIntegration

vSphereStoragevMotion(StoragevMotion)

This will provide a technical overview o new capabilities and enhancements as well as links to additional

inormation about each o these new storage eatures.

vSph VMFS-5vSphere5.0introducesanewversionofvSphereVMFS(VMFS),VMFS-5,whichcontainsmanyimportant

architecturalchangesthatenablegreaterscalabilityandperformancewhilereducingcomplexity.Although

numerous undamental changes have been made, the ollowing enhancements are signicant rom operational

and architectural aspects:

64TBdevicesupport

Uniedblocksize

Improvedsubblockmechanism

vSphere5.0facilitatesanondisruptiveupgradefromVMFS-3toVMFS-5,ensuringconsistencyacrossvirtual

infrastructures.Theuniedblocksize,1MB,enableseasierdeploymentsandreducedcomplexityfroman

architectural and operational aspect while maintaining the scalability and the exibility that were previously

foundonlywithlargeblocksizes.ItshouldbenotedthatvolumesthatareupgradedfromVMFS-3toVMFS-5

will retain their original block size, because modiying the block size would require a reormat o the volume. To

enable greater scalability and reduce storage overhead associated with small les, various enhancements have

beenmadetoVMFS-5.Theseenhancementsincludeoptimizedsubblocksizesandtheallocationofthese

blocks.Theyhaveresultedinsupportforlargevolumes(64TBonasingleextent)andhighervirtualmachine

densitywhilereducingtheoverheadassociatedwithsmallles.VMFS-5iscapableofallocating30.000

subblocksof8KBforlessuchasloglesandvirtualmachinemetadata(.vmxles).Forleswithasizesmaller

than1KB,smallleblocksareused.Thefollowingtabledepictsthemostsignicantarchitecturalchangesfor

VMFS-5incomparisontoVMFS-3.


4/15



Feature VMFS-3 VMFS-5

Supportforlargersingle-extentVMFSvolumesupto64TB No Yes

Supportforpass-throughRDMslargerthan2TB No Yes

Subblock or space eciency Yes(64KB,

maximum o

approx.3K)

Yes(8KB

maximum o

approx.30K)

Small-le support No Yes(1KB)

Uniedblocksize No Yes

Table 1.VMFS-toVMFS-Comparison

vSph Sog DrSVirtual machine provisioning has always imposed operational challenges. Monitoring datastore capacity and

I/Oloadhasproventobeverydicultandasaresultisoftenneglected.Duringtheprovisioningprocessof

virtual machines and virtual disks, datastores are oten randomly selected, leading to hot spots and over- or

underutilizeddatastores.StorageDRSisanewfeatureintroducedinvSphere5.0.Itprovidessmartvirtual

machineplacementandloadbalancingmechanismsbasedonI/Oandspacecapacity.StorageDRSwillhelp

decrease operational efort associated with the provisioning o virtual machines and the monitoring o the

storage environment.

Datastore Clusters

WithvSphere5.0,anewVMwarevCenterobjectcalledadatastoreclusterisintroduced.Datastoreclusters

formthebasisofStorageDRS.Adatastoreclusterisacollectionofdatastoresaggregatedintoasingleunitof

consumptionfromanadministratorsperspective.Whenadatastoreclusteriscreated,StorageDRScan

managethestorageresourcescomparablytohowStorageDRSmanagescomputeresourcesinacluster.As

with a cluster o hosts, a datastore cluster is used to aggregate storage resources, enabling smart and rapid

placement o new virtual machines and virtual disk drives as well as load balancing o existing workloads.

Figure1depictsadatastoreclusterof12TBformedbyfour3TBdatastores.

VM

VM VM VM VMVM

VM

VM VM

Datastore Cluster12TB

3TB 3TB 3TB 3TBDatastores

Figure 1.DatastoreCluster


5/15



Placement Recommendations

StorageDRSprovidesinitialplacementandongoingbalancingrecommendations,helpingvSphere

administratorsmakeplacementdecisionsbasedonspaceandI/Ocapacity.Duringtheprovisioningofavirtual

machine, a datastore cluster can be selected as the target destination or this virtual machine or virtual disk,afterwhicharecommendationforinitialplacementismadebasedonspaceandI/Ocapacity.Initialplacement

inamanualprovisioningprocesshasproventobeverycomplexinmostenvironments.Asaresult,crucial

provisioningfactorssuchascurrentspaceutilizationandI/Oloadareoftenignored.StorageDRSensuresthat

initial placement recommendations are made in accordance with space constraints and with respect to the

goalsofspaceandI/Oloadbalancing.ThesegoalsaimtominimizetheriskofstorageI/Obottlenecksandthe

performanceimpactonvirtualmachines.Figure2depictsthesedenablethresholds.

Figure 2.DeningThresholdsorStorageDRS

Ongoingbalancingrecommendationsaremadewhenoneormoredatastoresinadatastoreclusterexceedthe

user-congurablespaceutilizationorI/Olatencythresholds.Thesethresholdsaretypicallydenedduringthe

congurationofthedatastorecluster.StorageDRSappliesthedatastoreutilizationreportingmechanismofVMwarevCenterServer,tomakerecommendationswhenevertheconguredutilizedspacethresholdis

exceeded.I/Oloadisevaluatedbydefaultevery8hours.Whentheconguredmaximumspaceutilizationor

theI/Olatencythreshold(15msbydefault)isexceeded,StorageDRSwillcalculateallpossiblemoves,to

balance the load accordingly while considering the cost and the benet o the migration.


6/15



VM VM VM VMVM

VM

V M VM

VM

500GB 500GB 500GB 500GB

Datastore Cluster (Pod)

Apply

BalancingRecommendations 2TB

Datastores

Figure 3.DeningThresholdsorStorageDRS

Afnity Rules and Maintenance Mode

StorageDRSanityrulesenablecontroloverwhichvirtualdisksshouldorshouldnotbeplacedonthesame

datastorewithinadatastorecluster.Bydefault,avirtualmachinesvirtualdisksarekepttogetheronthesame

datastore.StorageDRSoersthreetypesofanityrules:

VMDKAnti-Anity

Virtual disks o a virtual machine with multiple virtual disks are placed on diferent datastores.

VMDKAnity

Virtual disks are kept together on the same datastore.

VMAnti-AnityTwo specied virtual machines, including associated disks, are placed on diferent datastores.

Inaddition,StorageDRSoersDatastoreMaintenanceMode,whichautomaticallyevacuatesallvirtualmachines

and virtual disk drives rom the selected datastore to the remaining datastores in the datastore cluster.

Datastore Cluster Datastore Cluster Datastore Cluster

VMDK Anity VMDK Anti-Anity VM Anti-Anity

VMVM

VM

VM

VM

VM

VMVM

VM

VM

Figure 4.AfnityRules

StorageDRSworkswithbothVMFS-andNFS-baseddatastores.ButmixingVMFSandNFSdatastoresina

single datastore cluster is currently not supported. More details about this eature, and implementation tips and

design considerations, are described in the documentation: Resource Management Guide, chapter 11.


7/15



vSph Sog aPIs Sog awnssvSphereStorageAPIs-StorageAwarenessisanewsetofAPIsthatwillenableVMwarevCenterServertodetect

thecapabilitiesofthestoragearrayLUNs/datastores,makingitmucheasiertoselecttheappropriatediskforvirtualmachineplacementorthecreationofdatastoreclusters.vSphereStorageAPIsStorageAwarenesscan

also provide help in acilitating, or example, the troubleshooting process or conversations between vSphere and

storageadministrators.Storagecapabilities,suchasRAIDlevel,thinorthickprovisioned,replicationstateand

muchmore,cannowbemadevisiblewithinVMwarevCenterServereitherviasystem-denedcapabilities,which

areper-datastoredescriptors,orviaattributesthatareexposedviaStorageViewsandSMSAPI.vSphere

StorageAPIsStorageAwarenessaimstoeliminatetheneedformaintainingmassivespreadsheetsdetailing

thestoragecapabilitiesofeachLUNthatareneededtoguaranteethecorrectSLAtovirtualmachines.

Pofl-Divn SogManagingdatastoresandmatchingtheSLArequirementsofvirtualmachineswiththeappropriatedatastore

canbechallengingandcumbersometasks.vSphere5.0introducesProle-DrivenStorage,whichwillenablerapidandintelligentplacementofvirtualmachinesbasedonSLA,availability,performanceorother

requirements and provided storage capabilities.

UsingProle-DrivenStorage,variousstoragecharacteristics,typicallydenedasatier,canberequestedina

virtual machine storage prole. These proles are used during provisioning, cloning and Storage vMotion to

ensure that only those datastores or datastore clusters that are compliant with the virtual machine storage

prole are made available. The virtual machine storage prole can also help select a similar type o datastores

whencreatingaStorageDRSdatastorecluster.Prole-DrivenStoragewillreducetheamountofmanual

administrationrequiredforvirtualmachineplacementwhileimprovingvirtualmachineSLAstoragecompliance.

APP

OS

Not

CompliantCompliant

Gold

Virtual Machine Storage Profile

Associated with Virtual Machine

Virtual Machine Storage Profile

Describing Storage

Requirements

Storage Capabilities

Surfaced by vSphere Storage

APIsStorage Awareness

or User Defined

Figure 5.VirtualMachineStorageProle


8/15



Prole-DrivenStoragedeliversthesebenetsbytakingadvantageofthefollowing:

FullintegrationwithvSphereStorageAPIsStorageAwareness,enablingusageofstoragecharacterization

supplied by storage vendors.

SupportforNFS,iSCSIandFibreChannel(FC)storage,andallstoragearraysontheHCL.

EnablingthevSphereadministratortotagstoragebasedoncustomer-orbusiness-specicdescriptions.

Usingstoragecharacterizationsand/oradministrator-deneddescriptionstocreatevirtualmachineplacement

rules in the orm o storage proles.

Providinganeasymeanstocheckavirtualmachinescompliancewiththeserules.Thisensuresthatthevirtual

machine is not deployed or migrated to an incorrect type o storage without the administrators being inormed

about it.

Virtual Machine Compliance Status

Asmentioned,virtualmachinestorageprolesenableyoutocheckcomplianceofallvirtualmachinesandthe

associatedvirtualdisksinasinglepaneofglass(Figure6).Buttheyalsoenableyoutoseethisonapervirtual

machine basis, ensuring that even virtual machine administrators who dont necessarily have access to all the

dierentlayerscanvalidatecomplianceonthevirtualmachinesummarystab(Figure7).

Figure 6.VirtualMachineStorageProlesSinglePaneoGlassorCompliance

Figure 7.PerVirtualMachineProleCompliance

Asaresult,managingstoragetiers,provisioning,migrating,cloningvirtualmachinesandcorrectvirtual

machineplacementinvSpheredeploymentshavebecomemoreecientanduserfriendly.Prole-DrivenStorage removes the need or maintaining complex and tedious spreadsheets and validating compliance

manually during every migration or creation o a virtual machine or virtual disk.


9/15



Fib Chnnl ov ehn Sow IniiovSphere4.0introducedsupportforhardwareFibreChanneloverEthernet(FCoE)adaptors.vSphere5.0will

taketheadoptionofFCoEtoanewlevelbyintroducingasoftwareFCoEadaptor.ThesoftwareFCoEadaptorwillrequireanetworkadaptorthatcansupportpartialFCoEooadcapabilitiesbeforeitcanbeenabled(Figure8).

Figure 8.AddingaSotwareFCoEAdaptor

AfterthesoftwareFCoEadaptorhasbeenenabled,anewadaptoriscreated.Afterthecreationoftheadapter,

devicesthatarepresentedtothisparticularadapterwillberecognized(Figure9)byvSphere,enablingthe

creationofanewdatastore.ThesoftwareFCoEadapterwillenableyoutouseFCoEcapabilitieswithoutthe

needtoresorttospecicFCoEhardwareadapters,creatingamoreagileandexibleinfrastructure.

Figure 9. FCoEAdaptor


10/15

T E C H N I C A L W H I T E p A p E R / 1 0


iSCSI Iniio enhncmnsWithvSphere4.0,anewiSCSIinitiatorwasreleasedthatwasmoreecientandcapableofdrivingmoreIOPS

thaneverbefore.TheentireiSCSIsoftwareinitiatorstackwasrewrittenandoptimizedforperformance.vSphere4.0enabledmultipleVMkernelnetworkadaptorstoconnecttotheiSCSIarray.Congurationwas

possible only through the command line and was considered to be a dicult task. vSphere 5.0 introduces the

abilitytofullycongureiSCSIthroughthevSphereclient,asdepictedinFigure10.

Figure 10.iSCSIInitiator


11/15



vSph Sog I/O ConolvSphereStorageI/OControlwasinitiallyintroducedinvSphere4.1toprovideI/Oprioritizationofvirtual

machinesrunningonaclusterofVMwareESXserversthathadaccesstoashared,iSCSIorFC,storagepool.ItextendedthefamiliarconstructsofsharesandlimitsthatexistedforCPUandmemory,toaddressstorage

utilizationthroughadynamicallocationofI/OqueueslotsacrossaclusterofVMwareESXservers.

vSphere5.0extendsStorageI/OControltoprovideclusterwideI/OsharesandlimitsforNFSdatastores.This

means that no single virtual machine should be able to create a bottleneck in any environment regardless o the

typeofsharedstorageused.StorageI/OControlautomaticallythrottlesavirtualmachinethatisconsuminga

disparateamountofI/Obandwidthwhentheconguredlatencythresholdhasbeenexceeded.Thisenables

othervirtualmachinesusingthesamedatastoretoreceivetheirfairshareofI/O.StorageDRSandStorageI/O

Controlaretheperfectpartnersforpreventingdeprecationofservice-levelagreementswhileprovidinglong-

termandshort-termI/Odistributionfairness.

vSph Sog aPIs ay IngionvSphereStorageAPIsArrayIntegration(VAAI)wererstintroducedwithvSphere4.1,enablingooad

capabilities support or three primitives:

Full copy enabling the storage array to make full copies of data within the array

Block zeroing enabling the array to zero out large numbers of blocks

Hardware-assisted locking providing an alternative mechanism to protect VMFS metadata

WithvSphere5.0,supportfortheVAAIprimitiveshasbeenenhancedandadditionalprimitiveshavebeen

introduced:

vSphereThinProvisioning(ThinProvisioning),enablingthereclamationofunusedspaceandmonitoringof

spaceusageforthin-provisionedLUNs

HardwareaccelerationforNAS

SCSIstandardizationbyT10compliancyforfullcopy,blockzeroingandhardware-assistedlocking

vSphere Thin Provisioning

vSphere5.0introducesmultipleVAAIenhancementsforenvironmentsusingarray-basedthin-provisioning

capabilities.Historically,thetwomajorchallengesofthin-provisionedLUNshavebeenthereclamationof

deadspaceandthechallengessurroundingthemonitoringofspaceusage.VAAIThinProvisioningintroduces

the ollowing:

DeadSpaceReclamationinformsthearrayaboutthedatastorespacethatisfreedwhenlesaredeletedor

removed rom the datastore by Storage vMotion. The array can then reclaim the reed blocks o space.

Out-of-SpaceConditionsmonitorsthespaceusageonthin-provisionedLUNstopreventrunningoutofphysical

space.Anew,advancedwarninghasbeenaddedtovSphere5.0forathin-provisionedout-of-spacecondition.

DeadSpaceReclamationoerstheabilitytoreclaimblocksofathin-provisionedLUNonthearraywhena

virtualdiskisdeletedormigratedtoadierentdatastoreby,forexample,StorageDRS(Figure10).Historically,

when virtual machines were migrated rom a datastore, the blocks used by the virtual machine prior to the

migration were still being reported as in use by the array. This meant that usage statistics rom the storage

array might have been misleading and expensive disk space possibly might have been wasted. With this new

VAAIprimitive,thestoragedevicewillbeinformedthattheblocksarenolongerused,resultinginbetter

reporting o disk space consumption and allowing or the reclamation o these unused blocks.


12/15



VMware ESX

VM VM VMVM

Storage vMotion2

VMFS-A

Obsolete

Virtual Machines

File Data Blocks

VMFS-B

Figure 11.ReclamationoDeadSpace

Anout-of-spaceconditionisthenightmareofeverystorageandvirtualizationadministratorinarray-based

thin-provisioned environments. Storage oversubscription in thin-provisioned environments can lead to

catastrophic scenarios when an out-o-space condition is encountered. vSphere 5.0 mitigates these problems

andsimpliesstoragemanagementthroughtheadditionofadvancedwarnings(Figure12)anderrorswhen

thresholdsarereachedforthin-provisioneddatastores.Enhancementsincludemechanismsthattemporarily

pause a virtual machine when disk space is exhausted, enabling the allocation o additional space to the

datastore, or the migration o an existing virtual machine, without resulting in the ailure o the virtual machine.

Figure 12.Out-o-SpaceCondition


13/15



Hardware Acceleration or NAS

HardwareAccelerationforNASwillenablefasterprovisioningandtheuseofthickvirtualdisksthroughtwo

newlyintroducedVAAIprimitives:

FullFileCloneSimilartoFullCopy.EnablesvirtualdiskstobeclonedbytheNASdevice

ReserveSpaceEnablescreationofthickvirtualdisklesonNAS

FullFileCloneenablesfastprovisioningofvirtualmachinesonNASdevices,similartotheexperience

FullCopyprovidedforiSCSI/FCarrays.

PriortovSphere5.0,avirtualdiskwascreatedasathin-provisioneddisk,notevenenablingthecreationofa

thickdisk.StartingwithvSphere5.0,VAAINASextensionsenableNASvendorstoreservespaceforanentire

virtualdisk(Figure13).ThisenablesthecreationofthickdisksonNFSdatastores.

Figure 13.CreationoThickDisks


14/15



NASVAAIplug-insarenotshippedwithvSphere5.0.Theyaredevelopedanddistributedbystoragevendors

but are signed by the VMware certication program to guarantee quality.

T10 CompliancyvSphere 4.1 introduced T10 compliancy or block zeroing, enabling vendors to utilize the T10 standards with the

default,shippedplug-in.vSphere5.0introducesenhancedsupportforT10,enablingtheusageofVAAI

primitives without the need to install a plug-in, as well as enabling support or many additional storage devices.

vSph Sog vMoionvSphereStoragevMotion(StoragevMotion)enableslivemigrationforrunningvirtualmachinedisklesfrom

one storage location to another with no downtime or service disruption. With vSphere 5.0, multiple

enhancements have been introduced to increase eciency o the Storage vMotion process, to improve overall

perormance, and or enhanced supportability. Storage vMotion in vSphere 5.0 now also supports the

migration o virtual machines with a vSphere snapshot and the migration o linked clones.

The enhancements to Storage vMotion include a new and more ecient migration process through the use o a

new eature called Mirror Mode. Mirror Mode enables a single-pass block copy o the source disk to the destination

diskbymirroringI/Osofcopiedblocks.NotonlyhastheeciencyofStoragevMotionincreased,butalsothe

migration time predictability, making it easier to plan migrations and reducing the elapsed time per migration.

ThemirrordriverisenabledonapervirtualmachinebasisandresideswithintheVMkernel.Whentheguest

OSofthevirtualmachinethatisundergoingtheprocessusingStoragevMotioninitiatesawritetoanalready

copiedblock,themirrordriver(Figure14)willsynchronouslymirrorthiswriteandwaitforboth

acknowledgementsbeforecommunicatingthistotheguestOS.

Guest OS

VMM/Guest

Datamover VMkernelMirror Driver

Source Destination

Userworld

Figure 14. DataFlowwithStoragevMotionMirrorMode


15/15


ConclsionvSphere5.0addsmanynewstoragefeaturestoanalreadyrichsetofcapabilitiessupportedinvSphere4.1.It

reduces complexity while providing greater scalability. Virtual machine provisioning historically has imposedoperational challenges. Monitoring and manually balancing workloads, or provisioning virtual machines based

onI/Ocapacityandspaceutilization,hasproventobeverydicultandhasoftenbeenneglected,leadingto

hotspotsandover-orunderutilizeddatastores.StorageDRSisanewfeatureintroducedinvSphere5.0that

providessmartvirtualmachineplacementandloadbalancingmechanismsbasedonI/Oandspacecapacity.

The result is a decrease in the operational efort associated with the provisioning o virtual machines and

monitoring o the storage environment.

About the Author

DuncanEppingisprincipalarchitectintheTechnicalMarketinggroupatVMwareandisfocusedonstorageand

VMwareESXi.Previously,heworkedatOracleandwithseveralconsultancycompanies,wherehehadmore

than 10 years o experience in the design and development o inrastructures and deployment best practices.

HewasamongtherstVMwarecertieddesignexperts(VCDX007).Heistheco-authorofmanybooks,including VMware vSphere 4.1 HA and DRS Technical Deepdive, the VMware vSphere 4.0 Quick Start Guide,

Foundation for Cloud Computing with VMware vSphere 4 and his most recent publication, Foundation for Cloud

Computing with VMware vCloud Director.

vMware, Inc.HillviewAvenuePaloAltoCAUSATel8-86-2Fax6-2-wwwvmwarecom

Copyright2VMware,IncAllrightsreservedThisproductisprotectedbyUSandinternationalcopyrightandintellectualpropertylawsVMwareproductsarecoveredbyoneormorepatentslistedat

http://wwwvmwarecom/go/patents VMware is a registered trademark or trademark o VMware, Inc in the United States and/or other jurisdictions All other marks and names mentioned herein may be

whats new vmware vsphere 50 storage technical whitepaper

Documents