Nutanix Complete ClusterA Technical Whitepaper

NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Table of Contents

ExecutiveSummary............................................................................... 3

Introduction........................................................................................... 4

LimitationsofCurrentArchitecture........................................... 4

TheGoogleApproach................................................................. 4

NutanixCompleteClusterArchitecture.............................................. 6

MainPillars.................................................................................. 6

ArchitectureOverview................................................................ 7

NutanixScale-OutConvergedStorageKeyComponents...... 8

KeyFeatures........................................................................................ 11

CapacityOptimization.............................................................. 11

EaseofManagement................................................................ 11

PerformanceandScalability..................................................... 12

HighAvailability........................................................................ 13

Conclusion........................................................................................... 15

EXECUTIV

ESUMMARY

3NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Executive SummaryStorageisthebiggestchallengeinvirtualizeddatacenterstoday.Thenetwork

storagearchitecturedesignedfifteenyearsagoforphysicalserversistoo

expensiveandcomplexforvirtualmachines.Withoutanenterprise-class

alternative,organizationsareforcedtousetraditionalsolutions,whichcant

keepupwithvirtualmachinesthataredynamic,growrapidlyinnumberand

continuetodemandnewlevelsofperformanceandcapacity.

NutanixCompleteClusterisascale-outcomputeandstorageinfrastructure

thatallowsorganizationstovirtualizetheirdatacenterswithoutrequiring

networkstorage(SANorNAS).Builtfromthegroundupforvirtualmachines,it

providescompletecomputeandstoragecapabilitiesalongwithenterprise-class

performance,scalability,availabilityanddatamanagementfeatures.Itleverages

industry-standardhardwarecomponents,solid-statedrivesandmarket-leading

hypervisorstoprovideanout-of-the-boxsolutionthatmakesvirtualization

extremelyeasyandcosteffective.

INTRODUCTIO

N

4NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

IntroductionLimitations of Current ArchitectureOrganizationsarebuildingtheirvirtualizationinfrastructureusingthe

traditionalservers-connected-to-storage-over-a-networkarchitecture,which

cantadapttotheever-changingdemandsofvirtualization.Inadditiontoslow

performance,networkstoragehasbecomethesinglebiggestsourceofcostand

complexityinvirtualizedenvironments.Thenetworkstorage-basedarchitecture

workedwellforphysicalserversthatservedrelativelystaticworkloads.

Virtualization,andnowCloudComputing,hasmadedatacentersextremely

dynamic;virtualmachinesarecreatedonthefly,movefromservertoserverand

dependheavilyonsharedresources.Thesecharacteristicsmakethemanagement

ofvirtualmachinesandtheirunderlyingphysicalinfrastructureextremely

complex.

Datavolumesaregrowingatarapidpaceinthedatacenter,thankstotheease

ofcreatingnewVMs.Intheenterprise,newinitiativeslikedesktopvirtualization

contributetothistrend.Serviceprovidersdealwithanevenlargernumberof

VMsastheybuilddatacenterstoservecustomerswhocantaffordthecost

andmanagementoverheadthatvirtualizationrequires.Thisgrowingpoolof

VMsisexertingtremendouscost,performanceandmanageabilitypressureon

thetraditionalarchitecturethatconnectscomputetostorageoveramulti-hop

network.

Theriseofsolid-statedrivesisanothertrendthatisrapidlywideningthegap

betweencomputeandstoragetiers.UseofSSDsthatare100X-1000Xfaster

thantraditionalharddiskswillmaketheexistingnetworkbottlenecksand

networkcomplexityevenworse,ifvirtualmachinesneedtoaccessthemovera

network.ManySAN/NASvendorsareaddingSSDstotheirsolutions,charginga

heftypremiumforthesedrivesandrequiringadditionalinvestmentsinnetwork

bandwidthtoaccessanalreadyexpensivetierofstorage.

The Google ApproachGoogleandotherleadingcloud-generationcompaniessuchasAmazon,Yahoo

andMicrosoft(Azure)realizedthatanetwork-storagebasedapproachwould

INTRODUCTIO

N

5NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

notworkfortheirdatacenters.Theybuiltsoftwaretechnology(suchasGoogle

FileSystem)thatcouldgluealargenumberofcommodityserverswithlocal

storageintoasinglecluster.ThisapproachallowedGoogletobuildaconverged

computeandstorageinfrastructurethatusedcommodityserverswithlocal

storageasitsbuildingblock.GoogleFileSystemrunsacrossaclusterofservers

andcreatesasinglepooloflocalstoragethatcanbeseamlesslyaccessedby

applicationsrunningonanyserverinthecluster.Itprovideshighavailability

toapplicationsbymaskingfailuresofharddisksandevencompleteservers.

GoogleFileSystemallowedGoogletobuilddatacenterswithmassivelyscalable

computeandstorage,withoutincurringthecostsandperformancelimitations

associatedwithnetworkstorage.

Nutanixhastakenasimilarscale-outapproachtobuildanenterprise-ready

computeandstorageinfrastructurethatisdesignedfromthegroundupfor

virtualmachines.

NUTANIX

COMPLETECLUSTERA

RCHITECTURE

6NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Nutanix Complete Cluster Architecture Main PillarsNutanixCompleteClusterwasdesignedfromscratchtosolvestoragechallenges

forvirtualmachinesbybuildingasystemthatleveragesthelatestadvancesin

systemarchitecture,hardwareandsoftwaretechnologies.Therearethreecore

pillarsoftheNutanixarchitecture:

Distributed Computing

TheNutanixarchitectureissimilartoGooglesarchitectureinthatitisascale-

outcomputeandstorageinfrastructurethateliminatestheneedfornetwork

storage.Atthesametime,NutanixbuildsuponGooglesarchitectureand

providesanenterprise-classsolution.WhileGoogleFileSystemisacustom

solutionthatworksforGooglesinternalapplications(search,Gmail,etc.),

Nutanixprovidesageneral-purposesolutionforvirtualizedenvironments.In

additiontoitsscale-outcapabilities,ithasthesameorbetterenterprise-class

datamanagementfeaturesthatarecommonlyprovidedbyadvancednetwork

storagesolutions,includinghighavailability,backup,snapshots,anddisaster

recovery.

Virtualization

TheNutanixarchitecturewasdesignedforvirtualmachinessoitsupportsall

hypervisorfunctionsthataresupportedbythetraditionalnetwork-storage

basedarchitecture,includingliveVMmigrationandhighavailability.In

addition,becausetheNutanixarchitectureisVM-aware,itovercomeslimitations

oftraditionalsolutionsthatwereoptimizedtoworkwithphysicalservers.For

example,whilethingsaremanagedonaperVMbasisoncomputeside,theunit

ofmanagementonstoragehastraditionallybeenaLUN.WhenaLUNisshared

bymanyVMs,itbecomesmoredifficulttoperformstorageoperationssuchas

backup,recovery,andsnapshotsonaper-VMbasis.Itisalsodifficulttoidentify

performancebottlenecksinaheavily-sharedenvironmentduetothechasm

betweencomputingandstoragetiers.TheNutanixarchitectureovercomesthese

limitations.

NUTANIX

COMPLETECLUSTERA

RCHITECTURE

7NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Solid-State Drives

TheNutanixarchitecturewasdesignedtotakeadvantageofenterprise-grade

solid-statedrives(SSDs).Itisimportanttonotethatthetraditionalstorage

systemsweredesignedforspinningmediaanditishardforthemtoleverage

SSDsefficientlyduetotheentirelydifferentaccesspatternsthatSSDsprovide.

Whileharddiskshavetodealwiththerotationandseeklatencies,SSDsdo

nothavesuchmechanicallimitations.Thisdifferencebetweenthetwomedia

requiresthesoftwaretobeoptimizeddifferentlyforperformance.Onecannot

simplytakesoftwarewrittenforharddisk-basedsystemsandhopetouseit

efficientlyonsolid-statedrives.TheNutanixarchitectureusesSSDstostorea

varietyoffrequently-accesseddata,fromVMmetadatatoprimarydatastorage,

bothinadistributedcacheforhigh-performanceandinpersistentstoragefor

quickretrieval.TomaximizetheperformancebenefitsofusingSSDs,theNutanix

architecture:

ReservesSSDsforI/O-intensivefunctions

Includesspace-savingtechniquesthatallowlargeamountsoflogicaldatato

bestoredinasmallphysicalspace

Migratescoldorinfrequently-useddatatoharddiskdrivesautomatically,

allowsadministratorstobypassSSDsforlow-priorityVMs

Architecture OverviewNutanixCompleteClusterisascale-outclusterofhigh-performancenodes,

orservers,eachrunningastandardhypervisorandthatcontainsprocessors,

memoryandlocalstorage,includingSSDs)andharddiskdrives.Eachnode

runsvirtualmachinesjustlikeastandardvirtualmachinehost.Inaddition,

localstoragefromallnodesisvirtualizedintoaunifiedpoolbyNutanixScale-

outConvergedStorage(SOCS)(Figure1).Ineffect,SOCSactslikeanadvanced

SANthatuseslocalSSDsanddisksfromallnodestostorevirtualmachinedata.

VirtualmachinesrunningontheclusterwritedatatoSOCSasiftheywere

writingtoaSAN.SOCSisVM-awareandprovidesadvanceddatamanagement

features.Itbringsdataclosertovirtualmachinesbystoringthedatalocallyon

thesystem,resultinginhigherperformanceatalowercost.NutanixComplete

Clustercanhorizontallyscalefromafewnodestoalargenumberofnodes,

enablingorganizationstoscaletheirinfrastructureastheirneedsgrow.

NUTANIX

COMPLETECLUSTERA

RCHITECTURE

8NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Server 1 Server 2 ..............

FusionIO HDDs

Virtual Machine / Virtual Disk

Server N

N u t a n i x S c a l e - O u t C o n v e r g e d S t o r a g e ( S O C S )

Figure 1: Nutanix Scale-Out Converged Storage Architecture

WithNutanixCompleteCluster,allvirtualizationfeatures,includinghigh

availabilityandliveVMmigration,continuetoworkseamlessly.Administrators

createvirtualmachinesonNutanixCompleteClusterusingtheirstandard

processes.NutanixSOCSprovidesstorageforthesevirtualmachinesintheform

ofvirtualdisks,orvDisks,whicharestandardiSCSIdevices.

Nutanix Scale-Out Converged Storage Key ComponentsThekeytotheNutanixarchitectureisSOCSapatent-pendingscale-out

convergedstoragelayerthathasthefollowinguniquesetofcapabilities:

Itisconvergedwiththecomputelayer.VMsandSOCSco-existonthesame

cluster.

ItisVM-aware.SOCSprovisionsstorageonaper-VMbasisandcanidentifyI/O

comingfromeachVM.

Itcanscaleoutfromafewnodestoalargenumberofnodes.

Ithasground-upintegrationwithsolid-statedrives.

Itprovideshighavailabilityagainstdiskornodefailures.

ItprovideshighperformancebymakingI/Oaccesslocal,leveragingsolid-state

drivesandemployingaseriesofpatent-pendingperformanceoptimizations.

Itprovidesuniquecapacityoptimizationcapabilities.

SOCSisenabledbythefollowingcomponents:

NUTANIX

COMPLETECLUSTERA

RCHITECTURE

9NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

n-Way Controller Cluster

WhiletraditionalSANsolutionstypicallyhave1,2,4or8controllers,ann-node

NutanixCompleteClusterhasncontrollers(Figure2).EverynodeonNutanix

CompleteClusterrunsaspecialvirtualmachine,calledaControllerVM.This

virtualmachineactsasavirtualcontrollerforSOCS.AllControllerVMsin

theclustercommunicatewitheachothertoformasingledistributedsystem.

UnliketraditionalSAN/NASsolutionsthatarelimitedtoasmallnumberoffixed

controllers,thisarchitecturecontinuestoscaleasmorenodesareadded.

Node 1 Node 2 ..............

SSDs HDDs

Virtual Machine / Virtual Disk

Node N

Control ler VM Control ler VM Control ler VM...... . . . . . . . .

.... ............ .... ............ .... ............

Figure 2: Nutanix Scale-Out Controller VM Architecture

Heat-Optimized Tiering Cache (HOTcache)

HOTcacheisahighperformancecachebackedbySSDsfromeachnodeina

cluster.WhenguestVMswritedata,thatdataisfirstwrittentoHOTcacheand

then,inthebackground,itisflushedtoSOCS.HOTcacheusesasequentialdata

layouttoprovidehighperformanceevenifworkloadsfromVMsgetmixedinto

arandomworkload.HOTcachekeepsonedatacopyonalocalSSDandanother

copyonadifferentnodesothatthereisnodataloss,eveninthecaseofanode

failure.

Distributed Metadata Service (Medusa)

Intraditionalstoragesystems,controllersoftenbecomeabottleneckasmore

storageisadded.Oneofthemainreasonsforthisissueisthatthestorage

metadataisstoredonthecontrollers.Traditionalsystemsscaletoasmall

numberofcontrollers,soasthenumberofVMsincreases,sodoestheI/Oload

oneachcontroller.Nutanixsdistributedmetadataservice,Medusa,distributes

NUTANIX

COMPLETECLUSTERA

RCHITECTURE

10NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

theclustermetadataacrosstheclusterforscalabilityandreplicatesthedata

onmultiplenodesforfaulttolerance.Theserviceishighlyavailableandcan

toleratemultiplemodulefailures.Incomparison,traditionalsolutionsthat

dependonafewstoragecontrollerscannottoleratemultiplecontrollerfailures.

Distributed Data Maintenance Service (Curator)

Nutanixsdistributeddatamaintenanceservice,Curator,isaMapReduce1

-basedframeworkforexecutingbackgrounddatamanagementoperationsina

massivelyparallelmanner.Suchoperationsinclude:

Migrationofcolddatatolowertiers(forHeat-OptimizedTiering)

Garbagecollectionofdatathathasbeendeleted

Dataconsistencythroughroutinechecksums

Replicationofdataincaseofnodeordiskfailures

Re-balancingofdatawhennodesareaddedorremoved

MigrationofdatatomaximizelocalaccesswhenaVMmovesfromonenode

toanother

FlashStore

FlashStoreisthepersistent,flash-basedstorageprovidedbythepoolingof

SSDsfromallnodesinthecluster.DataisfirstwrittentoFlashStoreandthen

movedofftoDiskStore,asitbecomescold.Ascolddatabecomeshotagain,

itisbroughtbackintoFlashStore.ForvDisksthatservelowpriorityVMs,

administratorshaveanoptiontoskipFlashStore.

DiskStore

DiskStoreisthehigh-capacitySATAstoragetierspreadacrossthecluster.

DiskStoreprovideslargestoragecapacityforcolddata.NutanixsHeat-

OptimizedTiering(HOT)combinesFlashStoreandDiskStoretoprovidehigh

performanceaswellashighcapacityatalowercost.

1 MapReduceisatechnologyoriginallybuiltatGoogleformassivelyparallelanalysisofdatainacluster

KEYFEATURES

11NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Key FeaturesNutanixCompleteClusternotonlyeliminatestheneedforaSAN,butitalso

providestopofthelineavailability,performanceanddatamanagement

features.Someofthekeyfeaturesaredescribedbelow.

Capacity OptimizationNutanix QuickClone

Nutanixenablesadministratorstorapidlydeploynewvirtualmachinesbyusing

itsQuickClonefeature.QuickClonesarewriteablesnapshotsthatbehavejust

likestandardvDisks-administratorscanattachthemtoaVM,writedataon

them,andevensnapshotthemfurther.Thisisusefulfordeployingnewvirtual

desktops,creatingtestanddevelopmentcopiesofaproductiondatabaseand

anyotherscenariorequiringcloneswithoutduplicatingthedata.Thesystem

alsosupportsread-onlysnapshotsforbackuppurposes.

Nutanix Thin Provisioning

Storageforvirtualmachinesisthinlyprovisionedinthesystem.Administrators

cansetthecapacityofavDiskbutphysicalstorageisallocatedonlywhen

required.Administratorscanalsosetaminimumreservationparameterthat

guaranteesthespecifiedamountofstorageforacollectionofvDisks.

Nutanix Converged Backup

TheConvergedBackupfeatureprovidesinstantbackupandrecoverycapabilities

forvDisks.Severalmonthsworthofbackupscanbekeptinsidetheappliance

withoutrequiringexternalbackupstorage.Whenrecoveryisnecessary,

administratorscaninstantaneouslyrestoreavDisktoanyofitspastbackups.The

appliancealsosupportsoffsitebackupsusingstandardthird-partytools.

Ease of ManagementEase of Deployment

NutanixCompleteClusterisaplug-and-playsolutionthatincludesallhardware

andsoftwarenecessarytorunalargenumberofvirtualserversorvirtual

desktops.AdministratorscansetitupandstartcreatingVMsinamatterof

minutes.

KEYFEATURES

12NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Next-Generation User Interface

NutanixCommandCenterisahighlyintuitiveFlex-baseduserinterface

thatprovidesadministratorcompletevisibilityacrosscomputeandstorage

resourcesinthecluster.Itenablesthemtotroubleshootissuesrelatedtoa

virtualmachineseasilybymappingeachVMtothephysicalresourcesinthe

system.NutanixCompleteClusteralsoprovidesacommand-lineinterfacefor

management.

Nutanix Scale-out Converged Storage (SOCS)

NutanixSOCSeliminatestheneedformanagingacomplexnetwork-basedstorage

infrastructure,makingiteasytomanagevirtualenvironmentsatanyscale.

Conformance to IT Standards

WhileNutanixCompleteClusterenablesaconvergedarchitecture,itcontinues

tosupportstandardtoolsandinterfacesthatITdepartmentsalreadyuse.For

example,thenodesrunanindustry-standardhypervisor(VMwareESXi)andall

ITprocessesandsoftwaretoolsthatworkwiththishypervisorcontinuetowork

withNutanix.Similarly,vDisksarestandardiSCSIdevicesthatareconnected

toVMsusingastandardiSCSIinitiatorinthehypervisor.Byleveragingsuch

standardinterfaces,NutanixCompleteClustercanseamlesslyfitintoanexisting

ITecosystem.

Performance and ScalabilitySolid-State Drives

NutanixCompleteClusterwasdesignedwithSSDsinmind.Theincludedserver-

basedSSDsprovidehigherperformancethanSAN-basedSSDsbecausetheyavoid

thenetworkbottleneck.Traditionalsystemshaveoftenbeenlimitedbythe

amountofmetadatatheycankeepinacontrollerscache.InNutanixComplete

Cluster,SSDsareusednotonlyforVMdata,butalsoforstoringSOCSmetadata

forfastaccess.KeepingrichmetadataonSSDsenablesSOCStoprovide

advanceddatamanagementcapabilities.Giventhescale-outarchitecture,SSD

capacityinthesystemgrowsasmorenodesareaddedtothecluster.

KEYFEATURES

13NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

Nutanix Heat-Optimize Tiering (HOT)

IntheNutanixcluster,avDiskcanbeallocatedamixofSSDandHDDcapacity.

Toensurethatonlyhigh-valuedatastaysonSSDs,SOCSmovescolddatatohigh-

capacitySATAdrivesinthebackgroundusingitsHOTfeature.

Scalability

NutanixCompleteClusterisdesignedtoscalefromafewnodestoalarge

numberofnodes.Everyaspectofthesystemwasdesignedwiththescalability

requirementsoftodaysvirtualizeddatacentersinmind.Forexample,there

isnocentralizedmetadatamasterinthesystem.Themetadatalayeritself

isdistributedacrossthecluster,eliminatingacommonbottleneckfoundin

mostscale-outsystems.Also,witheverymodulerunningaSOCSController,

thenumberofcontrollersinthesystemcanbemuchhigherthanatypical

network-basedstoragesolutionwithonlyafewcontrollers.Suchdesign

innovationsenablethesystemtostartsmallandscalemassively.

High AvailabilityNutanix Cluster RAID

NutanixCompleteClusterisahighlyavailablescale-outsystemwithnosingle

pointoffailure.UsingNutanixClusterRAID,dataisstripedacrossdiskswithina

nodeforhighperformanceandreplicatedacrosstheclusterforhighavailability.

Thisprovideshighavailabilityforvirtualmachinesevenifdisksorcomplete

nodesfail.

Nutanix Distributed Metadata Service (Medusa)

Nutanixsdistributedmetadataservice,Medusa,distributestheclustermetadata

acrosstheclusterforscalabilityandreplicatesthedataonmultiplenodesfor

faulttolerance.Theserviceishighlyavailableandcantoleratemultiplenode

failures.Incomparison,traditionalsolutionsthatdependonafewstorage

controllerscannottoleratemultiplefailures.

Nutanix Instant vDisk Motion

TheabilitytomigratealiveVMfromonehosttoanotherisaverypowerful

featureprovidedbytheindustry-standardhypervisors.Sofar,organizations

havebeenabletoleveragesuchlivemigrationcapabilitiesonlywithnetwork

KEYFEATURES

14NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

storage.Infact,manyorganizationswereforcedtousenetworkstorageonly

toachievelivemigrationandhighavailability.InNutanixCompleteCluster,live

VMmigrationissupportedevenwiththeconvergedarchitecturethatuseslocal

storage.vDisksintheappliancearelogicalentitiesthatarefullymobile.Thisis

enabledbyNutanixInstantvDiskMotionfeaturethatcanquicklymoveavDisk

fromonenodetoanother,whennecessary.

Backup and Recovery

Theabilitytoperformoff-sitedatabackupandrecoveryiskeytoan

organizationsdataprotectionstrategy.NutanixCompleteClusterisfully

compatiblewiththeVMwarevStorageAPIforDataProtection(VADP),and

providesfullyfunctionalbackupandrecoveryofvirtualmachines,through

integrationwithVADPcompatiblebackupandrecoveryproducts.

Disaster Recovery

DisasterRecoveryiskeytothebusinesscontinuityneedsofanenterprise.

NutanixCompleteClusterprovidesfailoverandfailbackcapabilitiesthrough

integrationwithVADPcompatibleDisasterRecoveryproducts.

CONCLUSIO

N

15NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER

ConclusionNutanixCompleteClusterisaninnovativesystemthateliminatestheneedfor

networkstoragewithoutcompromisingtherequirementsofenterpriseIT.It

isbuiltusinganext-generationscale-outarchitecturethathasbeenproven

atsomeofthemostinnovativecloud-generationcompanies.WithNutanix

CompleteCluster,organizationscanbuildacomputeandstorageinfrastructure

forvirtualmachinesthatishighlyavailable,fast,richindatamanagement

featuresandcangrowastheirneedsgrow.

Executive SummaryIntroductionLimitations of Current ArchitectureThe Google Approach

Nutanix Complete Cluster Architecture Main PillarsArchitecture OverviewNutanix Scale-Out Converged Storage Key Components

Key FeaturesCapacity OptimizationEase of ManagementPerformance and ScalabilityHigh Availability

Conclusion