tiering in glusterfs: hardware config considerations
TRANSCRIPT
TIERING IN GLUSTERFS:
Hardware Config Considerations
Veda Shankar, Red Hat
April 2016
AGENDA • Introduction • Basics: Concepts • Basics: Data Placement • Basics: Accessibility and Deployment • Tiering Architecture • Tiering Commands • Tiering Volume Settings • Tiering Performance • Tiering Demo
Distributed Volume
Two-way Distributed Replicated Volume
Three-way Distributed Replicated Volume
Dispersed Volume (Erasure Coded)
Distribute-ReplicateVolume(Primary)
Distribute-ReplicateVolume(Primary)
Tiering Architecture
Benefits of Tiering
Tiering Commands
A8achthe:ertothevolumebyexecu:ngthefollowingcommand:# gluster volume tier test-volume attach replica 2 server1:/gluster/ssd1 server2:/gluster/ssd2 Usethesetcommandtosetthevarious:ervolumeconfigura:onop:ons:# gluster volume set test-volume <option name> <value>Thestatuscommanddisplaysthe:eringvolumeinforma:on.# gluster volume tier test-volume status !Todetachthehot:erfromthevolume,firstini:atethedetachstartcommandandthenmonitortomakesurethatthedetachprocesshascompletedsuccessfully.Finallyusethecommitop:ontocompletethedetachprocess.# gluster volume tier test-volume detach start !# gluster volume tier test-volume status !# gluster volume tier test-volume commit !
Tiering Volume Options Op#on Descrip#on Allowed
ValuesDefaultValue
cluster.read-freq-threshold Specifiesthenumberofreads,inapromo:on/demo:oncycle,thatwouldmarkafileHOTforpromo:on.AnyfilethathasreadhitslessthanthisvaluewillbeconsideredasCOLDandwillbedemoted.
0-20 0
cluster.write-freq-threshold
Specifiesthenumberofwrites,inapromo:on/demo:oncycle,thatwouldmarkafileHOTforpromo:on.AnyfilethathaswritehitslessthanthisvaluewillbeconsideredasCOLDandwillbedemoted.
0-20 0
cluster.:er-promote-frequency Specifieshowfrequentlythe:erdaemonmustcheckforfilestopromote. 1–172800seconds
120seconds
cluster.:er-demote-frequency
Specifieshowfrequentlythe:erdaemonmustcheckforfilestodemote.
1–172800seconds
3600seconds
cluster.:er-mode Ifsettocachemode,promotesordemotesfilesbasedonwhetherthecacheisfullornot,asspecifiedwithwatermarks.Ifsettotestmode,periodicallydemotesorpromotesfilesautoma:callybasedonaccess.
Test|cache cache
TieringVolumeOptions Op#on Descrip#on Allowed
ValuesDefaultValue
cluster.:er-max-mb SpecifiesthemaximumnumberofMBthatmaybemigratedinanydirec:onfromeachnodeinagivencycle.
1–100000(100GB)
4000MB
cluster.:er-max-files
Specifiesthemaximumnumberoffilesthatmaybemigratedinanydirec:onfromeachnodeinagivencycle.
1–100000files
10000
cluster.watermark-hi Upperpercentagewatermarkforpromo:on.Ifhot:erfillsabovethispercentage,nopromo:onwillhappenanddemo:onwillhappenwithhighprobability.
1–99% 90%
cluster.watermark-low
Lowerpercentagewatermark.Ifhot:erislessfullthanthis,promo:onwillhappenanddemo:onwillnothappen.Ifgreaterthanthis,promo:on/demo:onwillhappenataprobabilityrela:vetohowfullthehot:eris.
1–99%
75%
QCT Storage Servers
Test Configuration
TwoQuantaPlexT21P-4U/DualNodeservers(fourGlusterfsnodes)wereconfigured,eachnodewitha40GbEinterface,withthetestworkloaddrivenby16clientnodes.Clientshavea10GbEinterface.
Test Configuration
SixQuantaGridD51PH-1Userverweretestedwithclusterandpublic10GbEnetworks.Thetestworkloadwasdrivenby16clientnodes.
DEMO
THANK YOU!
bit.ly/glustertestdrive
Test drive: