![Page 1: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/1.jpg)
Load Balancing and Intelligent Load BalancingJesús González Escalation Engineer
![Page 2: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/2.jpg)
Objectives
• Deep understanding of load balancing architecture
• Troubleshooting techniques
• Identify the root cause of any LB problem
![Page 3: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/3.jpg)
Agenda• Architecture
• Data Collector • Updates - Examples
• Performance Data Helper• Citrix and the PDH
• Where can it go wrong?• Black hole effect (Load Throttling - Intelligent Load Balancing)• Some servers get all the connections• Full load after installing an MUI
• Troubleshooting
![Page 4: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/4.jpg)
Architecture
![Page 5: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/5.jpg)
Architecture
Data Collector DC - Dynamic Store DS
LmsSS.dll - IMA Subsystem
MFRules.dll - XA countersLMS20Rules.dll - System counters
PDH.dll - Performance Data Helper
![Page 6: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/6.jpg)
Data Collector Updates
• Data Collector is updated every 30 s
• Only when the change in the load evaluator is bigger than 5%• Every 5 minutes we send a full update
• Connection logon or logoff
![Page 7: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/7.jpg)
Data Collector UpdatesServer load - CPU Load example
• Data Collector
XenApp
Time
30s 60s
15 20 23 25 23 50 98 34 45 56
X 20 23 25 23 50 98 34 45 56 40
X 23 25 23 50 98 34 45 56 40 12
38 41 40Server Load % :
5 minutes
![Page 8: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/8.jpg)
Data Collector UpdatesXA CPU Load != Task Manager CPU Load
• CPU load
• XA = 25%
• Task Manager = 1%
![Page 9: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/9.jpg)
Ctxnotif.dll
MFSrvSS.dll
LMSSS.DLL
Data Collector UpdatesAt Logon time - Default load evaluator
Data Collector
ICA Client
Dynamic Store
Server A
Server B
IMASe
rver
B
Serv
er A
Serv
er A
BIAS
WI/XML
IMA
X
![Page 10: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/10.jpg)
Performance Data Helper – API
The performance data helper interface calls the registry interface to retrieve performance data
Uses the PDH.DLL to access the PDH APIhttp://msdn2.microsoft.com/en-us/library/aa373083.aspx
![Page 11: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/11.jpg)
Performance Data Helper – C++ Example
pdhStatus = PdhAddCounter ( hQuery,
"\\Processor(0)\\% Processor Time",
0,
&hCounter);
YOU MUST CALL THE PERFORMANCE COUNTER BY ITS NAME
![Page 12: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/12.jpg)
Citrix and the PDH
• \\Processor(_Total)\\% Processor Time
• \\System\\Context Switches/sec
• \\Memory\\% Committed Bytes In Use
• \\Memory\\Page Faults/sec
• \\Memory\\Pages/sec
• \\PhysicalDisk(_Total)\\Disk Bytes/sec
• \\PhysicalDisk(_Total)\\Disk Reads/sec
• \\PhysicalDisk(_Total)\\Disk Writes/sec
![Page 13: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/13.jpg)
Performance Data Helper - Registry
![Page 14: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/14.jpg)
Performance Data Helper – File system
EnglishPerfc009.datPerfh009.dat
GermanPerfc007.datPerfh007.dat
![Page 15: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/15.jpg)
Performance Data Helper - perfmon
![Page 16: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/16.jpg)
Performance Data Helper - IMA
IMA(At start up)
![Page 17: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/17.jpg)
Citrix and the PDH
We also provide Performance counters
![Page 18: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/18.jpg)
Citrix and the PDHHKLM\SYSTEM\CurrentControlSet\Services\IMAService\Performance
![Page 19: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/19.jpg)
Where can it go wrong?
![Page 20: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/20.jpg)
Black hole effect
Problem Solution• At Peak logon times a recently boot up server will get all connections and might become unresponsive
• Cause: Server is unable to update the DC
• Load Throttling
• Intelligent load balancingHKLM\SOFTWARE\Citrix\IMA\LMS\
(DWORD ) UseILB = 1
(DWORD ) ILBMultiplier = 2
![Page 21: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/21.jpg)
Before Intelligent Load Biasing
BIASCurrent Load (0)
Max Load (10000)
0 sessions
Current Load (100)
Max Load (10000)
1 session
BIAS
Max Load (10000)
2 sessions
Current Load (200)
1 session comes 2nd session comes
Default BIAS = 10000/100 = 100
![Page 22: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/22.jpg)
Intelligent Load Biasing
BIAS
Current Load (0)
Max Load (10000)
ILBMultiplier (2)
0 sessions
Current Load (5000)
Max Load (10000)
1 session
BIAS
Max Load (10000)
2 sessions
Current Load (7500)ILBMultiplier (2)
1 session comes 2nd session comes
Load = [(Max Load – Current Load) / ILBMultiplier] + Load
![Page 23: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/23.jpg)
Load Throttling- Low logon rateData CollectorICA Client
Dynamic Store
Server A
Server B
Server B
Server AServer Load BIAS
0
0 0
0(10000-0)/2+0 = 5000100
(10000-0)/2+0 = 5000100
![Page 24: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/24.jpg)
Load Throttling- Low logon rate
Server load if Low logon rate
0
1000
2000
3000
4000
5000
6000
ILB=1ILB=0
![Page 25: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/25.jpg)
Load Throttling- Black hole effectData CollectorICA Client
Dynamic Store
Server A
Server B
Server B
Server AServer Load BIAS
0
6000 0
0(10000-0)/2+0 = 5000(10000-5000)/2+5000 = 7500
(10000-6000)/2+6000 = 8000
(10000-7500)/2+7500 = 8750
![Page 26: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/26.jpg)
Load Throttling
Server load at Peak logon times
0
2000
4000
6000
8000
10000
12000
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99
ILBMultiplier=2ILBMultiplier = 10ILBMultiplier = 20ILB=0
![Page 27: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/27.jpg)
Upgrade to W2K3 causes full load
Problem Solution• English W2K
\\System\\% Total Processor Time
• English W2K3
\\Processor(_Total)\\% Processor Time
• In W2K3, use \\Processor(_Total)\\% Processor Time
![Page 28: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/28.jpg)
Some servers get all the connections
Problem Solution• 1 server all the connections
• Cause: Fail to read performance counters causes Load = 0
• Fail to read performance counters causes Load = 10000
![Page 29: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/29.jpg)
Full load after installing an MUI
Problem Solution• After installing MUI with advance load evaluators it results in load = 10000
• Cause:
-> perfc007.dat (German) perfc009.dat (English)
• HKLM\Software\Citrix\IMA\LMS\
EnableTranslation=1 (dword)
![Page 30: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/30.jpg)
Full load after installing an MUI
• Jesús González in a German restaurant
English Menu (perfc009.dat)22 . Vegetable Soup
German Menu (perfc007.dat)22. Gemüsensuppe
![Page 31: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/31.jpg)
Troubleshooting
![Page 32: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/32.jpg)
TroubleshootingData Collector DC LmsSS.dll
LMS20Rules.dll
PDH.dll
\\Processor(_Total)\\% Processor TimeXXXXXXXXXXXXXXXX
Full Load
![Page 33: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/33.jpg)
TroubleshootingFailing to read performance counters => full load
• Use procmon (filemon) while restarting IMAMake sure the correct perfcXXX.dat file can be accessed.
• Consider rebuilding performance countershttp://support.microsoft.com/kb/300956/en-us
• Check that no performance counter are disabled[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfDisk\Performance]"Disable Performance Counters"=dword:00000001
• Use CDFControl to gather CDF traceshttp://support.citrix.com/article/CTX111961
![Page 34: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/34.jpg)
•LMSRuleDll_Interface::CreatePDHQuery() ERROR!!!!! rc = -1073738823
•-1073738823 -> FFFFFFFFC0000BB9 (HEX)
•http://msdn2.microsoft.com/en-us/library/aa373046.aspx
“The specified counter could not be found”
TroubleshootingCDF Traces example
![Page 35: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/35.jpg)
Summary
• Deep understanding of load balancing• BIAS, PDH…
• Understanding Intelligent load balancing• Load Throttling
• Typical scenarios• Full load versus 0 load, MUI…
• Troubleshooting techniques• So far we are able to resolve all the problems that arrive to EMEA tech support with those tips
![Page 36: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/36.jpg)
Before you leave…
• Session surveys are available online at www.citrixsynergy.com starting Thursday, 7 October• Provide your feedback and pick up a complimentary gift card at the registration desk
• Download presentations starting Friday, 15 October, from your My Organiser Tool located in your My Synergy Microsite event account
![Page 37: Load Balancing and Intelligent Load Balancing](https://reader035.vdocuments.mx/reader035/viewer/2022081505/56815e43550346895dccb547/html5/thumbnails/37.jpg)