ccat troubleshooting training – provisioning services april 2012 citrix consulting architecture...
TRANSCRIPT
![Page 1: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/1.jpg)
CCAT Troubleshooting Training – Provisioning Services
April 2012
Citrix Consulting Architecture Team
![Page 2: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/2.jpg)
• PVS Troubleshooting – Key Areas
• WireShark Analysis
• System Memory Dumps
• Key Takeaways
Agenda
![Page 3: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/3.jpg)
Troubleshooting PVS Issues
![Page 4: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/4.jpg)
PVS Ports used - RecapService Protocol Port Comments
DHCP UDP 67 DHCP Server port
UDP 68 DHCP Client port
PXE UDP 67 If PXE separate from DHCP server
UDP 4011 If PXE on DHCP server
TFTP UDP 69 Trivial File Transfer Protocol
TSB UDP 6969 Two-Stage Boot Service
Console TCP 54321-54322 SOAP Service
Database TCP 1433 Microsoft SQL Server
Stream Service UDP 6890-6909 Inter-server communications
UDP 6910-6930 Streaming, initial login on port 6910
![Page 5: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/5.jpg)
Troubleshooting
• Narrowing down scope of issue by determining where in the boot cycle the issue is occurring.
• Packet Traces• Capturing Kernel Dump Files
![Page 6: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/6.jpg)
Boot Stages
• DHCP – PXE boot downloading of bootstrap file• Login to the PVS Server• Streaming of the I/O Pre-BNIStack (Citrix Target Device
driver)• BNIStack up burst traffic
![Page 7: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/7.jpg)
Using Wireshark – Packet Trace Analysis
![Page 8: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/8.jpg)
Analyzing Packet Captures
![Page 9: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/9.jpg)
Packet Capture PXE TFTP download
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
![Page 10: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/10.jpg)
Common Issues During DHCP TFTP Download
Target does not receive IP address
Target gets IP but times out to TFTP
PXE Errors No boostrap found
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
![Page 11: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/11.jpg)
Login Process Traffic
TFTP
Provisioning Server 10.1.1.1
Target 10.1.1.100
Stream Process
TFTP - Login
![Page 12: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/12.jpg)
Common issues during login process
Incorrect Bootstrap IP information
Login port blocked
Target not entered in Database
Stream Service failed to start
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
![Page 13: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/13.jpg)
Streaming Pre-BniStack
TFTP
Stream Process
OS Booting UNDI Driver
Target 10.1.1.100
![Page 14: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/14.jpg)
Common issues pre BNIStack
• Large Send Offload
• Network instability
• Conflicts with UNDI Driver
![Page 15: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/15.jpg)
Determining if BNIStack Driver is up
TFTP
Stream Process
OS Booting BNISTACK Driver
Target 10.1.1.100
Provisioning Server 10.1.1.1
![Page 16: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/16.jpg)
Common issues post BNIStack
• Filter Driver conflicts
• Firewall/Antivirus Services kickoff
• Antivirus Updates being loaded
• Domain Profile creation
![Page 17: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/17.jpg)
Capturing Windows System Dumps
![Page 18: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/18.jpg)
Windows Kernel Dump files
![Page 19: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/19.jpg)
Capturing Kernel Dump Files
• Capturing Kernel Dump files using Win2k3 and XP
• Capturing Kernel Dump files using Win2k8 Vista and Windows 7
• Capturing Kernel Dump files in XenServer and ESX
![Page 20: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/20.jpg)
Capturing Kernel Dump Files Win2k3 and XP Limitations
• Crash dumps to be on same drive as pagefile
• Page file to be on the system drive
• Redirection of dump not supported
• Live debug can be accomplished via serial cable.
![Page 21: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/21.jpg)
Capturing Kernel Dump Files Win2k8 Vista and Windows 7
• Enhancements allow kernel dumps to be directed to a location other then a system drive
• Requires a local hard drive with enough free space for the dump file. (Pagefile Size)
![Page 22: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/22.jpg)
Capturing Kernel Dump Files Win2k8 Vista and Windows 7
• Create a new string value under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl named DedicatedDumpFile
• Create Value data box, type <drive>:\<dedicateddumpfile.sys>
• Create DWORD Value DumpFileSize use decimal under base and type the name of dump file in megabytes
• More detailed information in this process can be found:http://support.microsoft.com/kb/969028 http://support.citrix.com/article/ctx123642
![Page 23: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/23.jpg)
Capturing a Kernel dump via XenServer
• Vista, Windows 7, Win2k8 redirection only possible if XenTools are not installed
• Live Debug via WinDBG possible by redirecting serial to DOM 0
![Page 24: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/24.jpg)
Capturing a Kernel Dump Via Xenserver
DO
M 0
XEN SERVER
WinDBGSockPipe.EXESerial Redirect
![Page 25: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/25.jpg)
http://support.citrix.com/article/ctx124979
![Page 26: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/26.jpg)
Capturing Kernel dump via ESX
• Workstation 7 utility vmss2core
• Suspend hung VM
• Download suspended VM session (.vmss)
• Retrieve downloaded file and run command vmss2core –W filename.vmss
![Page 27: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/27.jpg)
http://support.citrix.com/article/ctx125086
![Page 28: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/28.jpg)
Booting Target Devices – TFTP vs. BDM
• Whiteboard Discussion
![Page 29: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/29.jpg)
PVS Retries – Potential Causes
• Network Traffic • High bandwidth utilization• Congested pipe • Application layer filtering• Physical network problems• UDP (LSO, Checksum Offload) Issues
• Check the vDisk Properties (System Tray)
• How to fix these issues? (Discussion)
![Page 30: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/30.jpg)
Key Takeaways
![Page 31: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/31.jpg)
Key Takeaways
• Stream Process Workflow
• How to identify where in the boot process a PVS Target is failing
• Steps to capture a kernel level dump file from a provisioned target
![Page 32: CCAT Troubleshooting Training – Provisioning Services April 2012 Citrix Consulting Architecture Team](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649d895503460f94a6eb71/html5/thumbnails/32.jpg)