troubleshooting k1000
TRANSCRIPT
1Dell World User Forum
UFIL523: Troubleshooting K1000
Mark Weathersbee, Senior TrainerVeryl White, Senior Trainer
Dell WorldUser Forum
3Dell World User Forum
Agenda
• Troubleshooting Overview - Circling the problem
• How to maintain your K1000 Appliance
• What’s running on your K1000
• System Performance Checklist
• Logs
• Deployments
• Agent Health
5Dell World User Forum
Troubleshooting Overview
1. Investigate
2. Analyze
3. Resolve
4. Verify
5. Document
Investigate
Analyze
ResolveVerify
Document
6Dell World User Forum
Investigate
Deployment – Check Logs
Scripting – Check Logs Connection – Check
Logs Patching – Check Logs Managed Installs –
Check Logs Replication – Check Logs Ticket Rules – Check
Logs
What Broke?
What Worked?
Where?
What Changed?
7Dell World User Forum
Analyze
• Look into what happened– Check logs– Check Details screen
• Compare expected outcome to reality
• What can be done to fix it? Does it work if you don’t use the
K1000? Does it work on other systems? Different Operating System? Different domain? Look in Admin Guide Look in Support KB
8Dell World User Forum
Resolution
• Once you know where it broke, how it broke, etc.
• Fix it!
• One thing at a time
Change the package
Change the settings
Change the network
Change the client
9Dell World User Forum
Verify & Document
• Make sure it works
• Remember what happened
Expected results on test machine?Try a larger setUse your Knowledge Base to document
11Dell World User Forum
Maintaining your K1000Ask yourself:
• When was our last backup?• Where are the backups?• Are we on the current server/agent
versions? • What is our agent check-in interval?• How many remote sites are still
running? • Are all of my agents current and
checking in?• What about network bandwidth? • Are there old/unused items in the
database?• Are my patch subscriptions current?
12Dell World User Forum
K1000 Appliance Health Cycle Checks
Backup
Upgrade Agent
Download Upgrade Packages
Upgrade Server Version
System Performance Check
Agent Tasks Check
Replication
Patches
Dell Updates
Log filesMunin Graphs
Oval Runs
1
2
3
4
1
2
3
4
5
6
78
14Dell World User Forum
What’s Running on your K1000 Appliance?
Agent – Have you checked what the Agents are doing behind your back? • Run Interval / Inventory Interval• Scripting Update / Ping Interval• Task Throughput / Run Now Tasks• AMP (Agent Messaging Protocol / Green Light)
Scheduled Tasks – Are you aware that there are “Scheduled Tasks” running? • Patching / OVAL / Replication / Metering• Agent Provisioning / Scheduled Reports / Ticket Rules• Dell Updates / Scripts / IP Scan / Distributions
System Performance – Have you checked and fine tune your K1000? • Web Server / Database / CPU / Memory / Disk IO (RAID) / Network• Hardware Spec / Virtual• Connections (agents/webui/downloads/uploads) / Tasks
15Dell World User Forum
Tasks & Queues
• What kind of tasks?• Where’s the task list?• How is the list processed?• What’s the Task Throughput setting do?• What do I do when things get stuck?
Types of Tasks• Inventory• Scripting• Upload Scripting Logs• Patching• Replication• Metering• Run Now• Dell Updates• Alerts• Bootstrap
16Dell World User Forum
Agent Task List
• What Task are “In Progress”?
• http://k1000/adminui/konductor_tasks.php
17Dell World User Forum
K1000 Task Throughput Update
Task Throughput
/adminui/settings_client.php
• Inventory Updates
• Patching Updates
• Scripts Execution
19Dell World User Forum
Replication Agent• Reduces load on the K1000
• Reduces network bandwidth bottlenecks
• Better utilization of network bandwidth
• Smart distribution
Replication Share
Replication Agent\\machinename\foldername\repl2\replicationitems folder Replication Clients
Failover
20Dell World User Forum
Replication Share
• Replication Off – during Peak Hours
• Do you need to “Replicate App Patches”?
• Do you need to “Replicate Dell Packages”?
• Have you select and limit to only the O/S Patching files required?
21Dell World User Forum
K1000 Troubleshooting Tools
• http://k1000/adminui/troubleshooting.php
• Network Troubleshooting and testing – ping, arp, dig, ifconfig, iostat, netstat, smbstatus
• Database Test
• Email Sending Test
• Services
23Dell World User Forum
How to Analyze K1000 Server Logshttp://k1000/adminui/settings_logs.php
• DD Status
• K1000 Logs
• Server Errors
• Munin Graph
• OpCode Cache
• Reporting Logs
• Reporting Error
• Client Errors
24Dell World User Forum
Logs > System Performance > Munin LogsApache• Accesses• Processes• Volume
Disk• File system usage• Inode usage
Exim• Mail throughput
KBOX• Connections• KBOX agent tasks• Long tasks• Task throughput
Samba• Status
MySQL• Throughput• Queries• Slow queries• Threads
Network• Traffic• Errors & collisions• Netstat• processes• VMstat
System• CPU usage• Load average• Memory usage• Swap in/out
25Dell World User Forum
Munin Graphs Tells a Talehttp://k1000/munin/k1000/k1000.html
• Apache - Webpage
• Disk - HDD
• Exim – Email
• Kbox – Agents
• MySQL - Database
• Network -
Traffic
• Processes – No.#
• Samba - Share
• System - Hardware
28Dell World User Forum
Keys to a Successful Managed Install
• Gather information: who, what, when etc..
• Double check file version with software version
• Research command lines
• Test
• Test
• Test
• Deploy to environment
29Dell World User Forum
Keys to a failed Managed Install
• Deploy any version of the software, patching can fix it right?
• Grab some command lines from the previous version you used, they’ll work right?
• Don’t test
• Deploy anytime, who cares if it’s silent
30Dell World User Forum
Where to begin troubleshooting a Managed Install
• Check the status
• Is the machine in the list?
• When to look at logs
32Dell World User Forum
Keys to a successful Script deployment
How to make scripts fail• Commands don’t care what OS they run
in?• Not using proper credentials for a script• Not scheduling the script properly
How to make a script succeed• Your script vs built in wizard• Appropriate dependency files• Do you have/need a verify step?• Test
34Dell World User Forum
Agent deployment configuration
• Advanced scan
• When to schedule
• How to update the agent after deployment
35Dell World User Forum
Agent deployment failures
• TCP ports/firewall
• Unable to download the agent file due to DNS
• Credentials
• UAC
38 Dell World User Forum
KACE Support Portal Migrating to Dell Software Support Portal
• Starting in November, all KACE Support Portal material will be migrated to the Dell Software Support Portal
• All service requests will be submitted online or by phone
• Same great content– Knowledge base articles– Video tutorials– Product documentation– JumpStart training
• Check out the Support Portal Getting Started videos