trevor mcdonald - nagios xi under the hood
TRANSCRIPT
Who am I?
●Support Manager, Nagios Enterprises
–tmcdonald on Support Forum
–https://github.com/tmcnag
–NWC2014 - Nagduino
●Non-Work
–World languages
–Computer security
Intro●Scope
–Nagios XI 5 w/ Nagios Core 4.1.1
–ndomod in use
–Bulk mode with NPCD
●Scope (but still mostly valid)
–Pre-2014 XI
–Pre-4 Core
–mod_gearman / DNX / remote agents
Intro●Files
–Flat files–RRD–MySQL–Postgres (sorta)
●Processes
–nagios–ndo2db–cron–npcd
Overview●Check is run
–Exit code & status output stashed
–Performance data split off
–Event handlers and/or notifications launched
●Perfdata processed
–Multi-step process
●Reports, Web GUI, etc.
Check is Run
●Check hits next_check_time (status.dat)
●execvp('/path/to/plugin', *args);
●Results are reaped and passed along
●They look like this
–PING OK - Packet loss = 0%, RTA = 0.40
ms|rta=0.401000ms;400.000000;800.000000;0.000000
pl=0%;40;80;0
Results are Processed
●Exit Code
●Status Output
–Performance Data is included here, everything after the “|”
character
●Not much* done with these
–*That I will be covering today
Exit Code/Status Output
●Goes many places:
–status.dat
–retention.dat
–nagios.log if non-OK
–syslog (optional, enabled by default)
–ndo database (optional*, enabled by default)
Performance Data●Split from status output after “|” character
●Handled by
–Nagios
–Cron
–NPCD
●Also goes many places:
–Flat files – XML files
–Databases – RRD files
Performance Data
Performance Data●Standardized format
'label'=value[UOM];[warn];[crit];[min];[max]
Performance Data●Standardized format
'label'=value[UOM];[warn];[crit];[min];[max]
rta=0.401000ms;400.000000;800.000000;0.000000 pl=0%;40;80;0
Performance Data●Standardized format
'label'=value[UOM];[warn];[crit];[min];[max]
rta=0.401000ms;400.000000;800.000000;0.000000 pl=0%;40;80;0
label value warn crit min
max
rta 0.401000ms 400.000000 800.000000
0.000000
pl 0% 40 80
0
Nagios
●Stores in:
–…/var/[host|service]-perfdata
●Using the form defined by:
–[host|service]_perfdata_file_template
●Then every 15 seconds (by default)–[host|service]_perfdata_file_processing_interval
●Nagios will run:–[host|service]_perfdata_file_processing_command
Nagios
●process-[host|service]-perfdata-file-bulk
●
/bin/mv
…/var/[host|service]-perfdata
…/var/spool/xidpe/$TIMET$.perfdata.[host|service]
Cron
●/usr/local/nagiosxi/cron/perfdataproc.php
●Runs every minute, but in a 55-second loop
●Moves files into …/var/spool/perfdata/
●Also handles Outbound Transfers
NPCD
●Does the real processing legwork
●Every 15 seconds by default:
–…/libexec/process_perfdata.pl
which places processed files into:
–…/share/perfdata/<hostname>/<servicedesc>.rrd
…/share/perfdata/<hostname>/<servicedesc>.xml
NPCD
Event Handlers/Notifications
●Standard Nagios logic takes over
●Event Handlers run on every state change
–Some only take action for certain states
●Notifications run after max_check_attempts
–XI Notification process is…
●Complicated
●Somewhat proprietary
Notifications
●Contacts have notification commands
–notify-[host|service]-by-email
–xi_[host|service]_notification_h
andler
●Core basically just calls sendmail with args
●XI can
–Use SMTP – Set importance
Notifications
●SMTP
–Authenticated, trusted
–Encryption possible
●Importance
–Marks as High-Priority
–Dependent on mail client
Notifications
●SMS
–Convenient
–Faster response times
●Methods
–Warning to email
–Critical to SMS
Notifications
●PHP script
/usr/bin/php
/usr/local/nagiosxi/scripts/handle_nagioscore_notification.php
●Plus tons of arguments
--notification-type
--servicestate
--contact
Notifications
●Pulls user info from db
–MySQL in XI 5
–Postgres in 2014 and older
●Formatted nicely
–Configurable
●Sent via PHPMailer
Reports/GUI●Pull primarily from these files
–nagios.log
–Archived logs in …/var/archives/
–RRD files in …/share/perfdata/…
●And from many db tables, such as
–nagios_acknowledgments
–nagios_statehistory
–nagios_notifications
Thanks To●Scott Wilkerson
–Explaining finer points of XI, general presentation advice
●John Frickson
–Clearing up Core logic
●Amy Lohmann
–Formatting and consistency
●Jesse Olson
–Guinea pig
Thank you!
Any Questions?
