Download - An Introduction to Sensu by Bethany Erskine
![Page 1: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/1.jpg)
SENSE AND SENSU-BILITYPainless Metrics And Monitoring
In The Cloud with Sensu
Bethany Erskinenycdevops Meetup
http://github.com/skymob/sensu-tutorial
Thursday, November 14, 13
![Page 2: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/2.jpg)
DO YOU LOVE YOUR
MONITORING SETUP?
Thursday, November 14, 13
![Page 3: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/3.jpg)
#MONITORINGLOVE
Thursday, November 14, 13
![Page 4: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/4.jpg)
MY STORY
+
(╯(╰,)
Thursday, November 14, 13
![Page 5: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/5.jpg)
Thursday, November 14, 13
![Page 6: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/6.jpg)
Thursday, November 14, 13
![Page 7: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/7.jpg)
WHY SENSU
✓Ruby
✓Plugins can be written in any language
✓sensu-chef cookbook
✓community
Thursday, November 14, 13
![Page 8: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/8.jpg)
WHY SENSU
✓re-use Nagios checks
✓metrics and checks all collected by one system
✓Graphite integration
✓easy to scale
Thursday, November 14, 13
![Page 9: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/9.jpg)
WHY SENSU
✓“Can I do X with Sensu?” probably!
Thursday, November 14, 13
![Page 10: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/10.jpg)
WHY SENSU
Thursday, November 14, 13
![Page 11: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/11.jpg)
WHY SENSU?
✓Sensu source is well-written and easy to parse
✓https://github.com/sensu
Thursday, November 14, 13
![Page 12: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/12.jpg)
WHY SENSU?
✓sensu-community-plugins
✓80 contributors
✓over 600 plugins
✓https://github.com/sensu/sensu-community-plugins
Thursday, November 14, 13
![Page 13: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/13.jpg)
TODAY at PAPERLESS
Two Sensu environments (prod/testing)~ 250 - 275 instances of sensu-client
4-6 Sensu-server instances25k Metrics/Hour to Graphite
1 custom dashboard1 custom CLI
Thursday, November 14, 13
![Page 14: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/14.jpg)
RESOURCES✓All of our Sensu infrastructure is
virtualized.
✓We typically give a sensu-server box 1.5GB RAM and 4 processors, scaling up RAM for any box running more than one Sensu service on it.
✓ 4GB RAM for a monolithic Sensu install (Rabbit, Redis, all Sensu components on one)
Thursday, November 14, 13
![Page 15: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/15.jpg)
AS WE GREWGrowing pains and lessons learned...
Thursday, November 14, 13
![Page 16: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/16.jpg)
NEEDS MORE SENSU
✓High load on Sensu server
✓Backed-up queues in RabbitMQ
✓TIP: set up check to monitor the RabbitMQ ready queue size, you'll want an email when the queue grows about 10K and stays there
Thursday, November 14, 13
![Page 17: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/17.jpg)
HOW TO SCALE
✓Add more sensu-server instances
✓No special configuration needed
✓checks will be distributed in round-robin fashion to the sensu-servers
Thursday, November 14, 13
![Page 18: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/18.jpg)
GRAPHITE PAINS
✓symptoms: backed up queues in RabbitMQ, spotty graphs
✓cluster couldn’t keep up with the large amount of metrics we were now serving it via AMQP
Thursday, November 14, 13
![Page 19: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/19.jpg)
GRAPHITE PAINS
✓Solution: stop collecting metrics every 10 seconds (excessive!)
✓moved staging metrics to staging Graphite cluster
✓Moved prod Graphite cluster to SSD
Thursday, November 14, 13
![Page 20: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/20.jpg)
THE MIGRATIONor, How To Quit Nagios in Ten Easy Steps
Thursday, November 14, 13
![Page 21: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/21.jpg)
STEP 1: NUKE AND PAVE
Thursday, November 14, 13
![Page 22: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/22.jpg)
STEP 2: PLANMETRICS AND MONITORING SURVEY
Thursday, November 14, 13
![Page 23: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/23.jpg)
METRICS AND MONITORING SURVEY
Thursday, November 14, 13
![Page 24: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/24.jpg)
STEP 3: DEFINE GLOBALS
✓CHECKS: must be actionable!
✓METRICS: go nuts
✓HANDLERS: EMAIL for everything initially, added Pagerduty later.
Thursday, November 14, 13
![Page 25: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/25.jpg)
OUR GLOBALS
✓CHECKS: disk usage, swap usage, zombie processes, RO filesystems
✓METRICS: vmstat, disk usage, cpu, memory, interface and disk perf
✓HANDLERS: Email, Campfire, Pagerduty
Thursday, November 14, 13
![Page 26: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/26.jpg)
STEP 4: DEFINE SPECIFICS
✓For each server role, define additional states to be checked and alerted on:
✓Process Checks
✓System Checks
✓Service Checks
✓Service Metrics
Thursday, November 14, 13
![Page 27: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/27.jpg)
STEP 5: SET UP A PLACE TO TEST
✓Set up a permanent testing Sensu stack using your CM tool of choice
✓we used sensu-chef cookbook
Thursday, November 14, 13
![Page 28: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/28.jpg)
STEP 6: SET A WORKFLOW
✓Develop and document a workflow for implementing, testing, deploying and signing off on checks
✓You’ll get the best coverage if anyone (developers or ops) can easily add checks and metrics to Sensu
Thursday, November 14, 13
![Page 29: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/29.jpg)
EXAMPLE WORKFLOW
✓add new sensu_check definitions to the appropriate cookbook in Chef
✓deploy new check to staging env using Chef
✓Pull Request with sample graphs or alerts
✓Code Review from colleague
✓Deploy to Prod
Thursday, November 14, 13
![Page 30: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/30.jpg)
SENSU IN CHEF
Thursday, November 14, 13
![Page 31: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/31.jpg)
STEP 7: EXECUTE WORKFLOW
✓Starting with the low-hanging fruit (plugins that already existed in sensu-community-plugins repository), configure and deploy each check in the worksheet to the testing Sensu server
✓deploy sensu-client to a few select machines
Thursday, November 14, 13
![Page 32: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/32.jpg)
STEP 8: WATCH THE WATCHER
✓Set up some bare-minimum 3rd party monitoring for the Sensu servers
Thursday, November 14, 13
![Page 33: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/33.jpg)
Thursday, November 14, 13
![Page 34: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/34.jpg)
MONITOR THE MONITOR
✓Other ideas: have Testing Sensu monitor Prod Sensu
✓Sensu can collect metrics about itself
Thursday, November 14, 13
![Page 35: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/35.jpg)
STEP 9: ROLLOUT
✓Deploy your Production server infrastructure
✓Roll out the client and checks to the rest of the your prod environments.
Thursday, November 14, 13
![Page 36: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/36.jpg)
STEP 10: TUNE
✓Laissez le bon alertes roulent!
✓Expect to need to tune thresholds and alert occurrences.
Thursday, November 14, 13
![Page 37: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/37.jpg)
SENSU ARCHITECTURE
Thursday, November 14, 13
![Page 38: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/38.jpg)
SENSU ARCHITECTURE
Thursday, November 14, 13
![Page 39: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/39.jpg)
OMNIBUS INSTALLER
is awesome
Thursday, November 14, 13
![Page 40: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/40.jpg)
LET’S PLAY WITH SENSU
If you haven’t been able to get your sandboxes up and running,
please pair with someone near you.
Thursday, November 14, 13
![Page 41: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/41.jpg)
SANDBOX GOALS✓Get familiar with Sensu
configuration
✓Install a Handler
✓Deploy a check
✓Trigger an alert on that check
✓Give you something to take home and hack on
Thursday, November 14, 13
![Page 42: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/42.jpg)
OOPS
If you mess anything up:
vagrant halt; vagrant up
Worst case:
vagrant destroy; vagrant up
Thursday, November 14, 13
![Page 43: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/43.jpg)
TWO VIRTUALBOXES
Sensu-Server and Sensu-ClientVagrant/Chef
Centos 6.4Sensu Version 0.10.2
Thursday, November 14, 13
![Page 44: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/44.jpg)
SENSU CONFIGURATION
✓Please open up a terminal and SSH into both your sensu-server and sensu-client VMs
✓sudo su -
✓cd /etc/sensu
Thursday, November 14, 13
![Page 45: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/45.jpg)
SENSU CONFIGURATION✓/etc/sensu/config.json - config for
redis, rabbitmq, api and dashboard
✓/etc/sensu/conf.d/ - checks go here
✓/etc/sensu/conf.d/client.json - client configuration, subscriptions
✓/etc/sensu/{extensions|handlers|mutators|plugins}
Thursday, November 14, 13
![Page 46: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/46.jpg)
TRIGGER AN ALERT!
On sensu-client:
service sensu-client stop
Thursday, November 14, 13
![Page 47: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/47.jpg)
CHECK YOUR DASHBOARD
✓Open a web browser and go to http://10.254.254.10:8080
✓username: admin / password: secret
Thursday, November 14, 13
![Page 48: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/48.jpg)
HANDLERS
✓A HANDLER takes action on an event using a pipe, TCP, UDP, AMQP, or a set of other handlers
✓Examples: send an email, send event to Pagerduty, send metrics to Graphite
✓Default is “debug”
Thursday, November 14, 13
![Page 49: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/49.jpg)
HANDLER EXAMPLES
✓BASIC: send an email to ops@
✓ADVANCED: attempt to remediate the alert (i.e. run a custom script that spins up additional ec2 instances)
Thursday, November 14, 13
![Page 50: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/50.jpg)
HANDLERS
✓Let’s configure an EMAIL handler to send a informative email for an event.
✓/etc/sensu/handlers/mailer.rb plugin is installed for you, we just need to configure and install it
Thursday, November 14, 13
![Page 51: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/51.jpg)
CONFIGURE THE PLUGIN
{ "mailer": { "mail_from": "[email protected]", "mail_to": "[email protected]" }}
ON SENSU SERVER:vim /etc/sensu/conf.d/handlers/mailer.json
Thursday, November 14, 13
![Page 52: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/52.jpg)
CONFIGURE THE HANDLER
cp /etc/sensu/conf.d/handlers/default.json /etc/sensu/conf.d/handlers/email.json
vim /etc/sensu/conf.d/handlers/email.json
Thursday, November 14, 13
![Page 53: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/53.jpg)
EMAIL.JSON
"handlers": { "email": { "type": "pipe", "command": "/etc/sensu/handlers/mailer.rb" }}
Thursday, November 14, 13
![Page 54: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/54.jpg)
CHECK GEM DEPENDENCIES
/opt/sensu/embedded/bin/gem list | grep mail
Thursday, November 14, 13
![Page 55: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/55.jpg)
FIX PERMISSIONS
chown -R .sensu /etc/sensu/conf.d/
Thursday, November 14, 13
![Page 56: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/56.jpg)
RESTART SERVICES
service sensu-server restart
tail -100 /var/log/sensu/sensu-server.log | grep mail
Thursday, November 14, 13
![Page 57: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/57.jpg)
CHECKS✓Sensu-client runs CHECKS that are
defined and scheduled either locally (standalone) or on the sensu-server (subscription).
✓A CHECK sends a RESULT as an EVENT to a HANDLER - this applies to anything - service checks, metrics, etc
Thursday, November 14, 13
![Page 58: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/58.jpg)
CHECK EXECUTION
✓Either scheduled by the server (subscription) or scheduled by the client (standalone)
✓Today we will configure a subscription-based check on the server that will run on our client
Thursday, November 14, 13
![Page 59: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/59.jpg)
LETS CONFIGURE A CHECK
✓Use check-procs.rb to make sure at least one instance of cornbread is running
Thursday, November 14, 13
![Page 60: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/60.jpg)
DETERMINE OUR CHECK COMMAND
On your SENSU CLIENT:
/opt/sensu/embedded/bin/ruby /etc/sensu/plugins/check-procs.rb -p cornbread -W1
Thursday, November 14, 13
![Page 61: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/61.jpg)
INSTALL OUR CHECK
✓On your SENSU SERVER:
✓vim /etc/sensu/conf.d/checks/cornbread_process.json
Thursday, November 14, 13
![Page 62: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/62.jpg)
CORNBREAD_PROCESS.JSON
Thursday, November 14, 13
![Page 63: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/63.jpg)
RESTART SERVICES
service sensu-server restart
tail -100 /var/log/sensu/sensu-server.log | grep cornbread
Thursday, November 14, 13
![Page 64: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/64.jpg)
CHECK YOUR DASHBOARD
Thursday, November 14, 13
![Page 65: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/65.jpg)
CHECK YOUR EMAIL
Thursday, November 14, 13
![Page 66: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/66.jpg)
SENSU API
✓REST API
✓HTTP/4567
✓on SENSU SERVER try:
curl -l http://localhost:4567/events \ | python -mjson.tool
Thursday, November 14, 13
![Page 67: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/67.jpg)
SENSU SERVICES
✓Sensu API
✓Sensu Server
✓Sensu Client
✓Sensu Dashboard
Thursday, November 14, 13
![Page 68: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/68.jpg)
EVERYTHING OK?
✓/etc/init.d/sensu-service {client|server|api|dashboard} {start|stop|status|restart}
✓ps -ef | grep sensu
✓tail -f /var/log/sensu/*.log
✓curl -l localhost:4567/info
Thursday, November 14, 13
![Page 69: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/69.jpg)
COOL SENSU TRICKS
Thursday, November 14, 13
![Page 70: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/70.jpg)
SEND DIRECTLY TO SENSUnetcat to: 127.0.0.0:3030
Thursday, November 14, 13
![Page 71: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/71.jpg)
AGGREGATE ALERTS
✓Handy for preventing alert floods
✓Alert when X% of checks are are not OK
Thursday, November 14, 13
![Page 72: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/72.jpg)
MY SENSU TIPS
✓install the RabbitMQ management web interface and bookmark it (see http://10.254.254.10:15672/#/ )
✓lock your plugins’ gem dependency versions
Thursday, November 14, 13
![Page 73: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/73.jpg)
TIPS TIPS TIPS
✓have alternate ways to access your Dashboard information
✓we integrated our command-line developer tools with Sensu API
✓we also created our own Ops dashboard that queries Sensu, Graphite and our app for data
Thursday, November 14, 13
![Page 74: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/74.jpg)
MORE TIPS
✓Put NGINX in front of sensu-dashboard
Thursday, November 14, 13
![Page 75: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/75.jpg)
HA SENSU
✓Redundancy is easy (bring up more sensu-servers)
✓Making Redis and RabbitMQ HA more challenging
✓We’re still running one solitary Redis and RabbitMQ but are OK with this risk for now
Thursday, November 14, 13
![Page 76: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/76.jpg)
WHERE TO GO FOR HELP
✓http://docs.sensuapp.org
✓IRC: #sensu - freenode
✓sensu-users mailing list
Thursday, November 14, 13
![Page 77: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/77.jpg)
QUESTIONS
Thursday, November 14, 13
![Page 78: An Introduction to Sensu by Bethany Erskine](https://reader034.vdocuments.mx/reader034/viewer/2022042614/554f455fb4c905423f8b48a5/html5/thumbnails/78.jpg)
THANK YOU
[email protected]@skymob - twitter
robotwitharose - #sensu on IRC (freenode)
Thursday, November 14, 13