serverspec and sensu - testing and monitoring collide

23
+ MICHAEL RICHARDSON - @M_RICHO Testing and Monitoring Collide

Upload: mrichardson

Post on 08-Sep-2014

9.954 views

Category:

Technology


1 download

DESCRIPTION

Lightning talk from Sydney DevOps Meetup

TRANSCRIPT

Page 1: Serverspec and Sensu - Testing and Monitoring collide

+

MICHAEL RICHARDSON - @M_RICHO

Testing and Monitoring Collide

Page 2: Serverspec and Sensu - Testing and Monitoring collide

WHAT IS IT?

RSpec Tests for checking your servers are configured correctly

Page 3: Serverspec and Sensu - Testing and Monitoring collide

CM/PROVISIONING TOOL AGNOSTIC

Page 4: Serverspec and Sensu - Testing and Monitoring collide

CM/PROVISIONING TOOL AGNOSTIC

SERVERS CAN BE BUILT USING• CHEF• PUPPET• CF-ENGINE• ANSIBLE

• SALT STACK• SHELL SCRIPTS• MANUAL HACK• WHAT EVER.

Page 5: Serverspec and Sensu - Testing and Monitoring collide

26 RESOURE TYPES INCLUDING• COMMAND• CRON• DEFAULT_GATEWAY• FILE• GROUP• HOST• INTERFACE

• IPTABLES• KERNEL_MODULE• PACKAGE• PORT• PROCESS• SERVICE• USER

Page 6: Serverspec and Sensu - Testing and Monitoring collide
Page 7: Serverspec and Sensu - Testing and Monitoring collide

That’s super fast

Page 8: Serverspec and Sensu - Testing and Monitoring collide

WHEN TO RUN SERVERSPEC TESTS?

??

Page 9: Serverspec and Sensu - Testing and Monitoring collide

WHEN TO RUN SERVERSPEC TESTS?

That’s super useful

• When developing or updating config management code (eg puppet/chef)

• When validating a node or environment

Page 10: Serverspec and Sensu - Testing and Monitoring collide

WHEN TO RUN SERVERSPEC TESTS?

Hell yeah. That’s a great idea.

• What about every 2 minutes on all hosts orchestrated via your monitoring tool

Page 11: Serverspec and Sensu - Testing and Monitoring collide

WHEN TO RUN SERVERSPEC TESTS?

Hell yeah. That’s a great idea.

• What about every 2 minutes on all hosts orchestrated via your monitoring tool

Why?• Fast• Lightweight• Overlaps with traditional

monitoring checks. (Eg. Is the apache service running)

• Simplifies monitoring tool config

Page 12: Serverspec and Sensu - Testing and Monitoring collide

WE USE SENSU FOR MONITORING

It’s • Simple• Extensible• Lightweight• Simple to install on linux and windows.

Page 13: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 1+GET SENSU TO JUST RUN THE RSPEC TESTS

Page 14: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 1+

RSpec stdout isn’t great to simply pass back to a monitoring tool.

Page 15: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 1+SURE, WE CAN TIDY UP RSPEC STD OUT

But now have no idea which tests are failing

Page 16: Serverspec and Sensu - Testing and Monitoring collide

2 USEFUL FEATURES TO THE RESCUE

1. Rspec has a JSON format option

2. Sensu-Client has a local socket to accept ‘arbitrary’ check results

Page 17: Serverspec and Sensu - Testing and Monitoring collide

RSPEC JSON OUTPUT

4 exampletests

Summary results

Page 18: Serverspec and Sensu - Testing and Monitoring collide

SENSU CLIENT SOCKET

Send results here

Failures appear here

Page 19: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 2+

Page 20: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 2+RSpec in json format

Send each test result to sensu socket

Report summary serverspec results to back to sensu defined check

Iterate over results

Page 21: Serverspec and Sensu - Testing and Monitoring collide

ITERATION 2+

Summary results Individual test failures

Page 22: Serverspec and Sensu - Testing and Monitoring collide

SUMMARY

1. Write comprehensive tests2. Don’t duplicate effort by writing the same thing

again as a monitoring check.3. Simply run your tests via a monitoring tool

Page 23: Serverspec and Sensu - Testing and Monitoring collide

THANK YOU

Michael Richardson - @m_richohttps://github.com/m-richo/sensu_check-rspec