Sock Puppets: Growing Your Puppet Codebase

Download Sock Puppets: Growing Your Puppet Codebase

Post on 10-May-2015

1.976 views

Category:

Technology

3 download

DESCRIPTION

Presenters: Fabio Lessa & Andrew Myers

TRANSCRIPT

  • 1.Sock Puppets: Growing yourpuppet codebaseFabio Lessa & Andrew Myers

2. Part 1 3. Empty server room 4. Project teamsteams 5. OPS team for appsupport 6. The codebase weve started withSomething that represents a mess 7. StandardPuppet Master setup 8. Code was split by environments 9. Workflow:1. Push change2. Wait for it to propagate to thatenvironment3. Check it workedIf you need that applied to anotherenvironment, go to 1 10. The real Workflow:1. Push change1.1. Stop the puppet daemon in all the critical nodes for thatenvironment1.2. Run puppetd manually in one hostto ensure it works1.3. Start the daemon again (on all the hosts you remember)2. Wait for it to propagate to that environment3. Check it worked (meh)If you need that applied to another environment, go to 1 (if youremember, that is) 11. ProductionUAT System Testing CI 12. Making changeswas painful 13. Part 2 14. "dont duplicate your manifests in your tests" Focuson the catalog problems he described earlier and test your logic. Dont test if puppet is doing its job, test that your logic its doing its job.- Nikolay Sturm 15. The goal...BeforeAfter 16. So how do we test/refactor 17. Compile Puppet Catalogues 18. Compare Versions 19. Application specific tests 20. Tying it all together 21. Part 3 22. Continuous Delivery 23. We were moving application deploys, whynot do the same with infrastructure? 24. MCollective 25. The new setup 26. Picture of the pipelines 27. info: Loading facts in /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/hostinfo/lib/facter/hostinfo.rbinfo: Loading facts in /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/concat/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/hostinfo.rbinfo: Loading facts in /var/lib/puppet/facts/zone.rbinfo: Loading facts in /var/lib/puppet/facts/read_facts.rbinfo: Loading facts in /var/lib/puppet/facts/stomp_host.rbinfo: Loading facts in /var/lib/puppet/facts/puppetmaster_host.rbinfo: Caching catalog for somehost.nbndc.localerr: Failed to apply catalog: Could not find dependency File[/root/.hgrc] forMercurial::Repo::Clone[proddc2-repo] at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/puppetmaster/manifests/repos.pp:36 28. info: Retrieving plugininfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/hostinfo.rbinfo: Loading facts in /var/lib/puppet/facts/zone.rbinfo: Loading facts in /var/lib/puppet/facts/read_facts.rbinfo: Loading facts in /var/lib/puppet/facts/stomp_host.rbinfo: Loading facts in /var/lib/puppet/facts/puppetmaster_host.rbinfo: Caching catalog for somehost.nbndc.localerr: Failed to apply catalog: You cannot specify more than one ofcontent, source, target at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/sbs/manifests/init.pp:73 29. notice: Class[Hostinfo]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Java::Sun/Java::Jdk::Base[jdk-1.6.0.31-1jpp.1.el5]/Java::Jdk::Default[1.6.0.31-sun]/Notify[java::jdk::default:sun]/message:current_value absent, should be Setting java the following java as default:1.6.0.31 sun 1.6.0 (noop)notice: Java::Jdk::Default[1.6.0.31-sun]: Would have triggered refreshfrom 1 eventsnotice: Java::Jdk::Base[jdk-1.6.0.31-1jpp.1.el5]: Would have triggeredrefresh from 1 eventsnotice: Class[Java::Sun]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Greenmail/File[/etc/greenmail]/ensure: current_valueabsent, should be directory (noop)err: /Stage[main]/Greenmail/File[/etc/greenmail/greenmail.conf]: Could notevaluate: Could not retrieve information from environment noop source(s)puppet://puppet/modules/greenmail/etc/greenmail/greenmail.conf at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/greenmail/manifests/init.pp:30notice: Class[Puppetagent]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Greenmail/File[/usr/greenmail]/ensure: current_valuedirectory, should be absent (noop)notice: /Stage[main]/Greenmail/File[/usr/greenmail/startgm.sh]/ensure:current_value file, should be absent (noop) 30. We were moving application deploys, whynot do the same with infrastructure? 31. Our MCollective agent 32. The new action 33. At every push, all nodes connect to the noop environment and run in noop mode 34. Picture of a report 35. Changemanagementprocess? 36. What changed? 37. What did you test? 38. Who will do it? 39. Part 4 40. New data centre is ready! 41. Should be easy to migrate.... 42. Okay so our manifests are incomplete,but we need to be able to rebuild machines...Virtualisation to the rescue... 43. Magnets? 44. What happens 45. Wrap Up 46. 120 pushes toproduction in thelast 8 months: 2failures 47. MTTR FTW! 5 minutes! 48. Thank you!

Recommended

View more >