Sock puppets

Download Sock puppets

Post on 25-Dec-2014

1.069 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

 

TRANSCRIPT

<ul><li> 1. Sock Puppets: Growing yourpuppet codebaseFabio Lessa &amp; Andrew Myers </li> <li> 2. Part 1 </li> <li> 3. Empty server room </li> <li> 4. Project teams teams </li> <li> 5. OPS team for app support </li> <li> 6. The codebase weve started withSomething that represents a mess </li> <li> 7. StandardPuppet Master setup </li> <li> 8. Code was split by environments </li> <li> 9. Workflow:1. Push change2. Wait for it to propagate to thatenvironment3. Check it workedIf you need that applied to anotherenvironment, go to 1 </li> <li> 10. The real Workflow:1. Push change1.1. Stop the puppet daemon in all the critical nodes for thatenvironment1.2. Run puppetd manually in one hostto ensure it works1.3. Start the daemon again (on all the hosts you remember)2. Wait for it to propagate to that environment3. Check it worked (meh)If you need that applied to another environment, go to 1 (if youremember, that is) </li> <li> 11. ProductionUAT System Testing CI </li> <li> 12. Making changeswas painful </li> <li> 13. Part 2 </li> <li> 14. "dont duplicate your manifests in your tests" Focus on the catalog problems he described earlier and test your logic. Dont test if puppet is doing its job, test that your logic its doing its job. - Nikolay Sturm </li> <li> 15. The goal...Before After </li> <li> 16. So how do we test/refactor </li> <li> 17. Compile Puppet Catalogues </li> <li> 18. Compare Versions </li> <li> 19. Application specific tests </li> <li> 20. Tying it all together </li> <li> 21. Part 3 </li> <li> 22. Continuous Delivery </li> <li> 23. We were moving application deploys, whynot do the same with infrastructure? </li> <li> 24. MCollective </li> <li> 25. The new setup </li> <li> 26. Picture of the pipelines </li> <li> 27. info: Loading facts in /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/hostinfo/lib/facter/hostinfo.rbinfo: Loading facts in /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/concat/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/hostinfo.rbinfo: Loading facts in /var/lib/puppet/facts/zone.rbinfo: Loading facts in /var/lib/puppet/facts/read_facts.rbinfo: Loading facts in /var/lib/puppet/facts/stomp_host.rbinfo: Loading facts in /var/lib/puppet/facts/puppetmaster_host.rbinfo: Caching catalog for somehost.nbndc.localerr: Failed to apply catalog: Could not find dependency File[/root/.hgrc] forMercurial::Repo::Clone[proddc2-repo] at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/puppetmaster/manifests/repos.pp:36 </li> <li> 28. info: Retrieving plugininfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbinfo: Loading facts in /var/lib/puppet/lib/facter/hostinfo.rbinfo: Loading facts in /var/lib/puppet/facts/zone.rbinfo: Loading facts in /var/lib/puppet/facts/read_facts.rbinfo: Loading facts in /var/lib/puppet/facts/stomp_host.rbinfo: Loading facts in /var/lib/puppet/facts/puppetmaster_host.rbinfo: Caching catalog for somehost.nbndc.localerr: Failed to apply catalog: You cannot specify more than one ofcontent, source, target at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/sbs/manifests/init.pp:73 </li> <li> 29. notice: Class[Hostinfo]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Java::Sun/Java::Jdk::Base[jdk-1.6.0.31-1jpp.1.el5]/Java::Jdk::Default[1.6.0.31-sun]/Notify[java::jdk::default:sun]/message:current_value absent, should be Setting java the following java as default:1.6.0.31 sun 1.6.0 (noop)notice: Java::Jdk::Default[1.6.0.31-sun]: Would have triggered refreshfrom 1 eventsnotice: Java::Jdk::Base[jdk-1.6.0.31-1jpp.1.el5]: Would have triggeredrefresh from 1 eventsnotice: Class[Java::Sun]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Greenmail/File[/etc/greenmail]/ensure: current_valueabsent, should be directory (noop)err: /Stage[main]/Greenmail/File[/etc/greenmail/greenmail.conf]: Could notevaluate: Could not retrieve information from environment noop source(s)puppet://puppet/modules/greenmail/etc/greenmail/greenmail.conf at /usr/share/puppet-recipes/noop/puppet-environments/puppet/modules/greenmail/manifests/init.pp:30notice: Class[Puppetagent]: Would have triggered refresh from 1 eventsnotice: /Stage[main]/Greenmail/File[/usr/greenmail]/ensure: current_valuedirectory, should be absent (noop)notice: /Stage[main]/Greenmail/File[/usr/greenmail/startgm.sh]/ensure:current_value file, should be absent (noop) </li> <li> 30. We were moving application deploys, whynot do the same with infrastructure? </li> <li> 31. Our MCollective agent </li> <li> 32. The new action </li> <li> 33. At every push, all nodes connect to the noop environment and run in noop mode </li> <li> 34. Picture of a report </li> <li> 35. Changemanagementprocess ? </li> <li> 36. What changed? </li> <li> 37. What did you test? </li> <li> 38. Who will do it? </li> <li> 39. Part 4 </li> <li> 40. New data centre is ready! </li> <li> 41. Should be easy to migrate.... </li> <li> 42. Okay so our manifests are incomplete,but we need to be able to rebuild machines... Virtualisation to the rescue... </li> <li> 43. Magnets? </li> <li> 44. What happens </li> <li> 45. Wrap Up </li> <li> 46. 120 pushes toproduction in thelast 8 months: 2failures </li> <li> 47. MTTR FTW! 5 minutes! </li> <li> 48. Thank you! </li> </ul>