Puppet At Twitter - Puppet Camp Silicon Valley

Download Puppet At Twitter - Puppet Camp Silicon Valley

Post on 23-Dec-2014




0 download

Embed Size (px)


Andrew Hamilton gives a talk about "How Twitter uses Puppet" at Puppet Camp Silicon Valley 2014


  • 1. Puppet @Twitter Andrew Hamilton @ahamilton55

2. Who am I? 3. What well cover 4. The Dev in DevOps 5. Version Control 6. Peer review of code 7. Review Board http://www.reviewboard.org/ 8. Define your style 9. OWNERS 10. OWNERS example Moving from Puppet 2.7 to Puppet 3.x Need to go from: source => puppet:////to: source => puppet:///modules// 11. But of course 12. You can automate all of this! 13. Review Board git review tools Style Guide puppet-lint OWNERS Script a hook and git review tools Version control hooks 14. And always iterate on this 15. Branching 16. Head Our true testing branch Where development happens Systems dont live on this branch 17. Testing Works along with our canary system for deployments Major changes can sit here for a couple days to weeks 18. Production 99% of our systems are on the production branch Testing on puppet and head should prevent any error from propagating to production 19. Workflow HeadTestingProductionDevelopCherry pickCherry pickReview BoardReview BoardReview BoardCommitTestCommitTestCommit 20. But it could be better 21. puppetmasterd 22. Each host runs 3 puppetmasterd processes Each can serve between ~1k 2k hosts We use round-robin DNS 23. ENC 24. Audubon 25. Facts Mostly deprecated, legacy Pulled info from Facter 26. Attributes Basic host information Two levels - server, role Allows for inheritance 27. Groups Allows for a logical grouping of hosts Tells you things like role, rack, cluster, etc 28. Pulling in info from the ENC is easy We have functions that pull in each type of change in a manifest attribute(, ) group(, ) 29. Our ENC data store is also what controls the puppet branch our hosts use We set a group called puppet_branch which is used by our client cronjob 30. Agents 31. 1. Find branch the host is using2. Find the port that branch uses on masters 3. Get a master from DNS 1. Health check the master 2. If health continue, else repeat until timeout4. Run puppet apply against the master and port 32. Modules 33. Service ModulesTeam ModulesBase Modules 34. Create a module for each system service Chain these together in your service modulesIf you do something often, create a module that every group can use 35. Build for self-service 36. Looking forward