puppetcamp sydney 2012 - building a multimaster environment
DESCRIPTION
How we built a distributed Multi-master environment.TRANSCRIPT
MultiMaster scaling for multiple regions
Greg Cockburn @gergnz
problem:
How do we provide a Puppet Service Globally When WAN pipes suck
what's in our tool box?
VMware ESX LDAP
F5 Load Balancers Puppet Enterprise Edition
Items that need to be addressed • Puppet Certificate management
• Node Classification and ENC replication
• Master Replication
• Master Availability
• Master Scalability
• Reporting and notifications
• Change Control
One Solution that Worked
Build a Puppeteer:
• This is a Puppet Master Master
• No Client Access
• Acts as a PuppetCA
• Central Point of Entry for Code Updates
• Ensures that the Puppet Masters are in sync
LDAP as an ENC: • Existing highly available UNIX/Linux backbone service
• Already replicated to every region
• Masters are configured to speak with their nearest LDAP
replica
• Provides an effective audit trail
• Node definitions are abstracted away from the Puppet
manifests
Replicating Puppet Configuration: • The Puppet Master is effective at syncing files
• Use the Puppet Fileserver to replicate the masters o manifests
o modules
o files
o templates
• The Puppeteer can 'kick' the other masters to force a run
• Create a puppet::master class to ensure, masters are
fully controlled
F5 Global Traffic Management (GTM) & DNS:
• Local Puppet Master addresses are returned to
clients based on the DNS server the request
came from
• If a Master is down then next nearest is returned
• Any Puppet Master globally can answer the
client
F5 Local Traffic Management (LTM):
• On sites with heavy loads this can be used to
rapidly scale the local Puppet Master service
• If a local Master is taken out of service F5 will
automatically send you to the nearest local
Master
All Tied Together:
Workflow – Adding a New Server
• Define the client characteristics in the LDAP ENC (eg.
Datacentre, Environment, Server Flavour)
• Configure the build tools
• PXE boot then server, OS is installed and puppet
bootstraps
• Once the client certificate is signed the server is
configured
Workflow (adding a master):
• Build a 'standard' client
• Redefine in ENC (LDAP) as a puppetmaster
• Destroy local certificates
• generate special certificates on puppetmaster using --
dns_alt_names
• rerun puppet and Master configurations will sync down
So What’s New:
Since this configuration was deployed Puppet Labs have
been busy:
• Puppet Sites - Will soon be released and addresses a lot
of the issues here
• PuppetDB – The new standard for stored configs
Special thanks to Jon Spinks @ Sourced Group
Sourced Group are a Puppet Labs partner providing integration services for Puppet
Enterprise Edition
Q & A
Please go and bother Jon Spinks to find out what Sourced have been doing with Puppet to automate Amazon Web
Services