puppet camp dc 2014: managing puppet with mcollective

Download Puppet Camp DC 2014: Managing Puppet with MCollective

Post on 27-Aug-2014

475 views

Category:

Software

4 download

Embed Size (px)

DESCRIPTION

Puppet Camp DC 2014: "Puppet Type and Provider Execution Presentation" by Jon Mosco, Corporation Services Company

TRANSCRIPT

  • Managing Puppet with MCollective Jon Mosco @jpmosco
  • Who am I? Infrastructure Engineer Puppet User for 2+ years Puppet Certied Professional Linux user for 10+ years (Slackware!) Metal Head m/
  • What is MCollective?
  • The Marionette Collective (MCollective) is a framework to build server orchestration or parallel job execution Works perfectly along side Puppet providing real-time command execution across your infrastructure This is what it really is
  • Configuration Management Focuses on single node management Great at managing one server at a time
  • Orchestration Coordination of management of multiple systems
  • MCollective Architecture Servers == Puppet Agent Clients Middleware == Puppet Master Client Middleware Server Server Server Server
  • Servers Systems that can be controlled via MCollective Run the daemon mcollectived puppet agent == mcollectived Multiple Agent Plugins small Ruby libraries that enable custom command execution
  • Clients Clients send and receive requests to servers Admin server for managing MCollective Security and connector plugin mco command-line client interactive usage or in scripts
  • Middleware ! Clients and servers communicate through the middleware system (ActiveMA, RabbitMQ) MCollective uses a publish/subscribe system for all communication Sends and receives messages using STOMP Used for transporting messages between clients and servers
  • Message Flow Broadcast paradigm Messages are sent from the administrator account on the client server to the middleware server and broadcast to all nodes Each agent is both a publisher and subscriber
  • MW MW = Middleware S = Server s s s Client mco
  • Plugins Enable MCollective to execute commands on the entire collective Provide the functionality to use with MCollective puppet agent, service, process, package, etc.
  • Integration with Puppet Pre-installed and congured with Puppet Enterprise Puppet Labs modules for opensource https://github.com/puppetlabs/puppetlabs- mcollective
  • PuppetLabs MCollective Module
  • Orchestrating Puppet MCollective can be used to control the Puppet Daemon The performance of a Puppet Master is related to the max concurrent Puppet nodes that are applying a catalog MCollective can determine how many servers are currently enabled and applying catalogs
  • mco puppet $ mco help puppet [14/180] ! Schedule runs, enable, disable and interrogate the Puppet Agent ! Usage: mco puppet [OPTIONS] [FILTERS] [CONCURRENCY|MESSAGE] Usage: mco puppet Usage: mco puppet disable [message] Usage: mco puppet runonce [PUPPET OPTIONS] Usage: mco puppet resource type name property1=value property2=value Usage: mco puppet runall [--rerun SECONDS] [PUPPET OPTIONS] ! The ACTION can be one of the following: ! count - return a total count of running, enabled, and disabled nodes enable - enable the Puppet Agent if it was previously disabled disable - disable the Puppet Agent preventing catalog from being applied resource - manage individual resources using the Puppet Type (RAL) system runall - invoke a puppet run on matching nodes, making sure to only run CONCURRENCY nodes at a time runonce - invoke a Puppet run on matching nodes status - shows a short summary about each Puppet Agent status summary - shows resource and run time summaries
  • Controlled Nodes Query and run Puppet from any node that has the client installed: $ mco puppet count Total Puppet nodes: 9 ! Nodes currently enabled: 9 Nodes currently disabled: 0 ! Nodes currently doing puppet runs: 0 Nodes currently stopped: 9 ! Nodes with daemons started: 9 Nodes without daemons started: 0 Daemons started but idling: 9
  • Puppet Status $ mco puppet status ! * [ ============================================================> ] 7 / 7 ! db02.example.com: Currently idling; last completed run 4 minutes 50 seconds ago fpvcs01.example.com: Currently idling; last completed run 24 minutes 44 seconds ago puppet.example.com: Currently idling; last completed run 6 minutes 48 seconds ago mco01.example.com: Currently idling; last completed run 24 minutes 35 seconds ago fp01.example.com: Currently idling; last completed run 4 minutes 55 seconds ago web01.example.net: Currently idling; last completed run 24 minutes 16 seconds ago mail02.example.com: Currently idling; last completed run 3 minutes 17 seconds ago ! Summary of Applying: false = 7 Summary of Daemon Running: running = 7 Summary of Enabled: enabled = 7 Summary of Idling: true = 7 Summary of Status: idling = 7 ! Finished processing 7 / 7 hosts in 96.09 ms
  • Running Puppet Basic Puppet run: $ mco puppet runonce ! A noop run: $ mco puppet runonce noop ! A run when noop is set: $ mco puppet runonce no-noop ! A splay run: $ mco puppet runonce splay splaylimit 120
  • Filters Host Filters -W, --with FILTER Combined classes and facts filter -S, --select FILTER Compound filter combining facts and classes -F, --wf, --with-fact fact=val Match hosts with a certain fact -C, --wc, --with-class CLASS Match hosts with a certain config management class -A, --wa, --with-agent AGENT Match hosts with a certain agent -I, --wi, --with-identity IDENT Match hosts with a certain configured identity $ mco puppet runonce web01.example.com * [ ============================================================> ] 9 / 9
  • Don't forget lters.
  • Invoking Actions with lters Facts $ mco nd -F osfamily=RedHat $ mco nd with-fact operatingsystemmajrelease=5 Classes Puppet provides a list of classes applied to a node: $ mco nd with-class /apache/
  • Agent Runs $ mco puppet runall 1 2013-01-16 16:14:26: Running all nodes with a concurrency of 1 2013-01-16 16:14:26: Discovering enabled Puppet nodes to manage 2013-01-16 16:14:29: Found 2 enabled nodes 2013-01-16 16:14:32: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:37: dev1.example.net schedule status: Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false' command 2013-01-16 16:14:38: 1 out of 2 hosts left to run in this iteration 2013-01-16 16:14:40: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:44: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:48: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:52: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:56: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:00: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:04: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:08: Currently 1 node applying the catalog; waiting for less than 1
  • Last run summary
  • Plotting Performance
  • MCollective Puppet agent can make changes based on the Puppet RAL Puppet Resources
  • Allowing resource control
  • Enable and Disable Puppet $ mco puppet disable message=turn off all the things $ mco puppet enable
  • Misc Plugin Examples $ mco process list java -I dradis.example.com ! * [ ============================================================> ] 1 / 1 ! dradis.example.com ! PID USER VSZ COMMAND 1380 logstash 1.794 GB /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -X 1436 elasticsea 2.298 GB /usr/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=t ! ! Summary of The Process List: ! Matched hosts: 1 Matched Processes: 2 Resident Size: 146.453 MB Virtual Size: 4.092 GB ! ! Finished processing 1 / 1 hosts in 113.93 ms
  • $ mco package bash status -I db02.example.com ! * [ ============================================================> ] 1 / 1 ! db02.example.com: bash-4.1.2-15.el6_4.x86_64 ! Summary of Arch: ! x86_64 = 1 ! Summary of Ensure: ! 4.1.2-15.el6_4 = 1 ! ! Finished processing 1 / 1 hosts in 128.67 ms