PUPPET EVOLUTIONSAlessandro Franceschi

@alvagante

Italian DevOps Day 2016

AL VAGANTE

• Founder and CTO at example42 GmbH

• Puppet user / trainer since 2007

• Author of the book "Extending Puppet"

• Author of ## Puppet modules, Puppi, Tiny Puppet

• github.com/alvagante

AGENDA

• Evolution of configuration management tools

• Puppet 4 and the evolution of the language

CFGMGMT TOOLS EVOLVE

• System resources (packages, services, files, users... )

• Applications (app specific resources)

What did we manage yesterday?

CFGMGMT TODAY

• Network devices

• Storage

• Cloud resources

• Containers

• Systems and applications, still

Managing whole IT infrastructures

MANAGING NETWORKS

• Proxy mode via puppet device(Puppet code executed on a remote node)

• Native client (Cisco, Huawei, Arista, Cumulus networks)

(Puppet code executed on the network device)

Alternative management modes

NETDEV STDLIB

• Puppet types for network resources

• Providers are in separated, vendor specific, modules

Abstracting network resources

domain_name name_server network_dns network_interface network_snmp network_trunk network_vlan ntp_config ntp_server port_channel radius radius_global radius_server radius_server_group search_domain snmp_community snmp_notification snmp_notification_receiver snmp_user syslog_server syslog_settings tacacs tacacs_global tacacs_server tacacs_server_group

Common types

puppet module install puppetlabs-netapp

Sample codenode 'ontap01.example.com' { netapp_aggregate { 'aggr1': ensure => present, diskcount => '6', } netapp_vserver { 'vserver01': ensure => present, rootvol => 'vserver01_root', rootvolaggr => 'aggr1', rootvolsecstyle => 'unix', } netapp_lif { 'vserver01_lif': ensure => present, homeport => 'e0c', homenode => 'ontap01', address => '10.0.207.5', vserver => 'vserver01', netmask => '255.255.255.0', dataprotocols => ['nfs'], } }

node 'vserver01.example.com' { netapp_export_policy { 'nfs_exports': ensure => present, } netapp_export_rule { 'nfs_exports:1': ensure => present, clientmatch => '10.0.0.0/8', protocol => ['nfs'], superusersecurity => 'none', rorule => ['sys','none'], rwrule => ['sys','none'], } netapp_volume { 'vserver01_root': exportpolicy => 'nfs_exports', } netapp_volume { 'nfsvol': ensure => present, aggregate => 'aggr1', initsize => '200g', exportpolicy => 'nfs_exports', junctionpath => '/nfsvol', } netapp_qtree { 'qtree1': ensure => present, volume => 'nfsvol', } netapp_nfs { 'vserver01': ensure => present, state => 'on', v3 => 'disabled', v40 => 'enabled',

MANAGING CLOUDS

• Public cloud resources management (AWS, Azure, Google Cloud, Digital Ocean...)

• Private cloud infrastructure management (VMWare, OpenStack)

Several cloud related modules

puppet module install puppetlabs-aws

ec2_instance { 'instance-name': ensure => present, region => 'us-west-1', image_id => 'ami-123456', instance_type => 't1.micro', }

ec2_securitygroup { 'name-of-group': ensure => present, region => 'us-east-1', description => 'a description of the group', ingress => [{ protocol => 'tcp', port => 80, cidr => '0.0.0.0/0', },{ security_group => 'other-security-group', }],}

cloudwatch_alarm ec2_autoscalinggroup ec2_elastic_ip ec2_instance ec2_launchconfiguration ec2_scalingpolicy ec2_securitygroup ec2_vpc ec2_vpc_customer_gateway ec2_vpc_dhcp_options ec2_vpc_internet_gateway ec2_vpc_routetable ec2_vpc_subnet ec2_vpc_vpn ec2_vpc_vpn_gateway elb_loadbalancer rds_db_parameter_group rds_db_securitygroup rds_instance route53_a_record route53_aaaa_record route53_cname_record route53_mx_record route53_ns_record route53_ptr_record route53_spf_record route53_srv_record route53_txt_record route53_zone sqs_queue

Sample code

Supported resources

puppet module install puppetlabs-azure

Sample codeazure_vm { 'sample': ensure => present, location => 'eastus', image => 'canonical:ubuntuserver:14.04.2-LTS:latest', user => 'azureuser', password => 'Password', size => 'Standard_A0', resource_group => 'testresacc01', }

MANAGING CONTAINERS

• Modules to install and configure Docker components (Engine, Swarm, Registry...), images and containers

• Modules to manage Docker ecosystem software (Kubernetes, Mesos...)

Managing containers infrastructures

puppet module install puppetlabs-docker_platform

Sample codeclass {'docker': tcp_bind => 'tcp://127.0.0.1:4243', socket_bind => 'unix:///var/run/docker.sock', version => '0.5.5', dns => '8.8.8.8', docker_users => [ 'user1', 'user2' ], }

docker::image { 'ubuntu': ensure => 'present', image_tag => 'precise', docker_file => '/tmp/Dockerfile', }

docker::run { 'helloworld': image => 'base', command => '/bin/sh -c "while true; do echo hello world; sleep 1; done"', }

docker::registry { 'example.docker.io:5000': username => 'user', password => 'secret', email => 'user@example.com', }

puppet module install puppetlabs-garethr-kubernetes

Sample codekubernetes_pod { 'sample-pod': ensure => present, metadata => { namespace => 'default', }, spec => { containers => [{ name => 'container-name', image => 'nginx', }] }, }

kubernetes_aws_elastic_block_store_volume_source kubernetes_binding kubernetes_capabilities kubernetes_capability kubernetes_ceph_fs_volume_source kubernetes_cinder_volume_source kubernetes_component_condition kubernetes_component_status kubernetes_component_status_list kubernetes_container kubernetes_container_port kubernetes_container_state kubernetes_container_state_running kubernetes_container_state_terminated kubernetes_container_state_waiting kubernetes_container_status kubernetes_daemon_endpoint kubernetes_delete_options kubernetes_downward_api_volume_file kubernetes_downward_api_volume_source kubernetes_empty_dir_volume_source kubernetes_endpoint_address kubernetes_endpoint_port kubernetes_endpoint_subset kubernetes_endpoints kubernetes_endpoints_list kubernetes_env_var kubernetes_env_var_source kubernetes_event kubernetes_event_list kubernetes_event_source kubernetes_exec_action kubernetes_fc_volume_source kubernetes_finalizer_name kubernetes_flocker_volume_source kubernetes_gce_persistent_disk_volume_source kubernetes_git_repo_volume_source kubernetes_glusterfs_volume_source kubernetes_handler kubernetes_host_path_volume_source kubernetes_http_get_action kubernetes_iscsi_volume_source kubernetes_lifecycle kubernetes_limit_range kubernetes_limit_range_item kubernetes_limit_range_list kubernetes_limit_range_spec kubernetes_list_meta kubernetes_load_balancer_ingress kubernetes_load_balancer_status kubernetes_local_object_reference kubernetes_namespace kubernetes_namespace_list kubernetes_namespace_spec kubernetes_namespace_status kubernetes_nfs_volume_source kubernetes_node kubernetes_node_address kubernetes_node_condition kubernetes_node_daemon_endpoints kubernetes_node_list kubernetes_node_spec kubernetes_node_status kubernetes_node_system_info kubernetes_object_field_selector kubernetes_object_meta kubernetes_object_reference kubernetes_patch kubernetes_persistent_volume kubernetes_persistent_volume_access_mode kubernetes_persistent_volume_claim kubernetes_persistent_volume_claim_list kubernetes_persistent_volume_claim_spec kubernetes_persistent_volume_claim_status kubernetes_persistent_volume_claim_volume_source kubernetes_persistent_volume_list kubernetes_persistent_volume_spec kubernetes_persistent_volume_status kubernetes_pod kubernetes_pod_condition kubernetes_pod_list kubernetes_pod_security_context kubernetes_pod_spec kubernetes_pod_status kubernetes_pod_template kubernetes_pod_template_list kubernetes_pod_template_spec kubernetes_probe kubernetes_rbd_volume_source kubernetes_replication_controller kubernetes_replication_controller_list kubernetes_replication_controller_spec kubernetes_replication_controller_status kubernetes_resource_quota kubernetes_resource_quota_list kubernetes_resource_quota_spec kubernetes_resource_quota_status kubernetes_resource_requirements kubernetes_se_linux_options kubernetes_secret kubernetes_secret_list kubernetes_secret_volume_source kubernetes_security_context kubernetes_service kubernetes_service_account kubernetes_service_account_list kubernetes_service_list kubernetes_service_port kubernetes_service_spec kubernetes_service_status kubernetes_status kubernetes_status_cause kubernetes_status_details kubernetes_tcp_socket_action kubernetes_volume kubernetes_volume_mount kubernetes_watch_event

Supported resources (Puppet types automatically generated from APIs)

Manages kubernetes resources but not its installation

PUPPET 4 (R)EVOLUTION

• All In One packages

• Puppet Server , Puppet Agent , Facter 3

• New Type System

• Language improvements

• Deprecations

Better language, new foundations

ALL IN ONE PACKAGES

• Dedicated repositories (Software Collections)

• File paths changed (a lot!)

• Client: puppet-agentInstalls Puppet, Hiera, Mcollective and the full Ruby stack

• Server : puppetserverClojure application, running inside a JVM (JRuby for ruby code)

Full Stack packaging

FACTER EVOLUTIONS

• Structured facts (can be array, hashes, booleans...)(Since Facter 2)

• Aggregated resolution(Facts values can be resolved in multiple steps. Since Facter 2)

• CFacter (or native facter : way faster, written in C)(Facter 3. Enabled by default from Puppet 4.2)

Facter reloaded

TYPE SYSTEM

• A new powerful type system

• Can be used for parameters evaluation

class myclass (

Integer[22-1024] $port,

Variant[Boolean,String] $ensure = present,

Hash $options_hash = { },

Boolean $test_enable = false,

Variant[Undef,String] $test_template = undef,

String[1] $debug_dir = '/tmp',

Everything has a type

ITERATIONS

• Iterations over resources are based on lambdas

• Dedicated functions: each, slice, filter, map, reduce

We can iterate inside our code, finally

# Iteration over arrays $pkg = [ php,php-mysql,apa]each($pkg) | $value | { package { $value: ensure => present, }}

# Iteration over hashes$h = { '1' => ['a','b','c'], '2' => 'foo' }each($h) | $key, $value | { notice "$key = $value"}

EPP TEMPLATES

• Alternative to Ruby Erb templates (still valid)• Epp function accepts an hash of parameters• Parameters have to be specified in the header (and can be

validated via the Type system)

Templates using Puppet DSL

Declaration of parameters is part of the epp function and specified within a hash:

content = epp('<path to template>', { 'header' => 'Copyright example42', 'informations' => ['Puppet 4', 'EPP'] }),

Parameters need to be specified as a header:

<%- | String $header = '', Array $informations = [] | -%> <%= $header %> List of informations: <% $informations.each | String $info | { -%> - <%= $info %> <% } -%>

PUPPET DSL FUNCTIONS

• Ruby no more needed for Puppet functions

• Autoloaded from a module's function directory cat tp/functions/is_empty.ppfunction tp::is_empty ( Any $input,) {

$output = $input ? { false => true, '' => true, undef => true, default => false, }

}

Functions written in Puppet DSL

cat tp/functions/ensure2bool.pp

function tp::ensure2bool ( Variant[Boolean,String] $input = present, $default = undef,) {

$output = $input ? { 'absent' => false, false => false, 'present' => true, true => true, default => $default, }

}

A LOT MORE...

• Directory environments

• Heredoc support

• New functions API (for ruby functions)

• A lot of code cleanups (and deprecations)

Other notable changes in Puppet 4

AND MORE TO COME

• Puppet lookup function (Data in Modules)

• Puppet Communication / Execution protocol

• Application Orchestration

Experimental or more recent additions

DEPRECATIONS

• Goodbye nodes inheritance• Empty strings evaluate are true in P4• Goodbye classes and defines in ruby• Variables can't start with capital letters or underscore• Reference deprecations: Class ['Ssh'] must be Class['ssh'] • No more hyphens in classes, defines and modules names• import statement• Variables dynamic scoping in erb templates

Stuff no more supported or changed in Puppet 4

FUTURE DEPRECATIONS

• Webrick and rack Puppet Master (only Puppet Server)• Support for Windows 2003• Non-Strict variables (error when accessing undefined

variables)• Automatic symlinks in file resources for ensure => /path• APIs: Puppet.newtype (Use Puppet::Type.newtype instead)• Puppet faces becomes a private API (custom faces in modules

won't be supported)• Config settings: cfacter, configtimeout, ignorecache

What won't work in Puppet 5

– Louis D. Brandeis

“There are no shortcuts in evolution”

Prezzo Early Bird (iscrizioni entro 06/05): 1.790,00 € + IVA Prezzo di Listino: 1.990,00 € + IVA

Organizers

TrainerAlessandro Franceschi