loops and unicorns - the future of the puppet language - puppetconf 2013

puppetconf.com #puppetconf

Loops and Unicorns Future of the Puppet Language Henrik Lindberg Consulting Engineer | Puppet Labs @hel


•  New parser •  New Language Features •  New approach – Opt in to New/Experimental Features – Maintaining Backwards Compatibility

Introduction

New Parser The future is already here…


•  Rewritten from the ground up •  Removes quirky grammar constraints •  Improves error messages •  Enabler for: – opt-in to new and experimental features – backwards compatibility

•  Use on command line, or in settings

New Parser

puppet apply –-parser future ...


De-Quirk

"This is a random number: ${fqdn_rand()}"

join([1,2,3])

$a = 0x0EH $b = 0778

Interpolation and functions work: Literal Array / Hash as argument in calls works: Numbers must now be valid:


Array & Hash

$a = [1,2,3] $b = [4,5,6] $c = $a + $b

Concatenate Arrays: Append to Array: Merge Hash:

$d = $a << 4

$a = {name => 'mary'} $b = {has => 'a little lamb'} $c = $a + $b


Misc

unless $something { . . .} else { . . . }

Unless Else: Assignment is an expression: Expression separator ';' :

$a = $b = 10 fqdn_rand($seed = 30)

$a = $x[1][1] $a = $x[1];[1]


Error Reporting

Error: Could not parse for environment production: Syntax error at 'node' at line 1 on node kermit.example.com

puppet apply -e '$a = node "a+b" { }' Then: Now:

Error: Invalid use of expression. A Node Definition does not produce a value at line 1:6 Error: The hostname 'a+b' contains illegal characters (only letters, digits, '_', '-', and '.' are allowed) at line 1:11 Error: Classes, definitions, and nodes may only appear at toplevel or inside other classes at line 1:6 Error: Could not parse for environment production: Found 3 errors. Giving up on node kermit.example.com

New Approach


•  Forgiving Grammar •  Validation is separate – Can validate a particular language version – Language version != Puppet version

•  Evaluation is separate – Can evaluate a particular language version way – Language version != Puppet version

Separation of Language Concerns

New Language Features


•  Iteration & Lambdas •  Puppet Bindings •  Heredoc support •  Puppet Templates

New Language Features

Iteration & Lambdas


Concepts

each($foo) |$x| { notice $x }

The lambda: Inline call:

$foo.each |$x| { notice $x } each($foo) |$x| { notice $x }


•  each •  select – reject •  collect •  reduce •  slice

Custom functions can take a lambda!

Iterating Functions


Each - Array

$a = ['a', 'b', 'c'] $a.each |$value| { notice $value }

Iterating over an array: Produces:

Notice: Scope(Class[main]): a Notice: Scope(Class[main]): b Notice: Scope(Class[main]): c


Each – Array (with index)

$a = ['a', 'b', 'c'] $a.each |$index, $value| { notice "$index = $value" }

Iterating over an array – with index: Produces:

Notice: Scope(Class[main]): 0 = a Notice: Scope(Class[main]): 1 = b Notice: Scope(Class[main]): 2 = c


Each - Hash

$a = {a => 10, b => 20, c => 30} $a.each |$key, $value| { notice "$key = $value" }

Iterating over a hash – with key and value: Produces:

Notice: Scope(Class[main]): a = 10 Notice: Scope(Class[main]): b = 20 Notice: Scope(Class[main]): c = 30


Each – Hash (elements)

$a = {a => 10, b => 20, c => 30} $a.each |$elem| { notice "${elem[0]} = ${elem[1]}" }

Iterating over a hash elements: Produces:

Notice: Scope(Class[main]): a = 10 Notice: Scope(Class[main]): b = 20 Notice: Scope(Class[main]): c = 30


Select / Reject

$a = [1, 2, 3] notice $a.select |$value| { $v == 2 } notice $a.reject |$value| { $v == 2 }

Select and Reject elements: Produces:

Notice: Scope(Class[main]): 2 Notice: Scope(Class[main]): 1 3


Collect

$a = [1, 2, 3] notice $a.collect |$value| { $v * 10 }

Transform each element with collect: Produces:

Notice: Scope(Class[main]): 10 20 30


Reduce

$a = [1, 2, 3] notice $a.reduce |$memo, $value| { $memo + $value }

Reduce all elements into one: Produces:

Notice: Scope(Class[main]): 6


Examples

$usernames.each |$x| { file { "/home/$x/.somerc": owner => $x } }

Set ownership of some "rc-file" for each user: Setting 'owner' and 'mode' from a Hash:

$users_with_mode = ['fred' => 0666, 'mary' => 0777 ] $users_with_mode.each |$user, $mode| { file {"/home/$user/.somerc": owner => $user, mode => $mode } }


Examples

$a.select |$x| { $x =~ /com$/ }.each |$x| { file { "/somewhere/$x": owner => $x } }

Filter and create resources: Include classes based on array of roles:

$roles.each |$x| { include "role_$x" }


Custom Function (Ruby)

pp_block = args[-1]

Lambda always last argument: Was it given? Call it:

pp_block.is_a? Puppet::Parser::AST::Lambda

# in a custom function, self is scope) pp_block.call(self, 'hello', 'world')


•  http://links.puppetlabs.com/arm2-iteration •  http://links.puppetlabs.com/arm2-examples

ARM 2 - Iteration

Heredoc

Not yet on master branch https://github.com/puppetlabs/puppet/pull/1659


Heredoc - Syntax

@( ["]<endtag>["] [:<syntax>] [/<escapes>] ) <text> [|][-] <endtag>

ENDS-‐HERE anything not in <text> "ENDS-‐HERE" with interpola2on

:json syntax check result

/tsrn$L turns on escape / turns on all

| set le7 margin

-‐ trim trailing

t tab s space r return n new-‐line $ $ L <end of line>


Heredoc – example Example:

#.........1.........2.........3.........4.........5.... $a = @(END) This is indented 2 spaces in the source, but produces a result flush left with the initial 'T' This line is thus indented 2 spaces. | END


Heredoc – example multiple on same line:

#.........1.........2.........3.........4.........5.... foo(@(FIRST), @(SECOND)) This is the text for the first heredoc FIRST This is the text for the second SECOND


For more examples and details •  http://links.puppetlabs.com/arm4-heredoc

ARM-4 Heredoc

Puppet Templates

Not yet on master branch https://github.com/puppetlabs/puppet/pull/1660


•  Embedded Puppet (EPP) - like ERB •  Same tags

§  <%, <%=, <%-, <%%, <%# § %>, -%>

•  Expressions are Puppet DSL •  Parameterized •  .epp file extension (by convention)

Puppet Templates


Use by calling epptemplate(<name> [,<params_hash>]) inline_epptemplate(<text>[,<params_hash>])

Puppet Templates

$x = 'human' inline_epptemplate('This is not the <%= $x %> you are looking for.', { 'x' => 'droid'}) # => 'This is not the droid you are looking for.'


•  Parameterized – declare parameters – set default values – parameter without value and no default = error

Puppet Templates

<%- ($x = 'human') -%> This is not the <%= $x %> you are looking for.


For more examples and details •  http://links.puppetlabs.com/arm3-

puppet_templates

ARM-3 Puppet Templates

Puppet Bindings / Data in Modules


•  More powerful data bindings •  For both Data, and Puppet Extensions •  Composes Hiera2 data in modules and

environment + Hiera1 •  Bindings in Ruby •  Opt in on command line or in settings

Puppet Binder

puppet apply --binder puppet apply –-parser future ...


•  Default: – All hiera-2 data (in default location) and all

(default) ruby bindings from all modules on module path composed

– Site level hiera-2 data and ruby bindings override contributions from modules.

•  Customize –  include alternatives, exclude bindings – add / reorganize overriding "layers"

Configuration


•  Composable •  Interpolation using Puppet DSL expressions •  Changed hiera.yaml syntax (for version 2)

Hiera 2


Minimal opt-in (in a module)

--- version: 2

hiera.yaml: data/common.yaml: data/${osfamily}.yaml:

--- myclass::myparam: '1.2.3'

--- myclass::myparam: '2.4.6'


For more examples and details •  http://links.puppetlabs.com/arm8-

puppet_bindings •  http://links.puppetlabs.com/arm9-

data_in_modules

ARM-8 & 9

Thank You Henrik Lindberg Consulting Engineer | Puppet Labs @hel

Collaborate. Automate. Ship.

Follow us on Twitter @puppetlabs youtube.com/puppetlabsinc slideshare.net/puppetlabs

Collaborate. Automate. Ship.

loops and unicorns - the future of the puppet language - puppetconf 2013

Technology

b notice

puppetconf examples

puppetconf array hash

puppetconf http

puppetconf concepts

puppetconf misc

settings new parser

hash elements