![Page 1: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/1.jpg)
...
AugeasAugeasSwiss-knife resources for your puppet treeSwiss-knife resources for your puppet tree
Julien Pivotto
Belgian Puppet User GroupHoliday is over Meetup!!! - November 12th, 2014
![Page 2: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/2.jpg)
..
whoamiwhoamiJulien PivottoJulien Pivotto
• Open-Source consultant at inuits.eu• FOSS defender since 2004• DevOps believer and evangelist• Puppet User since 2011• @roidelapluie on twitter/github
![Page 3: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/3.jpg)
..
..
ınuits.eu
![Page 4: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/4.jpg)
..
..
Sysadmin 101Sysadmin 101CC BY-SA 2.0 https://www.flickr.com/photos/arthur-caranta/2926332140
![Page 5: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/5.jpg)
..
Setting up a serviceSetting up a service
• Install the package• Change the configuration• Start the daemon
![Page 6: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/6.jpg)
..
3 steps.What can go wrong?
![Page 7: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/7.jpg)
..
PackagingPackaging
• Where is the package?• Which version do we need?• Does it conflict with something else?
![Page 8: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/8.jpg)
..
..
Dependencies HellDependencies Hell
CC BY-SA 2.0 https://www.flickr.com/photos/coconinonationalforest/4587053982
![Page 9: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/9.jpg)
..
ConfigurationConfiguration
• Where is the file?• How many files?• Configuration is in the database?• The file is *huge*
![Page 10: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/10.jpg)
..
Starting the serviceStarting the service
• Does not start▶ Bad config file▶ Stale lock file▶ Data corruption
• High Availability• Replication
![Page 11: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/11.jpg)
..
Let's talk about Puppet and filesLet's talk about Puppet and files
• Classical approach: File[] resource• Advanced approach: Concat[] define• Broken approach: Exec[sed] resource• Surgical approach: Augeas[] resource
![Page 12: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/12.jpg)
..
Hidden ways to manage filesHidden ways to manage files
• Ssh_authorized_key[]• Nagios_*• To purge or not to purge
![Page 13: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/13.jpg)
..
..
The File[] resourceThe File[] resourceCC BY 2.0 https://www.flickr.com/photos/80497449@N04/10567875696/
![Page 14: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/14.jpg)
..
FileFile
• Built-in puppet resource• Most used• Works with a lot of usecases• Text files, binary files
![Page 15: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/15.jpg)
..
.
.
file{"${::icinga::confdir_server}/cgi.cfg":ensure => present,content => template('icinga/redhat/cgi.cfg.erb'),owner => $::icinga::server_user,group => $::icinga::server_group,require => Class['icinga::config'],notify => [
Service[$::icinga::service_client],Service[$::icinga::service_server],Exec['fix_collected_permissions']
],}
![Page 16: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/16.jpg)
..
Content of a fileContent of a file
• content => String, template(), file()• source => puppet:///, /local/file
![Page 17: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/17.jpg)
..
File[] behaviourFile[] behaviour
• Array as "source": Puppet will pick the firstavailable one
• Multiple arguments to template(): Puppetwill concatenate them all
• File[/foo/bar] will autorequire File[/foo]
![Page 18: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/18.jpg)
..
Downside of File[]Downside of File[]
• You can only have at one "content"• That resource describe the whole file• Works in almost every situation
![Page 19: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/19.jpg)
..
..
concatPublic Domain http://commons.wikimedia.org/wiki/File:Adhesive_tapes_clear.JPG
![Page 20: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/20.jpg)
..
ConcatConcat
• A "reference" puppet module:puppetlabs/concat
• https://github.com/puppetlabs/puppetlabs-concat
• Provides definitions to manage file• Alternative modules:
▶ onyxpoint/pupmod-concat▶ theforeman/puppet-concat (fork of onyxpoint)
![Page 21: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/21.jpg)
..
Concat?Concat?
• Concat takes a bunch of snippets• Assemble them info a file• Each snippet is a define• The final file is a define
![Page 22: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/22.jpg)
..
.
.
concat { '/tmp/file':ensure => present,
}
concat::fragment { 'tmpfile':target => '/tmp/file',content => 'test contents',order => '01'
}
![Page 23: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/23.jpg)
..
Base and fragmentsBase and fragments
• Concat[] defines owner, ensure, mode of thefile
• Concat::Fragment[] defines the contents ofthe file
• One Concat[] has multipleConcat::Fragment[]
![Page 24: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/24.jpg)
..
Advantages of concatAdvantages of concat
• More flexibility▶ if▶ virtual resource▶ exported resources▶ create_resources
• Mix templates and files
![Page 25: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/25.jpg)
..
Disadvantages of concatDisadvantages of concat
• External Puppet module• Concat[] is the whole file• Performances
![Page 26: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/26.jpg)
..
..
Exec{sed: onlyif => grep}
CC BY-SA 3.0 http://commons.wikimedia.org/wiki/File:Ca%C3%AFn_par_Henri_Vidal.jpg
![Page 27: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/27.jpg)
..
..https://github.com/search?o=desc&q=exec+sed+onlyif+grep+language%3APuppet
![Page 28: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/28.jpg)
..
exec[sed] is br0kenexec[sed] is br0ken
• Which options to pass to sed and grep?• You should use as few Exec[] as possible• grep ....• Escape, regexes…
![Page 29: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/29.jpg)
..
Another alternative: conf.dAnother alternative: conf.d
• Some services support conf.d directories• But it is hard to change existing parameters• In which order are the files read?• Don't forget to purge
![Page 30: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/30.jpg)
..
..
Augeas
CC BY-SA 3.0 http://commons.wikimedia.org/wiki/File:Students_assisting_surgery.JPG
![Page 31: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/31.jpg)
..
AugeasAugeas
• Configuration editing tool• First release in 2007• API coded in C• Command-line tools• bindings for different languages
![Page 32: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/32.jpg)
..
Configuration editing toolConfiguration editing tool
• Parsing the configuration files• Turning them into a tree• Edit the tree & save the configuration
![Page 33: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/33.jpg)
..
.
.
$ cat /etc/nsswitch.conf# /etc/nsswitch.conf## Example configuration#
passwd: db filesgroup: db filesinitgroups: db [SUCCESS=continue] filesshadow: db filesgshadow: files
![Page 34: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/34.jpg)
..
.
.
augtool> ls /files/etc/nsswitch.conf/#comment[1] = /etc/nsswitch.conf#comment[2] = Example configurationdatabase[1]/ = passwddatabase[2]/ = groupdatabase[3]/ = initgroupsdatabase[4]/ = shadowdatabase[5]/ = gshadow
![Page 35: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/35.jpg)
..
.
.
augtool> ls /files/etc/nsswitch.conf/database[1]/service[1] = dbservice[2] = files
![Page 36: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/36.jpg)
..
Native format -> treeNative format -> tree
• Augeas understand comments• Augeas does not care about empty lines• The cli tool (augtool) has autocomplete• It recognize a lot of formats
![Page 37: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/37.jpg)
..
.
.
augtool> set /files/etc/nsswitch.conf/database[1]/service[last()+1] ldapaugtool> saveSaved 1 file(s)
![Page 38: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/38.jpg)
..
.
.
$ cat /etc/nsswitch.conf# /etc/nsswitch.conf## Example configuration#
passwd: db files ldapgroup: db filesinitgroups: db [SUCCESS=continue] filesshadow: db filesgshadow: files
![Page 39: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/39.jpg)
..
.
.
augtool> match /files/etc/nsswitch.conf/*/* ldap/files/etc/nsswitch.conf/database[1]/service[3]augtool> print /files/etc/nsswitch.conf/database[1]/files/etc/nsswitch.conf/database[1] = "passwd"/files/etc/nsswitch.conf/database[1]/service[1] = "db"/files/etc/nsswitch.conf/database[1]/service[2] = "files"/files/etc/nsswitch.conf/database[1]/service[3] = "ldap"
![Page 40: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/40.jpg)
..
.
.
augtool> rm /files/etc/nsswitch.conf/database[1]/service[3]rm : /files/etc/nsswitch.conf/database[1]/service[3] 1augtool> print /files/etc/nsswitch.conf/database[1]/files/etc/nsswitch.conf/database[1] = "passwd"/files/etc/nsswitch.conf/database[1]/service[1] = "db"/files/etc/nsswitch.conf/database[1]/service[2] = "files"augtool> saveSaved 1 file(s)
![Page 41: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/41.jpg)
..
One API to edit them allOne API to edit them all
• Can talk XML, ini, named, nginx, …• Only change what is needed• Ensure the syntax is right
![Page 42: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/42.jpg)
..
Augeas LensesAugeas Lenses
• Lenses are files that explain how to edit files• It contains paths and syntax• There are a lot of them available• You can write your own lenses
![Page 43: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/43.jpg)
..
”This brings the total number of lenses to178. […] It’s depressing to think that
Linux/Unix systems have managed to growthis many special snowflake formats.”
David Lutterkort, main developerabout Augeas 1.3.0
![Page 44: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/44.jpg)
..
178 lenses178 lensesactivemq_conf activemq_xml aliases aptconf
apt_update_manager backuppchosts bbhosts bootconf buildcarbon cgrules channels cobblermodules cobblersettings collectd
crypttab cyrus_imapd darkice debctrl desktop device_map dhcpddnsmasq dovecot dpkg dput ethers exports fai_diskconfig fonts
fuse gdm grub gshadow hostname inetd inputrc interfaces iproute2iptables jaas jmxaccess keepalived known_hosts koji krb5 ldif limits
login_defs logrotate mcollective memcached mke2fsmongodbserver mysql nagioscfg nagiosobjects netmasks nginx ntpntpd odbc openshift_config openshift_http openvpn pam passwd
pbuilder postfix_main postfix_transport postfix_virtualpuppet_auth qpid rabbitmq resolv rmt securetty sep services shells
shellvars_list sip_conf slapd smbusers squid sshd stunnelsubversion sudoers sysconfig systemd thttpd up2date vfstab
![Page 45: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/45.jpg)
..
A short lenseA short lense
.
.
module Hostname =autoload xfm
(* View: lns *)let lns = [ label "hostname" . store Rx.word . Util.eol ]
(* View: filter *)let filter = incl "/etc/hostname". incl "/etc/mailname"
let xfm = transform lns filter
![Page 46: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/46.jpg)
..
Puppet <3 augeasPuppet <3 augeas
• Native "augeas" resource• Support for pluginsync• Helpers available
![Page 47: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/47.jpg)
..
Puppet examplePuppet example
.
.
augeas { $name:context => "/files${fstab::variables::fstab_file}",changes => [
"rm ${fstab_match_line}",],onlyif => "match ${fstab_match_line} size > 0"
}
![Page 48: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/48.jpg)
..
Real usecasesReal usecases
• Change grub options• Modify /etc/hosts• Modify XML's (puppetlabs-tomcat)• Configure Jenkins
![Page 49: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/49.jpg)
..
PluginsyncPluginsync
• Puppet has pluginsync support for Augeas• Drop your lenses in your modules• lib/augeas/lenses• Use the "lens" parameter of the augeasresource
![Page 50: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/50.jpg)
..
Puppet examplePuppet example
.
.
augeas{"jboss_conf":context => "/files/etc/jbossas",changes => [
"set jbossas.conf/JBOSS_IP $ipaddress","set jbossas.conf/JAVA_HOME /usr",
],lens => "Jboss.aug",
}
![Page 51: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/51.jpg)
..
Augeas commandsAugeas commands
set rm mv clear insert …
![Page 52: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/52.jpg)
..
Augeas comparators (onlyif)Augeas comparators (onlyif)
match get
![Page 53: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/53.jpg)
..
AugeasprovidersAugeasproviders
• Helpers around augeas• Puppet modules• No augeas knowledge needed
![Page 54: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/54.jpg)
..
apacheapache
.
.
apache_setenv { "SPECIAL_PATH":ensure => present,value => "/foo/bin",
}
![Page 55: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/55.jpg)
..
kernel_parameterkernel_parameter
.
.
kernel_parameter { "quiet":ensure => present,bootmode => "normal",
}
![Page 56: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/56.jpg)
..
Conclusion
![Page 57: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/57.jpg)
..
DisadvantagesDisadvantages
• Learning required• Library to install• Writing lenses is hard
![Page 58: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/58.jpg)
..
AdvantagesAdvantages
• Augeas is a mature tool• Preserves comments in files• It fails (if needed)• Only changes what is needed• A lot of lenses available• Puppet integration• Helpers available
![Page 59: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/59.jpg)
..
Final noteFinal note
Most of the time, File[] resources are the wayto go. Augeas can help when you need tochange files generated by an application orthat you can not manage entirely.
![Page 60: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/60.jpg)
..
ReadingsReadings
• http://augeas.net/• http://augeasproviders.com/• https://docs.puppetlabs.com/
![Page 61: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/61.jpg)
..
Thank youThank you
Any question?Thanks to @raphink
![Page 62: Augeas, swiss knife resources for your puppet tree](https://reader033.vdocuments.mx/reader033/viewer/2022051112/55a37ec11a28abfb158b47d5/html5/thumbnails/62.jpg)
..
ContactContact
Julien [email protected]@roidelapluie
INUITS bvbaBelgium+32 473 441 636https://inuits.eu