an introduction to devops with chef

Rethinking IT: An Introduction to DevOps with ChefJulian C. DunnSenior Consultant, Opscode<[email protected]>

Wednesday, October 2, 13

mailto:[email protected]


Topics• Rethinking IT with DevOps• What Chef is and how it helps• Flavors of Chef• Use cases and demo


I came to hear about Chef.


Why are you throwing DevOps into this?


It’s how Chef works best• Chef is a tool.• It works best when developers and operations

work well together• You don’t need “DevOps” for Chef to be successful.• But it helps.


What is this DevOps? Sounds like a buzzword.


DevOps: Aligning Objectives• Cultural and Professional

movement• Development and

Operations working together

• Leveraging ideas & processes from other industries

• To enable the businessWednesday, October 2, 13

Technology: Business Engaging Customers


Speed of Globalization• 40 years for container

ships to move 70% of seaborne trade

• 22 years for internet access to reach 78% penetration in North America


Globalization• Online retail sales are 7%

of all retail sales• 75% of 2011 Thanksgiving

shoppers did so online• 42% of all retail

purchases were influenced by online research – accounting for ~50% of total retail spending.

WTO Trends in Globalization http://www.wto.org/english/res_e/booksp_e/anrep_e/wtr08-2b_e.pdf

http://www.flickr.com/photos/duke_raoul/2261478794/sizes/l/in/photostream/


http://www.wto.org/english/res_e/booksp_e/anrep_e/wtr08-2b_e.pdf






Mobile Devices• 42% are smartphones• 58% will be on next

purchase• 4.2 Billion phones

globally for 7.09 Billion people (USCB)

http://ssiknowledgewatch.com/2012/05/09/cell-phones-approach-total-penetration-globally-with-smartphones-moving-toward-market-dominance-2/

http://www.brightsideofnews.com/news/2011/1/26/digital-divide-global-household-penetration-rates-for-technology.aspx?pageid=1






Software: The Interface for Consumtpion


Drivers of IT Innovation

The Result: The Coded Business

How: Redefinition of how to use technology to create business value

Why: To rapidly deliver experiences, goods and services to customers

What: Consumer-facing businesses


The Rise of the Coded Business• Speed of change > speed of skills development• IT is moving from the back office to the front office• Digital consumption• Technology directly supports customer interactions• Companies must move faster to compete


Patterns of the Coded BusinessBusiness

Agility

Development Velocity and Consistency

Continuous Delivery

IT

IT enables Business Agility and becomes a strategic advantage rather than a cost center.


The Coded Business: Coming to Every Enterprise

Manufacturing Financial Services

Retail

Media and Entertainment

High Technology

Healthcare


Scale x Complexity > Skills


DevOps...• Cultural and professional movement• Grew directly from people who have experienced

this transition• Applies directly to traditional IT• Applies directly to other systems of survival in

our culture


DevOps: CAMS• Culture• Automation• Measurement• Sharing


Open Communication• Developers &

Operations talk and listen to one another

• Production & build metrics are available to all

• Current infrastructure is documented


Incentive & Responsibility Align• Create awesome

customer experiences• Responsibility &

accountability, not authority

• Responsible for your own uptime


Respect• You don’t have to be

everyone’s best friend• But you do need to

recognize contributions and treat each other well

• No asshole rule


Culture: Trust• Trust that everyone is

competent & working toward the common goals

• Without trust, the tools don’t matter


DevOps: CAMS• Culture• Automation• Measurement• Sharing

• Chef is (part of) “A”. Works best with C, M and S.


Infrastructure Automation with Chef


http://www.flickr.com/photos/steffenz/337700069/http://www.flickr.com/photos/kky/704056791/

Applications


http://www.flickr.com/photos/steffenz/337700069/




Infrastructure

http://www.flickr.com/photos/sbh/462754460/




http://www.flickr.com/photos/philliecasablanca/3354734116/

• Networking

• Files

• Directories

• Symlinks

• Mounts

• Routes

• Users

• Groups

• Tasks

• Packages

• Software

• Services

• Configuration

• Other Stuff

Collection of Resources


Acting in Concert

http://www.flickr.com/photos/glowjangles/4081048126/Wednesday, October 2, 13

http://www.flickr.com/photos/glowjangles/4081048126/

http://www.flickr.com/photos/glowjangles/4081048126/

To Provide a Service

http://www.flickr.com/photos/28309157@N08/3743455858/




And it Evolves





Application Server

See Node


Application Server

Application Database

See Nodes


Application Server

Application Databases

See Nodes Grow


Application Servers


See Nodes Grow


Application Servers


Load Balancer

See Nodes Grow


Application Servers


Load Balancers

See Nodes Grow


Application Servers

Application Database Cache

Load Balancers


See Nodes Grow


Application Servers


Load Balancers


Tied Together with Configuration


Application Servers


Load Balancers

Floating IP?


Infrastructure is a Snowflake


Load Balancers

Application Servers

NoSQL

Database Slaves

Application Cache

Database Cache

Database

Evolving Complexity


Complexity Increases Very Quickly

DC1

DC3

DC2


Configuration Management

http://www.flickr.com/photos/philliecasablanca/3354734116/Wednesday, October 2, 13



Golden Images are not the answer

• Gold is heavy

• Hard to transport

• Hard to mold

• Easy to lose configuration detail

http://www.flickr.com/photos/garysoup/2977173063/




JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

Typical Infrastructure


JBoss App

Memcache

Postgres Slaves

Postgres Master

NagiosGraphite

• Move SSH off port 22

• Let’s put it on 2022

New Compliance Mandate!


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

• edit /etc/ssh/sshd_config

1 2

3

4

5

6

6 Golden Image Updates


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

• Delete, launch

1 2

3 4 5 6 7

8 9

10 11

12

• Repeat

• Typically manually

12 Instance Replacements


• Don’t break anything!

• Bob just got fired =(

5

JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite 1 2

4 5 6 7

8 9

10 11

12

3

Done in Maintenance Windows


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

• Invalid configs!

Different IP Addresses?


Configuration Desperation

http://www.flickr.com/photos/francoforeshock/5716969942/Wednesday, October 2, 13

http://www.flickr.com/photos/francoforeshock/5716969942/

http://www.flickr.com/photos/francoforeshock/5716969942/

Chef Solves This Problem• But you already

guessed that, didn’t you?


Chef is Infrastructure as Code• Programmatically

provision and configure

• Treat like any other code base

• Reconstruct business from code repository, data backup, and bare metal resources.http://www.flickr.com/photos/louisb/4555295187/


http://www.flickr.com/photos/louisb/4555295187/

http://www.flickr.com/photos/louisb/4555295187/

Programs• Chef generates

configurations directly on nodes from their run list

• Reduce management complexity through abstraction

• Store the configuration of your programs in version control

http://www.flickr.com/photos/ssoosay/5126146763/




Declarative Interface to Resources• Define Policy• Say what, not how• Pull not Push

http://www.flickr.com/photos/bixentro/2591838509/




That Looks Like Thispackage "apache2"

template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]"end

service "apache2" do action [:enable,:start] supports :reload => trueend


Ohai"languages": { "ruby": {

}, "perl": { "version": "5.14.2", "archname": "x86_64-linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" }},

"network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", “UP","LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "scope": "Node" } }, }, "eth0": { "type": "eth", "number": "0",

"kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux"},"os": "linux","os_version": "3.2.0-32-virtual","ohai_time": 1369328621.3456137,


execute "load sysctl" do command "/sbin/sysctl -p" action :nothingend

bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3,pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048

# adjust shared memory and semaphorestemplate "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediatelyend

Decide what to declare


Recipes and Cookbooks• Recipes are collections of

Resources• Cookbooks contain recipes,

templates, files, custom resources, etc

• Code re-use and modularity

http://www.flickr.com/photos/shutterhacks/4474421855/




Run ListsServerServerServerServer

chef-server

API chef-client “recipe[ntp::client]”

node

ntp

client.rb



chef-server

API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”

node

ntp

client.rb

openssh

server.rb



chef-server

API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”,

“recipe[apache]”,“recipe[php]”

node

ntp

client.rb

openssh

server.rb

apache

default.rb

php

default.rbWednesday, October 2, 13

Roles

name "base"description "base"run_list [ "recipe[selinux::disabled]", "recipe[etchosts]", "recipe[yum::epel]", "recipe[debugtools]"]

name "webserver"description "webserver server"run_list [ "role[base]", "recipe[nginx::server]"]


Roles

Role

RecipeRecipeRecipe

Role

Role

RecipeRecipeRecipeRole

Recipe

ServerServerServerServer

chef-server

API

Knife



chef-server

API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”,

“recipe[apache]”,“recipe[php]”

node

ntp

client.rb

openssh

server.rb

apache

default.rb

php


RolesServerServerServerServer

chef-server

API chef-client

“role[webserver]”

node

ntp

client.rb

openssh

server.rb

apache

default.rb

php


RolesServerServerServerServer

chef-server

API

chef-client

“role[webserver]”

node

ntp

client.rb

openssh

server.rb

apache

default.rb

php

default.rb

chef-client

“role[database]”node

ntp

client.rb

openssh

server.rb

mysql

server.rb


http://www.flickr.com/photos/kathycsus/2686772625

• IP addresses

• Hostnames

• FQDNs

• Search for nodes with Roles

• Find configuration data

Search: Dynamic Infrastructure




Search for Nodespool_members = search("node","role:webserver")

template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end


Pass results into Templates# Set up application listeners here.

listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

So when this...


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

... becomes this ...


JBoss App

Memcache

PostgreSQL Slaves

PostgreSQL Master

NagiosGraphite

...this can happen automatically


NagiosGraphite

JBoss App

Memcache

PostgreSQL Slaves

To Add a New Server…

• 2x Web Server Configurations

• 2 Web Server Restarts

• 4x Database Configurations

• 8x Firewall Configurations

• DNS Service

• Network Configuration

• Deployer

• 8x Monitoring Changes

20+ Changes

12+ New Infrastructure Dependencies

4+ Hours

Count the Resources


NagiosGraphite

JBoss App

Memcache

PostgreSQL Slaves

To Add a New Server…

• 2x Web Server Configurations

• 2 Web Server Restarts

• 4x Database Configurations

• 8x Firewall Configurations

• DNS Service

• Network Configuration

• Deployer

• 8x Monitoring Changes

20+ Changes

12+ New Infrastructure Dependencies

4+ Hours

Count the Resources

Add 1 server20+ Changes


Managing Complexity Later


Managing Complexity LaterWe added:

• Load Balancers

• MemCache

• Search Appliances

• Lots of VM’s

• More Scale

Exponential Increase In:

• Configuration Changes

• Infrastructure Dependencies

• Skills Needed

• Greater Risk


And at Greater Scale...


And at Greater Scale...How Do we Manage This at Cloud Scale?

• Thousands of infrastructure dependencies and configurations needed for each change.

• Huge Amounts of Time

• Increased Cost of Correction of Manual Errors

• Huge Need for Talent

• Risk of Critical Skills Shortage


Automation is a JourneyFull

Automation

Common Automation

Tasks:Scripts, OS Compliance, Updates, etc.

Configuration ManagementDiscovery and

Visibility

Application Management

Continuous Deployment

The Path to the Coded BusinessWednesday, October 2, 13

Flavors of Chef Server


Landscape of Chef-Managed Infrastructure



• SaaS, hosted by Opscode• Manage up to 50,000 servers• Industry-leading SLAs• 24x7x365 Support Options• Get up and running quickly• Pay/grow as you need

Hosted Enterprise Chef



• All the power of Hosted Enterprise, behind the firewall

• Delivered as enterprise software• Implementation consulting customized to

customer’s needs

Enterprise Chef



• Core components of Chef Server developed as open source

• No enterprise features (LDAP, HA, etc.) but many people use it successfully

• Community support

Open Source Chef Server


Learn Chef!


Let’s Learn Chef!

• https://learnchef.opscode.com/• Next meetup: Hands on. Bring laptops!• Feel free to get started early


https://learnchef.opscode.com

https://learnchef.opscode.com

Thanks• Julian Dunn• [email protected]• @julian_dunn• www.juliandunn.net




http://www.juliandunn.net

http://www.juliandunn.net