managing infrastructure as code
TRANSCRIPT
![Page 1: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/1.jpg)
Managing Infrastructure as Code
Allan Shone
![Page 2: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/2.jpg)
Agenda• Look at some legacy concepts• Run through some ideas• Check out a few products• Put together some base requirements• Browse a few modern products• Throw in some more updated concepts• Dive into suitable products• Extra bits, concerns, and ideas
![Page 3: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/3.jpg)
Back in the day...• Infrastructure was extremely manual• Hosts, meta details, and further information was
recorded in text files, or other plain text based systems
• Hostnames based on function• Documentation was fragmented• Legacy infrastructure becomes unmanageable
and can be forgotten• Nightmare to keep track of
![Page 4: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/4.jpg)
Tools• Wiki / HTML Tables• Text Files
• Shared drives• Internal admin server• Shared FTP• Proprietary and specific software solutions• Shell scripts
![Page 5: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/5.jpg)
Problems?• Keeping host lists up-to-date• Multiple users managing infrastructure• Recent status indicators• Cumbersome interfaces• Time consuming data interactions• What about software?
![Page 6: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/6.jpg)
Ideas• Some sort of versioning• Easier interface for collaboration• Provision of host state• Start looking at automation
![Page 7: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/7.jpg)
Bits and Pieces
• Databases• Services• Applications• Cache• Routers
![Page 8: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/8.jpg)
Software?
• First, infrastructure requires orchestration• Then, software dependencies can be pushed within
each of those infrastructure items• Bare-metal is different with a separate set of
requirements• The premise for both though is still of value to the
general topic
![Page 9: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/9.jpg)
Basic Provisioners
![Page 10: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/10.jpg)
Ansible• Provides inheritance• Allows for variable configuration• Straight-forward to use with automation• Expressive with its syntax using YAML• Playbooks used for grouping of instructions• Playbooks versioned in a DIY fashion• Agentless model for deployment• Templating makes conf files a breeze
![Page 11: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/11.jpg)
Simple Software# Playbook for Application- hosts: “{{hosts}}” remote_user: root sudo: yes
roles: - common - app-server
# Directory Structureroles/common/handlers/main.ymlroles/common/tasks/main.ymlroles/common/templates/ntpd.confroles/app-server/handlers/main.ymlroles/app-server/tasks/main.ymlroles/app-server/templates/apache.confroles/app-server/vars/example.yml
![Page 12: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/12.jpg)
Provisioning Infrastructure- hosts: localhost connection: local gather_facts: false
tasks: - name: Provision instances ec2: key_name: my_key group: test instance_type: t2.micro image: “{{ami_id}}” count_tag: Name: Demo instance_tags: Name: Demo register: ec2
- name: Add Hosts to Host Group add_host: hostname={{ item.public_ip }} groups=ec2hosts with_items: ec2.instances - hosts: ec2hosts name: configuration play user: ec2-user gather_facts: true tasks: - name: Check NTP Service service: name=ntpd state=started
![Page 13: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/13.jpg)
Drawbacks
• Difficult to track created instances• Supplier specific wrapper• Versioning is DIY• Basic in terms of complete solution
![Page 14: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/14.jpg)
Chef
• Builds on Ruby for syntax• Fluent way of pushing modifications• Variable capabilities for ease of automation• Cookbooks used to group instructions• Cookbooks synchronised with the Chef Server• Server to client model
![Page 15: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/15.jpg)
Simple Instances
num_instances = 10
1.upto(num_instances) do |inst|
machine "my-machine-#{inst}" do
add_machine_options bootstrap_options: {
security_group_ids: 'test-sg',
subnet_id: 'subnet-1234567',
instance_type: 't2.micro'
}
end
end
![Page 16: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/16.jpg)
Resources
load_balancer "test-elb" do
machines [ "machine1", "machine2" ]
load_balancer_options :listeners => [{
:port => 80,
:protocol => :http,
:instance_port => 80,
:instance_protocol => :http,
}]
end
![Page 17: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/17.jpg)
Drawbacks
• Dedicated server for management• Uses Ruby natively, which could be a positive if you
work with Ruby or don’t mind• What is required for some may not exist unless the
necessary Plugin is available for it• OS and Package restrictions for nodes
![Page 18: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/18.jpg)
Puppet
• Simple syntax for configuration• Server model for deployments• Automation readily available• Parameterised configurations for easy environment
setup
![Page 19: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/19.jpg)
Software
package { 'apache2': provider=>'apt', ensure=>'installed'} notify { 'Apache2 is installed.':} service { 'apache2': ensure=>'running'} notify { 'Apache2 is running.':}
![Page 20: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/20.jpg)
ec2_securitygroup { 'sample-group': ensure => present, region => 'us-west-1', description => 'Group used for testing Puppet AWS module',}
ec2_instance { 'sample-instance': ensure => present, region => 'us-west-1', availability_zone => 'us-west-1a', image_id => 'ami-696e652c', instance_type => 't1.micro', security_groups => ['sample-group'],}
Resources
![Page 21: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/21.jpg)
Resource - Finalising
ec2_loadbalancer { 'sample-load-balancer': ensure => present, region => 'us-west-1', availability_zones => ['us-west-1a', 'us-west-1b'], instances => ['sample-instance', 'another-instance'], security_groups => ['sample-group'], listeners => [{ protocol => 'tcp', port => 80, }], }
![Page 22: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/22.jpg)
Drawbacks
• Requires learning the Puppet specific language for the actual infrastructure code
• Complex infrastructure can become quite cumbersome to manage
• Dependency based, order of execution can be tricky to control when it is required to be
![Page 23: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/23.jpg)
What about hosts?
![Page 24: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/24.jpg)
CloudFormation
• Complete “physical” infrastructure as code• Basic JSON file for definition• Services for usage easily interacted with• Tightly coupled with AWS• Versioned and stored within the console• Ease of automation
![Page 25: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/25.jpg)
Beginning{
"AWSTemplateFormatVersion": "2016-01-01","Description": "My Template","Parameters": {
"KeyName": {"Description": "EC2 KeyPair","Type": "AWS::EC2::KeyPair::KeyName","ConstraintDescription": "must be the name of an existing EC2 KeyPair."
}}
}
![Page 26: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/26.jpg)
Resources - Security Group"Resources": {
"WebServerSecurityGroup": {"Type": "AWS::EC2::SecurityGroup","Properties": {
"GroupDescription": "Minimal Access","SecurityGroupIngress": [{
"IpProtocol": "tcp","FromPort": "80","ToPort": "80","CidrIp": "0.0.0.0/0"
}]}
}}
![Page 27: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/27.jpg)
Resources - Instance
"Resources": {"WebServer": {
"Type": "AWS::EC2::Instance","Metadata": {
"AWS::CloudFormation::Init": {"install": {
"packages": {"yum": {
"httpd": []}
}}
}}
![Page 28: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/28.jpg)
Tying it together
"Outputs": {"WebsiteURL": {
"Value": {"Fn::Join": ["", [
"http://",{
"Fn::GetAtt": ["WebServer","PublicDnsName"
]},"/ping"
]]},"Description": "Website"
}}
![Page 29: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/29.jpg)
Drawbacks
• AWS specific• JSON for the configuration can be difficult to create
and maintain - No comments• Not idempotent• Templates are very large and can become quite
cumbersome to follow• Most functionality can be automated through the
command line interface within other tools
![Page 30: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/30.jpg)
Infrastructure pieces
• Software management, host management, resources• A general tool provides one but not the other• Arbitrary scripts can shoehorn this• Duplication and Inconsistencies would become
problematic with keeping data sets in different tools
![Page 31: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/31.jpg)
Combinations
• Software dependencies managed• Hosts instantiated or made available on demand• Configurations completed between environments to
allow for sand-boxed communication• Entire infrastructures brought up with a single
command as replica of production
![Page 32: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/32.jpg)
TerraForm
• Will orchestrate and provision• Syntax is easy to grasp and maintain• Configurations can be quite simple• Parameterised capabilities for ease of scripting with
environments
![Page 33: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/33.jpg)
Softwareresource "aws_instance" "web" { connection { user = "ubuntu" } instance_type = "m1.small" ami = "${lookup(var.aws_amis, var.aws_region)}" key_name = "${aws_key_pair.auth.id}" vpc_security_group_ids = ["${aws_security_group.default.id}"] subnet_id = "${aws_subnet.default.id}" provisioner "remote-exec" { inline = [ "sudo apt-get -y update", "sudo apt-get -y install nginx", "sudo service nginx start" ] }}
![Page 34: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/34.jpg)
Resourcesresource "aws_elb" "web" { name = "terraform-example-elb" subnets = ["${aws_subnet.default.id}"] security_groups = ["${aws_security_group.elb.id}"] instances = ["${aws_instance.web.id}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" }}resource "aws_key_pair" "auth" { key_name = "${var.key_name}" public_key = "${file(var.public_key_path)}"}
![Page 35: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/35.jpg)
Drawbacks
• Tightly integrated with vendors• Learning curve for syntax• Delays with updated services and functionality• Newcomer to the fully managed tool suite, some
features are incomplete or in progress
![Page 36: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/36.jpg)
ManageaCloud
• Complete solution, orchestration and provisioning• Simple, re-usable configuration• Built-in versioning for deployments and infrastructure• Open choice of vendor - no requirements• Framework approach for infrastructure management
![Page 37: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/37.jpg)
Macfile
• Configuration template• Complete infrastructure specification• Versioned to allow for ease of use, deployment, and
rollback• Simple syntax no vendor specifics
![Page 38: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/38.jpg)
App Instance
roles: demo_app: instance create: configuration: demo_application
infrastructures: demo_application_instance: name: demo provider: amazon location: us-east-1 hardware: t1.micro role: demo_app environment: - APP_BRANCH: master
![Page 39: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/39.jpg)
Resource
resources: elastic_load_balancer: create bash: aws elb create-load-balancer --load-balancer-name infrastructure.param.name --listeners infrastructure.param.listeners --availability-zones infrastructure.param.availability-zones --region infrastructure.param.region destroy bash: aws elb delete-load-balancer --load-balancer-name infrastructure.param.name --region infrastructure.param.region
![Page 40: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/40.jpg)
Resource Instance
infrastructures: load balancer 01: resource: elastic_load_balancer params: name: my-demo-load-balancer listeners: Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=80 availability-zones: us-east-1b us-east-1c region: us-east-1
![Page 41: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/41.jpg)
Associationactions: get_id: ssh: wget -q -O - http://169.254.169.254/latest/meta-data/instance-idregister_lb: create bash: aws elb register-instances-with-load-balancer --load-balancer-name infrastructure.param.load-balancer-name --instances infrastructure.param.instances --region infrastructure.param.regioninfrastructures: register_instance: ready: role.demo_app resource: register_lb params: load-balancer-name: my-demo-load-balancer instances: role.demo_app.get_id
![Page 42: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/42.jpg)
Drawbacks
• Most components are open source, not all at the present time
• No unified syntax for providers
![Page 43: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/43.jpg)
What about people?
![Page 44: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/44.jpg)
DevOps
• Largely, DevOps came about as a hybrid role to help manage and facilitate process change
• Automation is a key aspect• Not Operations, but not Development either (but is
still both)• Provide an interface between infrastructure and
environments and deployments made
![Page 45: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/45.jpg)
Concepts
• Even with automation, humans are still needed• Sanity checking and improving tools• Removing bottlenecks• Increasing developer and wider business productivity• Know the management tools and the details of how
the infrastructure functions
![Page 46: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/46.jpg)
Workflows
• Very important to focus on processes• Tools are wonderful, but processes need to be suitable
for the tool of choice• Automation will bring down the Op aspects of DevOps• Cross functional efforts to bring the automation to the
infrastructure• Size of infrastructure
![Page 47: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/47.jpg)
Infrastructure as Code
![Page 48: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/48.jpg)
Decisions
• Situations make decisions difficult• Complete solutions are not always necessary• Preference and team knowledge makes a difference• A product is not specifically good just because others
use it
![Page 49: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/49.jpg)
Options
• There’s always more options available than time to discuss - CFengine, Salt, Heat, OneOps
• It’s all about automation, and removing bottlenecks in cumbersome processes
![Page 50: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/50.jpg)
Future
• Abilities to share, extend, and work better with infrastructures
• Inheritance for roles, resources, and instances• Complete control with automation of infrastructure
sets• Simple options for deployment strategies
![Page 51: Managing Infrastructure as Code](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5871ddbb1a28ab6a7b8b4e79/html5/thumbnails/51.jpg)
Thank you!
Allan Shone
https://manageacloud.com