deep dive: infrastructure as code - aws - amazon s3and...deep dive: infrastructure as code steven...

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

Deep Dive: Infrastructure as Code Steven Bryen – Solutions Architect, AWS

Raj Wilkhu – Principal Engineer, JUST EAT Bruce Jackson – CTO, Myriad Group AG

You are on-board …

needs to experiment, innovate, reduce risk Business

of services and applications Continuous

Delivery

culture, automation, measurement, sharing DevOps

infrastructure-as-code Cloud

AWS CloudFormation •  Create templates of the infrastructure and

applications you want to run on AWS

•  Have the CloudFormation service automatically provision the required AWS resources and their relationships from the templates

•  Easily version control, replicate or update the infrastructure and applications using the templates

•  Integrates with other development, CI/CD, and management tools.

Create template – Example web application

security group

Auto Scaling group

EC2 instance

Elastic Load Balancing

DB Service

Inventory Service

Recommendations Service

ElastiCache memcached cluster

Software pkgs, config, & data CloudWatch

alarms

Create template – Resources

security group

Auto Scaling group

EC2 instance




alarms

"Resources" : { "SecurityGroup" : {}, "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "MinSize" : "1", "MaxSize" : "3", "LoadBalancerNames" : [ { "Ref" : "LoadBalancer" } ], ... } }, "LoadBalancer" : {}, "CacheCluster" : {}, "Alarm" : {} },

CloudFormation Template

Create template – Parameters

Auto Scaling group

EC2 instance

Recommendations Service Inventory

Service Customer DB

Service

Info to Customize Stack at Creation. Examples: Instance Type, App Pkg Version

"Parameters" : { "CustomerDBServiceEndPoint" : { "Description" : "URL of the Customer DB Service", "Type" : "String" }, "CustomerDBServiceKey" : { "Description" : "API key for the Customer DB Service", "Type" : "String", "NoEcho" : "true" }, "InstanceType" : { "Description" : "WebServer EC2 instance type", "Type" : "String", "Default" : "m3.medium", "AllowedValues" : ["m3.medium","m3.large","m3.xlarge"], "ConstraintDescription" : "Must be a valid instance type"


Create template – Outputs


"Resources" : { "LoadBalancer" : {}, ... }, "Outputs" : { "WebsiteDNSName" : { "Description" : "The DNS name of the website", "Value" : { "Fn::GetAtt" : [ "LoadBalancer", "DNSName" ] } } }


Create template – Deploy and configure software

Auto Scaling group

EC2 instance

Software pkgs, config, & data

"AWS::CloudFormation::Init": { "webapp-‐config": { "packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {}

}, "chef-‐config" : {} }

CloudFormation Template ü  Declarative ü  Debug-able ü  Updatable ü  Highly Secure ü  BIOT™ Bring In

Other Tools

Create template – Language features

Use a wide range of AWS services

ü  Auto Scaling ü  Amazon CloudFront ü  AWS CloudTrail ü  AWS CloudWatch ü  Amazon DynamoDB ü  Amazon EC2 ü  Amazon ElastiCache ü  AWS Elastic Beanstalk ü  AWS Elastic Load Balancing ü  Amazon Kinesis

ü  AWS Identity and Access Mgmt

ü  AWS OpsWorks ü  Amazon RDS ü  Amazon Redshift ü  Amazon Route 53 ü  Amazon S3 ü  Amazon SimpleDB ü  Amazon SNS ü  Amazon SQS ü  Amazon VPC

and more … As of April 2015

Nested CloudFormation Stacks

"myStackWithParams" : { "Type" : "AWS::CloudFormation::Stack", "Properties" : { "TemplateURL" : "https://s3.amazonaws.com/template-bucket/mystack.template", ”Parameters" : { "InstanceType" : "t1.micro",

"KeyName" : "mykey"

}

} }

Pass parameters to nested CloudFormation Stacks

Iterate on infrastructure

Update stack In-place Blue-Green

Faster

Cost-efficient

Simpler state and data migration

Working stack not touched

Extending AWS CloudFormation

Extend with Custom Resources

security group

Auto Scaling group

EC2 instance




alarms Web Analytics

Service AWS

CloudFormation

Provision AWS Resources

"Resources" : { "WebAnalyticsTrackingID" : { "Type" : "Custom::WebAnalyticsService::TrackingID", "Properties" : { "ServiceToken" : "arn:aws:sns:...", "Target" : {"Fn::GetAtt" : ["LoadBalancer", "DNSName"]}, "Plan" : "Gold" } }, ...

“Success” + Metadata

“Create, Update, Rollback, or Delete” + Metadata

Lambda-powered custom resources

security group

Auto Scaling group

EC2 instance




alarms

// Implement Custom Logic Here

Lookup an AMI ID

Lookup VPC ID and Subnet ID

Reverse an IP Address

Application Lifecycle

Infrastructure Lifecycle

EC2

SQS, SNS, Kinesis, etc.

Databases

VPC

IAM

Application Lifecycle

Download Packages, Install Software, Configure Apps, Bootstrap Apps,

Update Software, Restart Apps,

etc.

CloudFormation

• Templatize • Replicate • Automate

AWS::CloudFormation::Init

"AWS::CloudFormation::Init": {

"webapp-config": {

"packages" : {}, "sources" : {}, "files" : {},

"groups" : {}, "users" : {},

"commands" : {}, "services" : {}

Declarative

AWS::CloudFormation::Init Supports updates

"packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {}

AWS::CloudFormation::Init

"install_chef" : {},

"install_wordpress" : {

"commands" : {

"01_get_cookbook" : {}, ...,

"05_configure_node_run_list" : {

"command" : "knife node run_list add -z `knife node list -z` recipe[wordpress]",

"cwd" : "/var/chef/chef-repo",

"env" : { "HOME" : "/var/chef" }

Flexibility to bring in other tools such as AWS CodeDeploy and Chef

ow.ly/DiNkz

Use AWS::CloudFormation::Init "UserData": { "# Get the latest CloudFormation helper scripts package\n", "yum update -y aws-cfn-bootstrap\n", "# Trigger CloudFormation::Init configuration \n", "/opt/aws/bin/cfn-init --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n", "# Signal completion\n", "/opt/aws/bin/cfn-signal –e $? --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n"

Use CloudWatch Logs for debugging

"install_logs": {

"packages" : { ... "awslogs" ... },

"services" : { ... "awslogs" ... }

"files": {

"/tmp/cwlogs/cfn-logs.conf": {}

file = /var/log/cfn-init.log log_stream_name = {instance_id}/cfn-init.log file = /var/log/cfn-hup.log log_stream_name = {instance_id}/cfn-hup.log

ow.ly/E0zO3

Use CloudWatch Logs for debugging

ow.ly/E0zO3

Wait Conditions & DependsOn

Example Wait for EC2 instance to signal success

"Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "UserData" : { "Fn::Base64" : {"Ref" : "myWaitHandle"}},

"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]} } },

Example

"myWaitHandle" : { "Type" : "AWS::CloudFormation::WaitConditionHandle",

}, "myWaitCondition" : {

"Type" : "AWS::CloudFormation::WaitCondition", "DependsOn" : "Ec2Instance", "Properties" : { "Handle" : { "Ref" : "myWaitHandle" }, "Timeout" : "4500" }

}

Wait for EC2 instance to signal success

CI/CD Pipeline

CloudFormation in a CI/CD Pipeline

AWS CloudFormation Issue Tracker

App Developers

DevOps Engineers, Infrastructure Developers,

Systems Engineers

Dev Env Code Repo

App Pkgs, CloudFormation Templates, Etc.

CI Server

Test

Staging

Prod Code Review

"Infra-‐as-‐Code"

App Code & Templates

LONDON


Continuous Integration and Deployment

Raj Wilkhu – Principal Engineer, JUST EAT

JUST EAT Platform

•  High volume e-commerce platform •  45-60 minutes cycle •  Real time message delivery and confirmation network •  Hardware in 22,500 restaurants in the UK, 40,800 worldwide •  Peak traffic is 10,000% of normal daytime traffic •  3 cities •  Over 130 releases a month •  Primarily Windows

Building reliable and high quality software

•  Test-Driven Development, Continuous Deployment and Immutable Infrastructure

•  Every part of the infrastructure replaced several times a day •  Incentive to continuously speed up deployment process

Deploying components

•  Platform decomposed into components •  Each component defined by single AWS Cloudformation

template •  Single Build Artifact •  Instances boot and bootstrap themselves from

•  pre-baked AMI •  Content in S3 based on Cloudformation::Init metadata

Pre-baked AMI vs Dynamic config

•  System dependencies (eg OS features, .NET, ruby, gems, etc)

•  Static configurations

•  Continuous Deployment (latest code) •  Environment specific configuration •  Automation to deploy feature

Increasing Boot time

•  Unzip package •  Install and configure dependencies •  Execute deployment script(s) •  Warm up feature •  Tagged by deployment scripts

•  Baked via CI pipeline •  Tested and tagged

Pre-baked AMI CloudFormation::Init

Cloudformation template generation

•  Single json from multiple yaml and erb files •  Similar to HTML templating engines •  Organized by component •  Convention over configuration

"<%= @feature_name %>_pre_deploy": { "commands": { <% cmd_prefix = 250 services_to_disable = %w(aelookupsvc bits scdeviceenum trustedinstaller wuauserv) services_to_disable.each do |service_name| %> "<%= cmd_prefix %>_<%= @feature_name %>_stop_<%= service_name %>_service_because_unnecessary": { "waitAfterCompletion": "0", "command": "powershell -noninteractive -noprofile -command \"& { stop-service -Name <%= service_name %> -verbose }\"", "ignoreErrors": true }, "<%= cmd_prefix + 1 %>_<%= @feature_name %>_disable_<%= service_name %>_service_because_unnecessary": { "waitAfterCompletion": "0", "command": "powershell -noninteractive -noprofile -command \"& { set-service -Name <%= service_name %> -StartupType Disabled -verbose }\"", "ignoreErrors": true }, <% cmd_prefix += 2 end %>

--- :feature_config: :ami_id: ami-c99544ef :elb: true :use_logging_client: true :elb_dns: true :parameters: :ElbTarget: "HTTP:80/status" :AsgDesiredSizeDuringPeakTime: 10 :InstanceType: c4.xlarge

Example yaml: Example json.erb template:

{ Master

Immutable infrastructure - Cloudformation

Engineers

configs in yaml, template extensions in

json.erb + master

Tests in QA environment

Engineers

Component code

Continuous Integration Server (TeamCity Agent Cloud)

Build & Test Pipeline

Artifact

JustDeploy

Production/QA

EC2 Web/worker roles

Route 53

Rooms

Artifact

Git Git

Immutable infrastructure - Baking AMIs

Engineers

AMI Bakery

Continuous Integration Server (TeamCity Agenet Cloud)

EC2 base image

configurations and resources

Shared AMI

Latest Amazon OS image

Git

Automated Tests

config.yaml updated

Future

•  Automatically select last tested AMI image •  Use lambda to trigger AMI builds based on AMI release

notifications via SNS •  Failover to secondary region •  Publish more tools as OSS

–  https://github.com/justeat

@justeat_tech + @rajwilkhu http://tech.just-eat.com

Yes, we’re recruiting too. http://tech.just-eat.com/jobs


Deployment on the fly Bruce Jackson – CTO, Myriad Group AG

From IMPS to msngr

database

app server

web server web server web server

...x16

•  Move from a mid-2000 carrier platform to web scale

•  …with 38M users on 16 stacks

•  …in 6 months •  …with a team of 4

Basic foundations of our approach

Rules •  OSGi used for all

application stacks •  Make use of as much

managed infrastructure as possible

•  Avoid any environment configuration

•  Make deployment as familiar as coding

Rationale •  Rapid creation of services

against interfaces •  We don’t have any devops

so outsource management where possible

•  No files/scripts that only one person understands

•  The team is made up of developers

What did we do? •  Created utility libraries

that use code annotations and AWS tags for configuration

•  Use Eclipse features to define bundles to deploy on systems

What did we do (2) ?

•  Created Eclipse tooling to manage deployment & code/feature checkout

A quick demonstration…

LONDON

deep dive: infrastructure as code - aws - amazon s3and...deep dive: infrastructure as code steven...

Documents