AWS London Loft: CloudFormation Workshop

Download AWS London Loft: CloudFormation Workshop

Post on 13-Feb-2017

216 views

Category:

Documents

3 download

Embed Size (px)

TRANSCRIPT

  • AWS London Loft: CloudFormation WorkshopTemplated AWS Resources

    Tom Maddox

    Solutions Architect

    tmaddox@amazon.co.uk

  • Who am I?

    Gardener (Capacity Planning)

    Motorcyclist (Agility)

    Mobile App Writer

    Problem Solver

    Technology Geek

    Solutions Architect

  • What will we cover?

    Introduction to CloudFormation

    Template anatomy

    Bootstrapping instances

    Managing stacks

    Best Practices

    Demo

  • Managing complex cloud

    architectures

  • Modern cloud architectures have lots of bits

  • Modern cloud architectures have lots of bits

    S3 buckets

    Auto Scaling groups

    EC2 instances

    CloudFront distributions

    Elastic Load Balancing

    RDS databases

    Elasticache clusters

    Security groups

    SNS topics & subscriptions

    EBS volumes

    CloudWatch alarms

    Elastic IPs

    VPCs

    Route tables

    IAM users, roles, and policies

    Route 53 hosted zones

  • Use the console?

    Time consuming

    Error prone

    Not repeatableNot (easily) auditable

    S3 buckets

    Auto Scaling groups

    EC2 instances

    CloudFront distributions

    Elastic Load Balancing

    RDS databases

    Elasticache clusters

    Security groups

    SNS topics & subscriptions

    EBS volumes

    CloudWatch alarms

    Elastic IPs

    VPCs

    Route tables

    IAM users, roles, and policies

    Route 53 hosted zones

  • Write a script?

    What happens if an API call fails?

    How do I roll back?

    How can I update my environment?How long should I pause for?

    S3 buckets

    Auto Scaling groups

    EC2 instances

    CloudFront distributions

    Elastic Load Balancing

    RDS databases

    Elasticache clusters

    Security groups

    SNS topics & subscriptions

    EBS volumes

    CloudWatch alarms

    Elastic IPs

    VPCs

    Route tables

    IAM users, roles, and policies

    Route 53 hosted zones

  • Use CloudFormation?

  • AWS Application Management Services

    Elastic Beanstalk OpsWorks CloudFormation EC2

    Convenience Control

    Higher-level services Do it yourself

  • CloudFormation Overview

    Define & manage your infrastructure as code (JSON templates)

    Declarative approach with managed infrastructure dependencies

    Self documenting (no more spreadsheets of SG rules)

    Intelligent updating of resources and automated rollback capabilities

    AWS API interactions taken care for you

    Reproducibility

    Versioning

  • Continuous integration for your complete

    stack

    Version Control Jenkins

    Test

    Live

    Amazon

    S3

    AWS

    CloudFormation

    App commit

    Infra commit

    Pull

    Deploy new

    template

    Deploy

    new app

  • Continuous integration for your complete

    stack

    Version Control Jenkins

    Test

    Live

    Amazon

    S3

    AWS

    CloudFormation

    App commit

    Infra commit

    Pull

    Promote new template

    Promote

    new app

  • Template anatomy

  • Template structure

    "Parameters"

    "Mappings"

    "Conditions"

    "Resources"

    "Outputs"

  • Resources

  • Parameters

    "Parameters" : {

    InstanceType : {

    Description : The EC2 Instance Type to launch.,

    Type : String,

    AllowedValues : [t1.micro, m1.small, m1.medium]

    }

    },

    InstanceType : { Ref : InstanceType }

  • Outputs

    "Outputs" : {

    "InstancePublicDnsName" : {

    "Description" : "The public DNS name of the newly created EC2 instance",

    "Value" : { Fn::GetAtt" : [ "Ec2Instance, PublicDnsName ] }

    }

    }

  • Conditions

    "Environment" : {

    "Description" : "Specifies if this a Dev QA or Prod Environment",

    "Type" : "String",

    "Default" : "Dev",

    "AllowedValues" : [ "Dev", "QA", "Prod"]

    },

    "Conditions" : {

    "ProdEnvironment" : { "Fn::Equals" : [ { "Ref" : "Environment" }, "Prod" ]}

    },

    "InstanceType" : { "Fn::If" : [ "ProdEnvironment", m3.2xlarge, m3.medium ] }

  • Mappings

    "Mappings" : {

    "RegionMap" : {

    "us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" },

    "us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" },

    "eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" },

    "ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" },

    "ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" }

    }

    },

    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]},

  • Helpful Links

    http://aws.amazon.com/cloudformation/aws-cloudformation-templates/

    Search: AWS CloudFormation Templates

    Helpful solution-based templates (eg: Active Directory domain forest, or LAMP stack)

    http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/CHAP_TemplateQuickRef.html

    Search: AWS CloudFormation Snippets

    Handy snippets for common AWS services (eg: Autoscaling group)

    http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html

    - Search: AWS CloudFormation Template Reference

    - API docs for each CloudFormation resource type

    http://aws.amazon.com/cloudformation/aws-cloudformation-templates/http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/CHAP_TemplateQuickRef.htmlhttp://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html

  • Demo!

  • Bootstrapping instances

  • Use AWS::CloudFormation::Init

    Declarative Specify the end state, not how to get there

    Debugable cfn-init.log can integrate into CloudWatch Logs

    Secure Supports IAM roles

    Updatable - Supported as part of the stack update process

    Bring your own tools Use cfn-init to bootstrap Chef, Puppet etc.

  • Use AWS::CloudFormation::Init

    "Metadata" : {

    "AWS::CloudFormation::Init" : {

    "webapp-config": {

    "packages" : {},

    "sources" : {},

    "files" : {},

    "groups" : {},

    "users" : {},

    "commands" : {},

    "services" : {}

    }

    }

    },

  • Using CloudFormation::Init to update

    instances

    "packages" : {},

    "sources" : {},

    "files" : {},

    "groups" : {},

    "users" : {},

    "commands" : {},

    "services" : {}

    1. Update instance metadata in the template

    2. UpdateStack

    Instance

    Metadata

    cfn-hup

    3. AWS CloudFormation daemon updates configuration

  • Managing your stacks

  • Organize by layers & environments

    Frontend Services

    E-Commerce Website

    Backend Services

    Search, Payment Gateway, Reviews, Recommendations

    Shared Services

    Common Monitoring tools, Queues

    BaseNetwork

    VPCs, Subnets, VPNs, NATs

    Identity IAM Users, Groups, Roles

  • Apply SOA principles

    "Parameters" : {

    RecommendationsEndPoint : {

    Description : URL of the recommendations ELB,

    Type : String

    }

    },

    "Outputs" : {

    "RecommendationsEndPoint" : {

    "Description" : "URL of the recommendations ELB",

    "Value" : { Fn::GetAtt" : [ "RecommendationsELB, PublicDnsName ] }

    }

    }

    E-Commerce Website Recommendations Engine

    Depends On

    Wire

  • Nested stacks for reusability

    ELB_AND_AS

    Resources : {

    ELB,

    AutoScaling

    }

    Website1

    Resources : {

    NestedStack,

    RDS

    }

    Website2

    Resources : {

    NestedStack,

    DynamoDB

    }

    Website1

    Resources : {

    ELB,

    AutoScaling,

    RDS

    }

    Website2

    Resources : {

    ELB,

    AutoScaling,

    DynamoDB

    }

  • Deploying Applications

  • Choose a deployment style

    In-place Blue-Green

    Stacks

    Templates

    Amazon

    Route 53

  • Use AutoScalingRollingUpdate for zero downtime

    "UpdatePolicy" : {

    "AutoScalingRollingUpdate" : {

    "MaxBatchSize" : 2,

    "MinInstancesInService" : 2,

    "PauseTime" : PT20M

    }

    }

  • Use

    AutoScalingRollingUpdate

    for zero downtime

    Developer deploys new template

    containing change to Launch

    Configuration

    Auto Scaling group

    CloudFormation Stack

    CloudFormation replaces a single batch of EC2 instances, the others remain in service

    ELB health check confirms new instances are healthy (otherwise rollback)

    Next batch of EC2 instances are replaced

  • Optimizing the ASG UpdatePolicy

    On test environments you can optimize for speed and replace all instances

    at once

    Once live, you should update the ASG in batches making sure you dont have

    downtime

  • for example

    "UpdatePolicy": {

    "AutoScalingRollingUpdate": {

    "PauseTime": "PT0S",

    "MaxBatchSize": 6",

    "MinInstancesInService": "0"

    }

    }

    "UpdatePolicy": {

    "AutoScalingRollingUpdate": {

    "PauseTime": "PT15S",

    "MaxBatchSize": "2",

    "MinInstancesInService": "2"

    }

    }

    For a service with an ASG with 6 instances

    TEST LIVE

  • Best Practices

  • Validate your templates

    ValidateTemplate API validates:

    JSON syntax

    Absence of circular dependencies

    Template structure

  • Use parameter types

    "Parameters" : {

Recommended

View more >