immutable deployments with aws cloudformation and aws lambda

Download Immutable Deployments with AWS CloudFormation and AWS Lambda

Post on 08-Jan-2017

22.365 views

Category:

Technology

7 download

Embed Size (px)

TRANSCRIPT

  • (and AWS Lambda)

  • & aoepeople!

  • E-Commerce: Magento

    CMS: TYPO3

    Portals: ZF, Symfony,

    Mobile Searchperience: ElasticSearch

    250+ people world-wide

    (in 8 locations)

    Global Enterprise Projects

    Infrastructure: AWS

  • CloudFormation Lambda Immutable

  • /i(m)myoodb()l/

    adjective

    unchanging over time or unable to be changed.

    disposable ephemeral

  • Pet Cattle

  • not disposable disposable

  • disposable

    not disposable

    disposable

  • Static Resources Build VPC Build

  • Build Bucket

    IAM Setup

    VPC

    Static Resources

    Build

    Manual Setup

    separate

    CloudFormation

    Stacks use resources

    from underlying stacks as input

    parameters Lambda functions,

    Monitoring,

  • Build Bucket

    IAM Setup

    Environment A (e.g. prod) Environment B (e.g. stage)

    VPC VPC

    Static Resources Static Resources

    Build Build Build Build

    Build X Build X+1 Build X Build X+1

    Manual Setup

    different scopes (build, environment,

    account, global)

  • Build Bucket

    IAM Setup

    Environment A (e.g. prod) Environment B (e.g. stage) Environment C (e.g. qa) Environment D (e.g. dev)

    IAM Setup

    VPC VPC VPC VPC

    Static Resources Static Resources Static Resources Static Resources

    Build Build Build Build Build Build Build Build

    Build X Build X+1 Build X Build X+1 Build X Build X+1 Build X Build X+1

    Manual Setup Manual Setup

    Build Bucket Access

  • Private subnets

    Public subnets

    ElastiCache (Redis)

    with replication groups

    for cache and sessions

    RDS (multi-az) with

    DB subnet group

    Bastion

    server

    s3: media

    storage*

    Route 53: DNS

    configuration CloudFront

    distribution

    SSL

    Certificates

    Security group for Varnish servers

    Security group for Magento servers

    Security group for Load Balancer

    Static Resources

  • Build

    Auto Scaling group

    Auto Scaling group

    Elastic Load

    Balancer

    Auto-

    Scaling

    Group

    Launch

    Configurati

    on

    Scaling

    Policy

  • Build

    Auto Scaling group

    Auto Scaling group

    Elastic Load

    Balancer

    Auto-

    Scaling

    Group

    Launch

    Configurati

    on

    Scaling

    Policy

    Auto Scaling group Auto Scaling group

  • +1

    Baking AMIs

  • Chef vs. Puppet?

    http://fbrnc.net/blog/2015/11/

    how-to-provision-an-ec2-instance

    Ansible! BASH!

    http://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instancehttp://fbrnc.net/blog/2015/11/how-to-provision-an-ec2-instance

  • Keep it simple!

  • ()

  • the last mile

    most underestimated CFN feature!

  • var r = require('cfn-response'); exports.handler = function (event, context) { [] var res = {}; if (event.RequestType == 'Create') { res.Password = randomPassword(20); } r.send(event, context, r.SUCCESS, res); };

  • "IndexerDb": { "Type": "AWS::RDS::DBInstance", "Properties": { [...] "MasterUserPassword": {"Fn::GetAtt": ["GenerateDbPassword", "Password"]}, [...] } },

    "GenerateDbPassword": { "Type": "Custom::PasswordGenerator", "Properties": { "ServiceToken": {"Ref": "PasswordGeneratorArn"} } },

  • launch some ASGs (set DesiredCapacity) create database passwords tag all resources (incl. ElastiCache!) restore database and media files (one-time) install scripts (db migrations,) detect Varnish backends wait for healthy backends in the ELBs run infrastructure tests cache warming update Route53 records sets delete old stacks

  • DesiredCapacity =

    Number of Instances in current ASG x 1.2

    better safe than sorry

  • DesiredCapacity =

    Number of Instances in current ASG x 1.2

  • DesiredCapacity =

    Number of Instances in current ASG x 1.2

    ScalingPolicy takes care

  • "CountInstances": { "Type": "Custom::InstanceCounter", "Properties": { "ServiceToken": {"Ref": "InstanceCounter"}, "AutoScalingGroupTags": [ {"Key": "Environment", "Value": "prod"}, {"Key": "Type", "Value": "Frontend"} ], "Min": 1, "Max": 10, "Factor": "1.5" } },

    "FrontendAsg": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { [...] "DesiredCapacity": {"Fn::GetAtt": ["CountInstances", "Count"]}, "Tags": [ {"Key": "Environment", "Value": "prod", "PropagateAtLaunch": true}, {"Key": "Type", "Value": Frontend", "PropagateAtLaunch": true} ] } },

  • Update Route53 record set

  • Update Route53 record set

    Delete old stacks

  • "UpdateR53": { "DependsOn": [ "MagentoWaitCondition", "MagentoWorkerWaitCondition", "Elb"], "Type": "Custom::Route53Update", "Properties": { "ServiceToken": {"Ref": "R53Updater"}, "Name": {"Fn::Join": ["", [ "www-", {"Ref": "EnvironmentName"} ]]}, "HostedZoneId": {"Ref": "HostedZoneId"}, "AliasTargetDNSName": {"Fn::GetAtt": ["Elb", "DNSName"]}, "AliasTargetHostedZoneId": {"Fn::GetAtt": ["Elb", "CanonicalHostedZoneNameID"]}, "Comment": "Updated via CloudFormation/Lambda" } },

    "DeleteStacks": { "Condition": "DeleteOldStacks", "DependsOn": ["UpdateR53"], "Type": "Custom::StackDeleter", "Properties": { "ServiceToken": {"Ref": "StackDeleter"}, "TagFilter": { "Environment": {"Ref": "EnvironmentName"}, "Type": "Deployment"}, "ExceptStackName": {"Ref": "AWS::StackName"} } }

  • +1

    Baking AMIs

    Tools+CI

  • Have fun filling out 47 form fields! :)

  • aoepeople/stackformation

    command-line tool (Symfony console, uses

    AWS SDK for PHP)

    integrates nicely into your CI (Jenknis,)

  • pre-process

    Userdata

    Lambda JS Template

    superset of CFN template (JSON)

    Parameters lookup from other stacks, env vars,

    Template

    Parameter

    Values

    create/ update

    Stack policies, Tags,

  • Stack magento-stage-build-5

    Stack magento-stage-build-6

    Stack magento-prod-build-6

    Stack magento-prod-build-5

    CloudFormation Template

    merge & pre-process

    CloudFormation+X Template(s)

    + Dynamic Parameters

    + Stack Policies

    + Behavior

    + Tags

    Blueprint magento-{env:ENVIRONMENT}-build-{env:BUILD}

    Blueprint magento-{env:ENVIRONMENT}-setup

    Stack magento-stage-setup

    Stack magento-prod-setup

  • blueprints: - stackname: 'magento-{env:BUILD}' template: 'magento.template' stackPolicy: 'policy.json' OnFailure: 'DO_NOTHING' parameters: Build: '{env:BUILD}' KeyPair: '{var:KeyPair}' VPC: '{resource:setupstack:VPC}' Subnet: '{resource:setupstack:Subnet}' InstanceSg: '{resource:setupstack:InstanceSg}' InstanceProfile: '{output:setupstack:InstanceProfile}' BootAmi: 'ami-06116566' tags: Environment: 'prod' Build: '{env:BUILD}'

    enforce immutability by denying updates!

  • aoepeople/cfn-vpc

    aoepeople/cfn-lambdahelper

    aoepeople/cfn-amibaker

    via composer

  • so we can integrate this into our CI pipeline

  • aoepeople/awsinspector

    command-line tool (Symfony console, uses

    AWS SDK for PHP)

    Domain models for PHP

  • $repository = new \AwsInspector\Model\Elb\Repository(); $dns = $repository->findElbsByTags([ 'Environment' => 'deploy', 'Build' => 554, 'Type' => 'Frontend ])->getFirst()->getDNSName();

  • > bin/awsinspector.php ec2:ssh -t Environment:prod c Type c Build

    filter by tag

    Please select an instance [0] i-1033ed9b (Type: Frontend; Environment: prod; Build: 477) [1] i-4ff36ec8 (Type: Backend; Environment: prod ; Build: 477) [2] i-5ab4322b (Type: Worker; Environment: prod; Build: 477) [3] i-705ad42f (Type: Worker; Environment: prod; Build: 476) >

    will take jump hosts into account (ProxyCommand)

    auto-detects your local

    (encrypted) private keys

    multiplexed ssh connections

    run commands directly

  • logins, orders,

    deployments, scaling activity,

    JMeter response time, error rate,

    CPU, load, network I/O,

  • correlate metrics from

    various sources

  • Grafana ElasticSearch (Service)

    time-series database

  • leftovers from disposed resources

  • Ready for testing

    (WaitCondition)

    setup terminate

    instances

    Install JMeter

    Download

    testcase from

    S3