aws re:invent 2016: infrastructure continuous delivery using aws cloudformation (dev313)

Download AWS re:Invent 2016: Infrastructure Continuous Delivery Using AWS CloudFormation (DEV313)

Post on 23-Jan-2018




2 download

Embed Size (px)


  1. 1. 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dominic Divakaruni, Anil Kumar - AWS CloudFormation 11/30/2016 Infrastructure Continuous Delivery using AWS CloudFormation
  2. 2. What to expect from this session Well show you how to: Architect your infrastructure using AWS CloudFormation Use AWS CloudFormation to set up AWS CodePipeline pipelines Continuously deliver changes to stacks as you make changes to your templates Demo
  3. 3. Lets look at release processes
  4. 4. Integration tests with other systems Load testing UI tests Penetration testing Release processes have four major phases Source Build Test Production Check-in source code such as .java files. Peer review new code Compile code Unit tests Style checkers Code metrics Create container images Deployment to production environments
  5. 5. Release processes levels Source Build Test Production Continuous integration Continuous delivery Continuous deployment
  6. 6. Continuous deployment Release Processes levels Source Build Test Production Continuous integration Our focus today Continuous delivery
  7. 7. What about continuous delivery tools and processes for infrastructure?
  8. 8. What do we need for infrastructure continuous delivery? A way to treat infrastructure as code. Tools to manage the workflow that creates and updates infrastructure resources. Tools to properly test and inspect your changes for defects and potential issues
  9. 9. What do we need for infrastructure continuous delivery? Infrastructure as code A practice in which infrastructure is provisioned and managed using code and software development techniques, such as version control and continuous integration. Workflow Build, test, and deploy your code every time there is a code change, based on the release process models you define, enabling you to rapidly and reliably deliver changes.
  10. 10. Infrastructure as Code We need Workflow AWS CodePipelineAWS CloudFormation
  11. 11. AWS CloudFormation Create templates of your infrastructure Version control /code review /update templates like code CloudFormation provisions AWS resources based on dependency needs Integrates with development, CI/CD, management tools No additional charge to use
  12. 12. Author templates in JSON or YAML Use Change Sets to preview your changes Continuous delivery workflows for stacks Support for AWS Serverless App Model Enable cross-stack references with Exports Key new features
  13. 13. YAML Author CloudFormation templates in JSON or YAML.
  14. 14. Syntax enhancements JSON YAML with enhancements ! function short form Fn::Sub function substitutes variables
  15. 15. Preview the set of actions CloudFormation will take on your behalf before you create or update stacks. CloudFormation Change Sets Change Sets show you what resources will be created, updated or replaced. This ensures that only expected operations are executed.
  16. 16. Cross Stack References (Exports) Network Stack Outputs: VPC Description: reference VPC Value: !Ref VPC Export: Name: ProdVPC App Stack Resources: myTargetGroup: Type: AWS::ELBV2::TargetGroup Properties: VpcId: Fn::ImportValue: ProdVPC Allows you to share information between independent stacks. Export a stacks output values. Other stacks in the same account and region can import the exported values.
  17. 17. Nested stacks Application Resources: NetworkResources: Type: AWS::CloudFormation::Stack Network Resources Resources: MyVPC Type: AWS::EC2::VPC ECS Service Resources: MyService: Type: AWS::ECS::Service Create a stack composed of multiple templates. Compose and re-use templates with frequently used resources
  18. 18. Considerations for Exports and Nested Stacks Nested Stacks Cross Stack References Recommended uses cases Advantages Considerations Template reuse Use multiple templates but manage as single stack Sharing common resources Allows for independent stacks based on resource lifecycle or ownership. Convenient management. One stack manages all resources and nested stacks. Creation order and dependencies are managed Separation of concern Share databases and VPCs Lets you limit blast radius with safeguards Updates and rollbacks have a wide surface area Reusing templates that have custom resource names Replacing updates requires changes to the importing stacks to execute. Does not manage creation order
  19. 19. Use case
  20. 20. Lets examine a sample application Deconstruct the application into the necessary AWS resources Create CloudFormation templates based your management needs Model your continuous delivery pipeline Continuously deliver infrastrucure changes as you iterate on your architecture Use CloudFormation to model, provision and manage changes to your pipeline
  21. 21. Microservices application based on Amazon ECS Two interconnecting microservices deployed as ECS services (website-service and product-service). The application runs on a highly available ECS cluster deployed across multiple availability zones with auto scaling Available at
  22. 22. Reference architecture Public Subnet Private Subnet Availability Zone Internet Gateway Public Subnet Private Subnet Availability Zone Application Load Balancer NAT GatewayNAT Gateway ECS Cluster CloudWatch Logs (Container Logs) ECS Host ECS Host ECS Host ECS HostAuto Scaling Group
  23. 23. Decompose into AWS resource types NAT Gateway Elastic IP Default Public Route Public Subnet 1Private Subnet 1 Default Private Route Private Route Table NAT Gateway Elastic IP Public Subnet 2Private Subnet 2 Default Private Route Private Route Table AvailabilityZone1AvailabilityZone2 VPC Internet Gateway Public Route Table Load Balancer Security Group ECS Host Security Group Application Load Balancer Load Balancer Listener Load Balancer Default TargetGroup ECS Cluster Auto Scaling Group Auto Scaling Launch Configuration ECS (IAM) Role IAM Instance Profile ECS Service ECS Task Definition CloudWatch Log Group TargetGroup Listener Rule Service Role ECS Service ECS Task Definition CloudWatch Log Group TargetGroup Listener Rule Service Role Network Security Load Balancing Front End Service ECS Cluster Back End Service
  24. 24. Build CloudFormation templates based on this logical grouping Template Description Network VPC, AZs, subnets, routing, NAT and internet gateways Security groups Security groups for the application Load balancers ALBs that are deployed to the public subnets ECS cluster ECS cluster deployed to private subnets Back end service ECS service and task definition for the back end app Front end service ECS service and task definition for the webpage
  25. 25. Setup your template to flow configuration to each other Network Template Security Template Load Balancing Template Front End svc Template ECS Cluster Template Back End Svc Template Outputs Load Balancer Listener Load Balancer DNS Name ECS Cluster Load Balancer Security Group ECS Host Security Group VPC Public Subnets Private Subnets
  26. 26. Network Security Load Balancing ECS Cluster Front End Back End with nested stacks Use these templates to build your stacks with cross-stack references Network Security Load Balancing ECS Cluster Front End Back End Parent Template Microservices Stack Nested templates Templates Individual Stacks
  27. 27. Applying continuous delivery
  28. 28. Applying continuous delivery for your infrastructure Continuous delivery service for fast and reliable application and infrastructure updates Builds, tests and deploys your code each time there is a code change. Built in actions for AWS CloudFormation AWS CodePipeline
  29. 29. How does this align with release phases? Source Test Deploy Source stage for CloudFormation templates can be AWS CodeCommit, S3 or GitHub Use CloudFormation Change Sets to verify deployments prior to execution Create, update or delete Stacks or Change Sets.
  30. 30. Model your pipelines Iterate more often on your application and infrastructure code Launch new versions in Dev and promote to prod Manage your network resources separately per its own cadence. Maintain separate, mirror sandbox and production network environments. Production VPC, Security Groups, Load Balancing Sandbox VPC, Security Groups, Load Balancing Production ECS Cluster, Application Front & Back Ends Dev ECS Cluster, Application Front & Back Ends Application PipelineNetwork Resources Pipeline
  31. 31. Pipeline for network resources Source repo Networking resources for Sanbox/Dev environments Individual stacks. Ordered to account for dependencies Change Sets to preview changes to prod Manual approval before you changes are applied to prod Apply Changes to Prod 1 2 3 4 5
  32. 32. Pipeline for your application Pipeline triggered as soon as new versions are posted Run your tests and clean up your dev environment when done, so you arent charged for the instances you dont use. Review to ensure resource modification or replacement is what you expect Continuously deliver changes to Prod 1 2 3 4
  33. 33. Create and manage your pipeline using CloudFormation Pipeline Artifact Store S3 Bucket Pipeline Notifications SNS Email Notifications Pipeline IAM Roles CloudFormation template to setup your pipeline Could be provisioned in a separate stack with IAM resources with cross-stack refs
  34. 34. Create and manage your pipeline using CloudFormation Choose deploy action with CloudFormation as the provider CloudFormation has enabled several action modes REPLACE_ON_FAILURE creates a new stack if one doesnt exist, updates it if it does or replaces it if its in a failed state You can use template