managing aws infrastructure using cloudformation

Download Managing AWS infrastructure using CloudFormation

Post on 13-Apr-2017




12 download

Embed Size (px)


AWS User Group Norway Managing AWS infrastructure using CloudFormation by Anton Babenko

23.11.2015Oslo, Norway

About the groupAnders Bjrnestad

Anton Babenko

Arne SolheimOlaf SkaugThis is a group for people interested in Amazon Web Services. Anyone can participate, ranging from AWS evangelists to the curious. The main focus of the group is to build up a community around AWS with socializing and talks on topics like innovations, development and possibilities. Having trouble with a stack? Maybe you'll meet someone with a solution or an approach that you haven't already tried.AWS Certified SysOps,Sr. Software Eng at Telenor Digital AWS Certified Architect and Developer, Sr. Consultant at Webstep Organizers:

Cloud Architect at Nordcloud CTO at Nordeca Insight

Todays meeting

1) News from AWS2) Anton Babenko - Managing AWS infrastructure using CloudFormation3) Arne Solheim - CloudfrontPizza and drinks sponsored by:


Agenda:Basics of CloudFormationUse casesBest practicesChallenges and limitations

AWS CloudFormation

... an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.

IaC approach by AWS (validation and version control of AWS resources)


AWS CloudFormationAn easy way to create & manage a collection of AWS resources.Allows orderly and predictable provisioning and updating of resources.Allows you to version control your AWS infrastructure.Deploy and update stacks using console, command line or API.You only pay for the resources you create.

{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "JSON string", "Metadata" : { template metadata }, "Parameters" : { set of parameters }, "Mappings" : { set of mappings }, "Conditions" : { set of conditions }, "Resources" : { set of resources - REQUIRED }, "Outputs" : { set of outputs }}

Template anatomy

Can be locally or from s3

Metadata - allows to include extra JSON objects describing template (kind of longer description)Parameters - Type, AllowedValues, AllowedPattern, Mix/Max values/length, NoEchoMappings - findInMap, key/values maps, constantsConditions - define logical statement which should be true in order to create specific resource (for example, `if environment = prod then use larger ec2 instance`)REQUIRED Resources - Outputs - aws cloudformation describe-stack


Collection of AWS resourcesSupports:CreateUpdateDescribeDelete


A stack is a collection of AWS resources that you can manage as a single unit.

Reuse Templates to Replicate Stacks in Multiple Environments

Manage resourcesBootstrapping of instancesApplication deployment

Use cases

Manage AWS resourcesInitial bootstrapping of instancesUse Ansible, Puppet, Chef or AWS OpsWorks for more real thingsApplication deploymentConsider using AWS alternatives also (CodeDeploy, Elastic Beanstalk, ECS)

Use cases

Use generator tool (JSON is not for humans)Python: Ruby: Ruby: Scala:

Manage AWS resources

Use generator tools - user-data escaping hellRuby example 2::UserData => base64(interpolate(file('')))Ruby example 1 (erb template):"UserData": { "Fn::Base64": { "Fn::Join": [ "", [ ] ] }Cloudformation JSON example 1:"UserData": { "Fn::Base64": { "Fn::Join": [ "", ["#!/bin/bash -ex","\n", "yum update -y","\n", "# here is my long shell script. Imagine how long it can be :)"] ] }}Cloudformation JSON example 2:"UserData": "here is long & base64 encoded string. Imagine how long it can be :)"

Use Cloudformer to create templates based on existing AWS resources


I have created AWS resources manually

CloudFormation Designer view of CloudFormer stackTemplate:

3 ways to organize stacks

Think about multi-layered architecture and service-oriented architecture (SOA).

Reuse Templates to Replicate Stacks in Multiple Environments

Fit small deploymentsSmall size limit (50kb when load from local file; 450kb when load from S3)

Stand-alone stacks

Easy execution - aws-cliAllow to put reusable parts into separate stacksOne fails = all fails = all rollbackPass parameters to the internal stacks through parentReference to resources in parent: { "Fn::GetAtt" : [ "myVPCStack", "Outputs.VPCId" ] }

Nested stacks

2 websites shared same VPC/subnets

Manual execution one after anotherPass outputs as inputs to downstream templatePass parameters to each stack directlyIndependent failure = independent rollbackCant use WaitContitions with other stacks

Pipelined stacks

Continuous Integration: Infrastructure & ApplicationInfrastructure:Parametrize everythingValidate templates (AWS CLI)Version templatesProbably run on separate AWS account first (CI)

Application deployment:EC2 cloud-init + new application version = new deploy

CloudFormation challengesCant import already created resources without deleting them firstNot all AWS resources/features/services are supported by CloudFormation (eg, EC2 keypairs)No officially supported CloudFormation generator availableNo way to see what kind of changes are going to be appliedFailed state what to do ?Do not update resources created by CloudFormation manually

_FAILED state - resolve manually, console will describe what resource failed, delete stack, contact support.UPDATE_ROLLBACK_FAILED - contact support immediately :(

CloudFormation limitationJSON format is not very human-friendlyNo iteration and limited conditional support (and/or/not/equal)Limited ability to adjust stack based on dynamic conditions Managing dependencies between templatesTieing together inputs/outputs is not directly supported

SummaryUse JSON generatorsKeep templates maintainable and single purposedProbably start with stand-alone stack and iterateDecide how you can handle failures (rollback just one stack or all)Integrate with CI similar way how you do with your applicationUse CloudFormation for very primitive application deployments

Available templates: CloudFormation Masterclass: (DVO304) AWS CloudFormation Best Practices: Links

Questions ?

Thank you!See you at DevOps Norway meetup 14th of December 2015:

Manage AWS infrastructure (as code) using Terraform