tear it down, build it back up: empowering developers with amazon cloudformation
TRANSCRIPT
James Andrew Vaughn (Andy) @MindTouch
Tear It Down, Build It Back Up: Empowering Developers with
Amazon CloudFormation
James Andrew Vaughn (Andy)
• Software Architect at MindTouch• @modethirteen on Twitter & GitHub• Interests
• Software Build and Testing Automation• Frontend Web Performance• Web Components & Polymer• SSO and Identity Management
@modethirteen
Agenda• What is Amazon CloudFormation? Why use it?
• Managing your release testing and production infrastructure code
• Give developers the power (`cause knowledge is power!)
@modethirteen
All of our customers host their brand on our common, hosted infrastructure.
One mistake and all customer brands look bad #yousuck
@modethirteen
Before CloudFormation• Infrastructure had grown organically over years
• Hand rolled scripts with boto.py to create different EC2 instance types, and manual Puppet runs to configure them
• Non EC2 AWS Resources managed by hand
• No infrastructure in different zones or fast, programatic disaster recovery for entire infrastructure
• Developers were ignorant of production infrastructure
@modethirteen
Developers cannot be isolated from the infrastructure where their code will ultimately run
@modethirteen
Code gives context to problems solved and provides audit trail
for infrastructure design
@modethirteen
CloudFormation: Define creation of AWS resources (EC2 as well as Security Groups, SQS, RDS, etc)
Puppet, Chef, SaltStack, Ansible: Define actions that occur within EC2 instances once they’ve been provisioned
@modethirteen
CloudFormation vs Terraform• Access to nearly every AWS
resource. Better support for VPC, Security Groups, IAM, Cloudfront, SQS
• Stable and mature
• JSON infrastructure templates can be generated by Troposphere (with Python logic)
• Vendor neutrality: AWS, OpenStack, Heroku, etc
• Can execute infrastructure plans as a dry run
• DSL for generating infrastructure templates (HCL)
• If one resource fails to build, subsequent rebuild will only build tainted resource and those dependent on it
• Open source so AWS API coverage can be improved by community
Google Docs: Terraform AWS Coverage
@modethirteen
CloudFormation Stacks
Main Stack
Sub Stacks
A stack is a collection of AWS resources that can be configured
@modethirteen
CloudFormation StacksResources are things that can be queried, configured in the AWS API (including
CloudFormation sub stacks). Examples: Listing S3 buckets, Adding Route 53 DNS entries, Taking DB snapshots
@modethirteen
DatabaseStack
ElasticSearchStack
App Server PoolStack
Main Stack
• AutoScaling::AutoScalingGroup• AutoScaling::LaunchConfiguration• IAM::InstanceProfile• IAM::User
• AutoScaling::AutoScalingGroup• AutoScaling::LaunchConfiguration• CloudFormation::WaitCondition• IAM::InstanceProfile• IAM::User
• RDS::DBInstance• IAM::InstanceProfile• IAM::User
@modethirteen
Custom Resources
• CloudFormation::CustomResource
• Sends custom HTTP message (Service Token) to any of your endpoints, and continues stack execution after response
• AWS SNS
• AWS Lambda
• Node.JS
• Your choice!
@modethirteen
CloudFormation StacksStack parameters come from API input, version controlled JSON
templates, or from the output of other stacks
@modethirteen
• MySQL Storage Engine
App Server PoolStack
DatabaseStack
ElasticSearchStack
App Server PoolStack
Main Stack
• ElasticSearch Version• App Server Pool EC2 Group Name
• ElasticSearch EC2 Group Name• RDS MySQL IP & Port
@modethirteen
CloudFormation StacksParameters of stack can be outputted to dependent stacks.
Example: IP’s, Security Policies, Custom Values, etc.
@modethirteen
Template: {…}
App Server PoolStack
DatabaseStack
ElasticSearchStack
App Server PoolStack
Main Stack
• MySQL Storage Engine
• ElasticSearch EC2 Group Name• RDS MySQL IP & Port
• ElasticSearch Version• App Server Pool EC2 Group Name
Template: {…}
@modethirteen
Puppet / Chef / SaltStack / Ansible
• Stack includes an EC2 Instance or AutoScaling Group Resource
• Resource includes a “UserData” metadata section, for bootstrapping an instance or group of instances
• Include data that cloud-init uses to install instance configuration tool of choice
• curl http://169.254.169.254/latest/user-data
• Example:
• cloud-init installs puppet from UserData commands
• cloud-init runs puppet (configures instance and installs cfn-signal)
• cfn-signal notifies CloudFormation that puppet was success or failure
@modethirteen
Lessons Learned• Goal was to put entire existing AWS infrastructure into
CloudFormation, no immediate value was attained
• Difficult getting buy in for incremental improvements to infrastructure management
• Existing resources cannot be migrated to CloudFormation
• Know the caveats of deleting AWS Resources, they can fail a stack tear down
• AWS Resources missing from CloudFormation API can be mitigated with Custom Resources
• Must understand what a resource does when it updates
@modethirteen
The Teams
• Are developer teams responsible for their own container / infrastructure templates, are operators part of these teams
• Are developers just as responsible for troubleshooting when infrastructure goes down
• What are operator obligations to developers
• What are developer obligations to operations
@modethirteen
TL;DR• Your product is application code, data, services, and
servers
• CloudFormation deploys your product to production
• CloudFormation deploys your product for development and testing
• Your developers can make better decisions
• Your operators can make better decisions
• Your customers / users are happy