aws london loft: cloudformation workshop
Post on 13-Feb-2017
216 views
Embed Size (px)
TRANSCRIPT
AWS London Loft: CloudFormation WorkshopTemplated AWS Resources
Tom Maddox
Solutions Architect
tmaddox@amazon.co.uk
Who am I?
Gardener (Capacity Planning)
Motorcyclist (Agility)
Mobile App Writer
Problem Solver
Technology Geek
Solutions Architect
What will we cover?
Introduction to CloudFormation
Template anatomy
Bootstrapping instances
Managing stacks
Best Practices
Demo
Managing complex cloud
architectures
Modern cloud architectures have lots of bits
Modern cloud architectures have lots of bits
S3 buckets
Auto Scaling groups
EC2 instances
CloudFront distributions
Elastic Load Balancing
RDS databases
Elasticache clusters
Security groups
SNS topics & subscriptions
EBS volumes
CloudWatch alarms
Elastic IPs
VPCs
Route tables
IAM users, roles, and policies
Route 53 hosted zones
Use the console?
Time consuming
Error prone
Not repeatableNot (easily) auditable
S3 buckets
Auto Scaling groups
EC2 instances
CloudFront distributions
Elastic Load Balancing
RDS databases
Elasticache clusters
Security groups
SNS topics & subscriptions
EBS volumes
CloudWatch alarms
Elastic IPs
VPCs
Route tables
IAM users, roles, and policies
Route 53 hosted zones
Write a script?
What happens if an API call fails?
How do I roll back?
How can I update my environment?How long should I pause for?
S3 buckets
Auto Scaling groups
EC2 instances
CloudFront distributions
Elastic Load Balancing
RDS databases
Elasticache clusters
Security groups
SNS topics & subscriptions
EBS volumes
CloudWatch alarms
Elastic IPs
VPCs
Route tables
IAM users, roles, and policies
Route 53 hosted zones
Use CloudFormation?
AWS Application Management Services
Elastic Beanstalk OpsWorks CloudFormation EC2
Convenience Control
Higher-level services Do it yourself
CloudFormation Overview
Define & manage your infrastructure as code (JSON templates)
Declarative approach with managed infrastructure dependencies
Self documenting (no more spreadsheets of SG rules)
Intelligent updating of resources and automated rollback capabilities
AWS API interactions taken care for you
Reproducibility
Versioning
Continuous integration for your complete
stack
Version Control Jenkins
Test
Live
Amazon
S3
AWS
CloudFormation
App commit
Infra commit
Pull
Deploy new
template
Deploy
new app
Continuous integration for your complete
stack
Version Control Jenkins
Test
Live
Amazon
S3
AWS
CloudFormation
App commit
Infra commit
Pull
Promote new template
Promote
new app
Template anatomy
Template structure
"Parameters"
"Mappings"
"Conditions"
"Resources"
"Outputs"
Resources
Parameters
"Parameters" : {
InstanceType : {
Description : The EC2 Instance Type to launch.,
Type : String,
AllowedValues : [t1.micro, m1.small, m1.medium]
}
},
InstanceType : { Ref : InstanceType }
Outputs
"Outputs" : {
"InstancePublicDnsName" : {
"Description" : "The public DNS name of the newly created EC2 instance",
"Value" : { Fn::GetAtt" : [ "Ec2Instance, PublicDnsName ] }
}
}
Conditions
"Environment" : {
"Description" : "Specifies if this a Dev QA or Prod Environment",
"Type" : "String",
"Default" : "Dev",
"AllowedValues" : [ "Dev", "QA", "Prod"]
},
"Conditions" : {
"ProdEnvironment" : { "Fn::Equals" : [ { "Ref" : "Environment" }, "Prod" ]}
},
"InstanceType" : { "Fn::If" : [ "ProdEnvironment", m3.2xlarge, m3.medium ] }
Mappings
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" },
"us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" },
"eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" },
"ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" },
"ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" }
}
},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]},
Helpful Links
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
Search: AWS CloudFormation Templates
Helpful solution-based templates (eg: Active Directory domain forest, or LAMP stack)
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/CHAP_TemplateQuickRef.html
Search: AWS CloudFormation Snippets
Handy snippets for common AWS services (eg: Autoscaling group)
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html
- Search: AWS CloudFormation Template Reference
- API docs for each CloudFormation resource type
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/CHAP_TemplateQuickRef.htmlhttp://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html
Demo!
Bootstrapping instances
Use AWS::CloudFormation::Init
Declarative Specify the end state, not how to get there
Debugable cfn-init.log can integrate into CloudWatch Logs
Secure Supports IAM roles
Updatable - Supported as part of the stack update process
Bring your own tools Use cfn-init to bootstrap Chef, Puppet etc.
Use AWS::CloudFormation::Init
"Metadata" : {
"AWS::CloudFormation::Init" : {
"webapp-config": {
"packages" : {},
"sources" : {},
"files" : {},
"groups" : {},
"users" : {},
"commands" : {},
"services" : {}
}
}
},
Using CloudFormation::Init to update
instances
"packages" : {},
"sources" : {},
"files" : {},
"groups" : {},
"users" : {},
"commands" : {},
"services" : {}
1. Update instance metadata in the template
2. UpdateStack
Instance
Metadata
cfn-hup
3. AWS CloudFormation daemon updates configuration
Managing your stacks
Organize by layers & environments
Frontend Services
E-Commerce Website
Backend Services
Search, Payment Gateway, Reviews, Recommendations
Shared Services
Common Monitoring tools, Queues
BaseNetwork
VPCs, Subnets, VPNs, NATs
Identity IAM Users, Groups, Roles
Apply SOA principles
"Parameters" : {
RecommendationsEndPoint : {
Description : URL of the recommendations ELB,
Type : String
}
},
"Outputs" : {
"RecommendationsEndPoint" : {
"Description" : "URL of the recommendations ELB",
"Value" : { Fn::GetAtt" : [ "RecommendationsELB, PublicDnsName ] }
}
}
E-Commerce Website Recommendations Engine
Depends On
Wire
Nested stacks for reusability
ELB_AND_AS
Resources : {
ELB,
AutoScaling
}
Website1
Resources : {
NestedStack,
RDS
}
Website2
Resources : {
NestedStack,
DynamoDB
}
Website1
Resources : {
ELB,
AutoScaling,
RDS
}
Website2
Resources : {
ELB,
AutoScaling,
DynamoDB
}
Deploying Applications
Choose a deployment style
In-place Blue-Green
Stacks
Templates
Amazon
Route 53
Use AutoScalingRollingUpdate for zero downtime
"UpdatePolicy" : {
"AutoScalingRollingUpdate" : {
"MaxBatchSize" : 2,
"MinInstancesInService" : 2,
"PauseTime" : PT20M
}
}
Use
AutoScalingRollingUpdate
for zero downtime
Developer deploys new template
containing change to Launch
Configuration
Auto Scaling group
CloudFormation Stack
CloudFormation replaces a single batch of EC2 instances, the others remain in service
ELB health check confirms new instances are healthy (otherwise rollback)
Next batch of EC2 instances are replaced
Optimizing the ASG UpdatePolicy
On test environments you can optimize for speed and replace all instances
at once
Once live, you should update the ASG in batches making sure you dont have
downtime
for example
"UpdatePolicy": {
"AutoScalingRollingUpdate": {
"PauseTime": "PT0S",
"MaxBatchSize": 6",
"MinInstancesInService": "0"
}
}
"UpdatePolicy": {
"AutoScalingRollingUpdate": {
"PauseTime": "PT15S",
"MaxBatchSize": "2",
"MinInstancesInService": "2"
}
}
For a service with an ASG with 6 instances
TEST LIVE
Best Practices
Validate your templates
ValidateTemplate API validates:
JSON syntax
Absence of circular dependencies
Template structure
Use parameter types
"Parameters" : {