Scaling in all directions on AWS

IGMAR PALSENBERG • SOFTWARE ARCHITECT • COOLGAMES • @PALSENBERG

I’m a coderI’m an architect

I do devops

H O W TO D O T H I S O N AW S

W H AT I S S C A L I N G ?

T H E A P P L I C AT I O N P E R S P E C T I V E

Agenda

Scaling and scalable architectureAWS, but a lot also applies to othercloud providers (Google, Azure)Ways to do autoscaling and about thesoftware architecture required

This is about…

What is scaling ?

Add more nodes at an existing pool of resources

Add more resources to a single node

Do It YourselfElastic beanstalkCode deploy

Ways to do this on AWS

Cloud formationApplication management, build on EC2, Autoscaling and Opsworks

A combination of Elastic Beanstalk, Code Pipeline, Cloudformation and Opsworks

Infrastructure as a template

Elastic BeanstalkCode Deploy

Why Do It Yourself ?

Cloud formationDocker

GoJava SETomcat.NET

Node.JSPHP

Do It YourselfUser

ELB

Cloud watch

Autoscaling

Instances

Autoscaling

Cloud watchELBAutoscaling group

Euh, what ?

Attached to 2 autoscaling groups. Can be external or internal.

2 groups to support rolling code updates and rollbacks

Metrics used to trigger autoscaling actions

CloudwatchELBAutoscaling group

How does autoscaling work

Loadbalancer, serving as a HTTP / HTTPS / TCP endpoint. Distributes based on round robin.

Grouped set of instances in one region, launched from the same AMI.

Cloud watch alarms act as a trigger point for autoscaling actions.

Details matter.It’s worth waiting to get it right

Create a decent health check Pick your autoscaling policy wisely. Scale up fast, scale down slow.Pick decent instance sizes.

Pick sane metrics, metric vs hits should be linear.

• The health check tells the loadbalancer that an instance is ready to accept traffic.• If it returns OK, but starts returning errors (5xx), or timeouts, the instance is

considered unhealthy and is replaced.• key metrics :1. Ping target2. Timeout3. Interval4. Healthy / unhealthy threshold

Health check ?

Creating your own AMI’s

•Gives you immutable infrastructure

•Packer build files are JSON, ideal for version control

•Build can be automated in a CI environment

•Bash, Salt Puppet or Chef to compose an image

•Produces images for EC2, Azure, Google Compute, Digital Ocean, Docker, VMWare, VirtualBox

•Single binary, no dependencies

The Application

The architecture

User

eu-west-1 ap-southeast-2 us-west-2

Time to face reality

MonitorAssume failureBe reactive

Loadbalancer, serving as a HTTP / HTTPS / TCP endpoint. Distributes based on round robin.

• Isolate all the things

• Act autonomously

• Do one thing, and do it well

• Go asynchronously

• Gives services a home address

• Dedicated error channel

Cloud watch alarms act as a trigger point for autoscaling actions.

Global state is evil

MonitorAssume failureBe reactive

• Isolate all the things

• Act autonomously

• Do one thing, and do it well

• Go asynchronously

• Gives services a home address

• Dedicated error channel

Handle errors gracefully

Reactive

• Use a message broker, or pick Akka Cluster

• Always use sane, short timeouts : 0.5 - 1 second is a good default

• Pick a framework that supplies you with the tools to get started

• Use a proper test environment. That’s not your laptop

• Don’t assume consistency. Assume that things are eventually consistent

• If you’re not sure : Guess. If you’re wrong : Apologise and compensate

• Transitions are fine within a microservice. Outside of that, create a reverse action,so that in every stage, it can be undo, undoing the whole action as such

• Split up read and write in two separate channels

Use services that scale

Do and don’t on AWS

• RDS

• Memcached

• GlusterFS / local storage

Don’t Do

•S3

•DynamoDB

•Redshift

•Route53

•SQS

http://aws.amazon.comhttp://www.packer.iohttp://www.reactivemanifesto.orghttps://www.playframework.com

Links to reading material

Demo time !!!!!

Questions ?

Thank you!

IGMAR PALSENBERG • SOFTWARE ARCHITECT • COOLGAMES • @PALSENBERG