scaling the platform for your startup

Scaling the Platform for Your Startup

Andreas ChatzakisAWS Solutions Architecture

Why are you here?

• Building the technology platform for your startup• You want to prepare for success• Learn about design patterns & scalability• A pragmatic approach for startups

Priorities for startups

• Racing within a window of opportunity• Small team with no legacy• Focus on solving a problem• Avoid over-engineering & re-engineering• Reduce risk of failure when you go viral

A scalable architecture

• Can support growth in users, traffic, data size • Without practical limits• Without a drop in performance• Seamlessly - just by adding more resources• Efficiently - in terms of cost per user

First some basics

AWS Regions

US-WEST (Oregon)

EU-WEST (Ireland)ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC (Sydney)

ASIA PAC (Singapore)

CHINA (Beijing)

Availability Zones (AZs)

US-WEST (Oregon)

EU-WEST (Ireland)ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC (Sydney)


CHINA (Beijing)EU-CENTRAL (Frankfurt)


CHINA (Beijing)

Edge Locations

EnterpriseApplications

Virtual Desktop Sharing & Collaboration

PlatformServices

Analytics

Hadoop

Real-timeStreaming Data

DataWarehouse

DataPipelines

App Services

Queuing & Notifications

Workflow

App streaming

Transcoding

Email

Search

Deployment & Management

One-click web app deployment

Dev/ops resource management

Resource Templates

Mobile Services

Identity

Sync

Mobile Analytics

Push Notifications

Administration& Security

Identity Management

Access Control

Usage Auditing

Key Storage

MonitoringAnd Logs

CoreServices

Compute(VMs, Auto-scaling and Load Balancing)

Storage(Object, Block and Archival)

CDNDatabases(Relational, NoSQL, Caching)

Networking(VPC, DX, DNS)

Infrastructure Regions Availability Zones Points of Presence

Day 1 – Dev & private beta

Single host

THE server(e.g. Apache,

MySQL)

Elastic IPwww.example.com

Amazon Route 53DNS service

Server Image (AMI)

Day 2 - Public beta

We need a bigger server

• Add larger & faster storage (EBS)• Use the right instance type• Easy to change instance sizes• Not our long term strategy• Will hit an endpoint eventually• No fault tolerance

Separating web and DB

• More capacity• Scale each tier individually• Tailor instance for each tier

– Instance type– Storage

• Security– Security groups– DB in a private VPC subnet

But how do I choose what DB technology I need?

SQL? NoSQL?

Why start with a Relational DB?

• SQL is versatile & feature-rich• Lots of existing code, tools, knowledge• Clear patterns to scalability (for read-heavy apps)• Reality: eventually you will have a polyglot data layer

– There will be workloads where NoSQL is a better fit– Combination of both Relational and NoSQL– Use the right tool for each workload

Key Insight: Relational Databases are Complex

• Our experience running Amazon.com taught us that relational databases can be a pain to manage and operate with high availability

• Poorly managed relational databases are a leading cause of lost sleep and downtime in the IT world!

• Especially for startups with small teams

Relational Databases

Fully managed; zero admin

AmazonRDS

Deploy

Amazon Aurora

Improving efficiency

Offload static content• Amazon S3: highly available hosting that scales

– Static files (JavaScript, CSS, images)– User uploads

• S3 URLs – serve directly from S3• Let the web server focus on dynamic content

Masterclass Live: Amazon S3

Upcoming Session at the AWS Pop Up Loft

Wednesday 16th September, 10.00

Amazon CloudFront• Worldwide network of edge locations• Cache on the edge

– Reduce latency– Reduce load on origin servers – Static and dynamic content– Even few seconds caching of popular content can have huge impact

• Connection optimizations– Optimize transfer route– Reuse connections– Benefits even non cachable content

CloudFront

CloudFront for static & dynamic content

AmazonRoute 53

EC2 instance(s)

S3 bucket

Static content

Dynamic content

css/*js/*Images/*

Default(*)

CloudFront

distribution

Database caching• Faster response from RAM• Reduce load on database

Application server

1. If data in cache, return result

2. If not in cache, read from DB

RDS database

Amazon ElastiCache

3. And store in cache

Amazon ElastiCache: in-memory cache

• Simple to Deploy • Managed

– Automatically replaces failed nodes– Patch management

• Elastic• Compatible ElastiCache

Day 3 – Paying customers

High Availability

Availability Zone a

RDS DB instance

Web server

S3 bucket forstatic assets

www.example.com


Amazon CloudFront

ElastiCachenode 1

High Availability

Availability Zone a

RDS DB instance

Availability Zone b

Web server

Web server


www.example.com


Amazon CloudFront

ElastiCachenode 1

High Availability

Availability Zone a

RDS DB instance

Availability Zone b

www.example.com


Elastic LoadBalancing

Web server

Web server


Amazon CloudFront

ElastiCachenode 1

Elastic Load Balancing

• Managed Load Balancing Service• Fault tolerant• Health Checks• Distributes traffic across AZs• Elastic – automatically scales its capacity

Data Layer HA

Availability Zone a

RDS DB instance

Availability Zone b

www.example.com



Web server

Web server


ElastiCachenode 1

Amazon CloudFront

Availability Zone a

RDS DB instance

Availability Zone b

www.example.com



Web server

Web server

RDS DBstandby


ElastiCachenode 1

Amazon CloudFront

Data Layer HA

Data Layer HA

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



Web server

Web server

RDS DBstandby

Data Layer HA

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



Web server

Web server

RDS DBstandby

ElastiCachenode 2

User sessions• Problem: Often stored on local disk (not shared)

• Quickfix: ELB Session stickiness• Solution: DynamoDB


Web server

Web server

Logged in Logged out

Amazon DynamoDB• Managed document and key-value NoSQL DB• Simple to launch and scale• To millions of IOPS• Both reads and writes

• Consistent, fast performance• Durable: perfect for storage of session data

https://github.com/aws/aws-dynamodb-session-tomcat

http://docs.aws.amazon.com/aws-sdk-php/guide/latest/feature-dynamodb-session-handler.html

AWS bootcamp: Architecting Highly Available Applications on AWS

Free Training at the AWS Pop Up Loft

9th October 10.00

Day 4 – Let’s go viral!

Replace guesswork with elastic IT

Startups pre-AWS

Demand

Unhappy Customers

Waste $$$

Traditional

Capacity

Capacity

Demand

AWS Cloud

Scaling the web tier

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



Web server

Web server

RDS DBstandby

ElastiCachenode 2

Scaling the web tier

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



Web server

Web server

RDS DBstandby

ElastiCachenode 2

Web server

Web server

Automatic resizing of compute clusters based on demand

Feature Details

Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs.

Integrated to Amazon CloudWatch

Use metrics gathered by CloudWatch to drive scaling.

Instance types Run Auto Scaling for on-‐demand and Spot Instances. Compatible with VPC.

aws autoscaling create-auto-scaling-group--auto-scaling-group-name MyGroup--launch-configuration-name MyConfig--min-size 4--max-size 200--availability-zones us-west-2c, us-west-2b

Auto Scaling Trigger auto-scaling policy

Amazon CloudWatch

Decompose into small, loosely coupled, stateless

building blocks

Prerequisite

What does this mean in practice?

• Only store transient data on local disk• Needs to persist beyond a single http request?– Then store it elsewhere

User uploads

User Sessions

Amazon S3

AWS DynamoDB

Application Data

Amazon RDS

Having decomposed into small, loosely coupled, stateless building blocks

You can now Scale out with ease

Having done that…

Having decomposed into small, loosely coupled, stateless building blocks

We can also Scale back with ease

Having done that…

Take the shortcut

• While this architecture is simple you still need to deal with: – Configuration details– Deploying code to multiple instances– Maintaining multiple environments (Dev, Test, Prod)– Maintain different versions of the application

• Solution: Use AWS Elastic Beanstalk

AWS Elastic Beanstalk (EB)• Easily deploy, monitor, and scale three-tier web applications and services.

• Infrastructure provisioned and managed by EB • You maintain control.• Preconfigured application containers • Easily customizable.• Support for these platforms:

Deploy your Apps with AWS Elastic Beanstalk


Tuesday 15th September, 16.00

Day 5 – Add more features

Mobile

PushNotifications

MobileAnalytics Cognito Cognito

Sync

Analytics

Kinesis DataPipelineRedShift EMR

Your Applications

AWS Global Infrastructure

Network

VPC DirectConnect Route 53

Storage

EBS S3 Glacier CloudFront

Database

DynamoDBRDS ElastiCache

Deployment & Management

ElasticBeanstalk OpsWorks Cloud

FormationCodeDeploy

CodePipeline

CodeCommit

Security & Administration

CloudWatch Config CloudTrail IAM Directory KMS

Application

SQS SWF AppStream

ElasticTranscoder SES Cloud

SearchSNS

Enterprise Applications

WorkSpaces WorkMail WorkDocs

Compute

EC2 ELB AutoScalingLambdaECS

AWS building blocksInherently Scalable & Highly Available Scalable & Highly Available

a Elastic Load Balancinga Amazon CloudFront

a Amazon Route53a Amazon S3a Amazon SNS / SQSa Amazon SESa Amazon CloudSearch

a AWS Lambdaa …

a Amazon DynamoDBa Amazon Redshift

a Amazon RDSa Amazon Elasticachea …

4 Amazon EC24 Amazon VPC

Automated Configurable With the right architecture

Stay focused as you scale your team

AWSCloud-‐Based

Infrastructure

YourBusiness

More Time to Focus onYour Business

Configuring Your Cloud Assets

70%

30%70%

On-‐PremiseInfrastructure

30%

Managing All of the “Undifferentiated Heavy Lifting”

Don’t reinvent the wheel

• Notification system• E-Mail component• Search engine• Workflow engine• Queue• Transcoding system• Monitoring system

Amazon CloudSearch

Amazon SQSAmazon SNS

Amazon Elastic Transcoder

Amazon SWFAmazon SES

If you find yourself writing your own…

Search features

• Freetext AND Structured• Synonyms• Stemming• Relevance• Complex scoring• Faceting• Geospatial

Amazon CloudSearch is a fully managed search service in the cloud for your website or application

Rich search feature set

Source: IDC, Nielsen, Twitter blog

Faceting Highlighting Autocompletesuggestions

Geospatial search

34 languages covering major geographies

Ease of operation

Auto-scaling

Highavailability

Self-healingclusters

Datadurability

Day 6 – Growing fast

Prepare for the challenges

• Increase in concurrent users• Data size growth• More features => more DB tables• Technical debt (e.g. inefficient queries)

– Reduce it– Manage it

Scaling Relational DBs – option 1

Increase Database instance specs– Larger instance type (RAM, CPU, Network)– More storage – Faster storage (IOPS)– Amazon Aurora

My SQL compatible

Available, durable, and fault tolerant

5X better performance of high-end MySQL database

Highly scalable and secure

Up to 64TB of storage

Amazon Aurora1/10th the cost of the leading commercial database solutions

Amazon Aurora - Write performance

• MySQL Sysbench

• R3.8XL with 32 cores and 244 GB RAM

• 4 client machines with 1,000 threads each

Amazon Aurora - Read performance

• MySQL Sysbench

• R3.8XL with 32 cores and 244 GB RAM

• Single client with 1,000 threads

Scaling Relational DBs – option 2

Read Replicas (Master – Slave)– Scale out beyond capacity of single DB instance– Available in Amazon RDS for MySQL, PostgreSQL and Amazon Aurora– Replication lag– Writes => master– Reads with tolerance to stale data => read replica (slave)– Reads with need for most recent data => master

Scaling the DB

Web server

Web server

Web server

Web server

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



RDS DBstandby

ElastiCachenode 2

Scaling the DB

Web server

Web server

Web server

Web server

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



RDS DBstandby

ElastiCachenode 2

RDS read replica

Scaling the DB

Web server

Web server

Web server

Web server

Availability Zone a

RDS DB instance

ElastiCachenode 1

Availability Zone b


www.example.com



RDS DBstandby

ElastiCachenode 2

RDS read replica

RDS read replica

Amazon Aurora Replicas have less replication lag

2.6 3.4 3.9 5.4

1,000 2,000 5,000 10,0000

50,000

100,000

150,000

200,000

250,000

300,000

350,000

Updates per second

Read replica lag in milliseconds

Read replica lag

Aurora

RDS MySQL;;30,000 IOPS (Single AZ)

Updates persecond

Amazon Aurora

RDS MySQL30K IOPS(single AZ)

1,000 2.62 ms 0 s

2,000 3.42 ms 1 s

5,000 3.94 ms 60 s

10,000 5.38 ms 300 s

Write workload250 tablesQuery cache on for Amazon Aurora, off for MySQL (best settings)

What if your app is write-heavy?

Challenge: You will eventually hit the write throughput or storage limit of the master node

Solutions:• Federation (splitting into multiple DBs based on function)• Sharding (splitting one data set up across multiple hosts)

Database federation• Split up databases by function/purpose

• Harder to do cross-function queries

• Essentially delaying the need for something like sharding/NoSQL until much further down the line

• Won’t help with single huge functions/tables

Forums DB

Users DB

Products DB

Sharded horizontal scaling

• More complex at the application layer

• ORM support can help• No practical limit on scalability

• Operation complexity/sophistication

• Shard by function or key space

• RDBMS or NoSQL

User ShardID

002345 A002346 B002347 C002348 B002349 A

Shard CShard B

Shard A

NoSQL data stores

• Trade query & integrity features of Relational DBs for– More flexible data model – Horizontal scalability & predictable performance

DynamoDBProvisioned read/write performance per table

Massive and Seamless Scale

• Distributed system that can scale both reads and writes– Sharding + Replicas

• Automatic partitioning:– Data set size growth– Provisioned capacity increases table

Low provisioned throughput

TablePartition

SSD

Region

Illustrative diagram only

Availability ZoneAvailability Zone Availability Zone

ReplicaPartition

SSD

ReplicaPartition

SSD

Increased provisioned throughput


Region

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

TablePartition

SSD

High provisioned throughput

Region


Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Table

Partition

Monitoring

CloudwatchOperationally you will need:• Metrics

– Default– Custom

• Alarms• Centralized logs

AWS Marketplace & Partners Can Help

• Find, research, buy software• Aligns with EC2 usage model • Launch in minutes• Marketplace billing integrated into your AWS account

• 20+ categories

Learn more at: aws.amazon.com/marketplace

Loose Coupling

Amazon Simple Queuing Service (SQS)

Tight coupling

• Place asynchronous tasks into Amazon SQS• Respond quickly to end users• Protects backend systems from spikes• Process at own pace

• Protect end users from backend problems• Item will be processed eventually

SQS

Get Message

Back End EC2 Instance

Put Message

Front End EC2 Instance

Decoupling in action

Loose coupling

Upload photo Resize photo

EC2 Instances


Loose coupling

Q

Upload photo Resize photo

Upload photoResize photo

EC2 Instances

Amazon SQS

EC2 Instances


Loose coupling

Q

Upload photo Resize photoUpload photo Resize photoUpload photo Resize photo

Upload photoResize photoResize photoResize photo

EC2 Instances

Amazon SQS

EC2 Instances

Introducing AWS Lambda

Amazon S3 Bucket Events AWS Lambda

Original image Thumbnailed image

1

2

3

Event-Driven Compute in the Cloud

Lambda functions: Stateless, request-driven code execution• Triggered by events in other services:

• PUT to an Amazon S3 bucket• Write to an Amazon DynamoDB table• Record in an Amazon Kinesis stream• Amazon SNS Message received• Changes in Amazon Cognito data

• Makes it easy to…• Transform data as it reaches the cloud• Perform data-driven auditing, analysis, and notification• Kick off workflows

Data Triggers – Amazon Simple Notification Service

Lambda FunctionSNSCloudWatchMetric

Data Triggers – Amazon Cognito

Lambda FunctionCognito

Data Triggers: Amazon DynamoDB

AWS LambdaAmazon DynamoDBTable and Stream

Send SNS Push notifications

Update another table

Data Triggers – Amazon Kinesis

IoTDevice

Kinesis Lambda DynamoDB

Dynamic content generation based on incoming news text

and images

Real time log processing for

prediction analytics

Thumbnailinginstallation site photos

for mobile use

Real time processing and recording of inbound traffic from

a range of social media platforms

Large scale distributed search across blog

content

Operational analytics and real time troubleshooting

Introducing the AWS API Gateway

Internet

Mobile Apps

Websites

Services

API Gateway

AWS Lambda functions

AWS

API Gateway Cache

Endpoints on Amazon EC2 / Amazon Elastic

Beanstalk

Any other publicly accessible endpoint

Amazon CloudWatch Monitoring

No server is easier to manage than "no server”.

Werner Vogels

Startups without Servers


Thursday 17th September, 11.00Thursday 1st October, 10.00

Build a data-driven company

Experiment and collect data

• Build an MVP• Run A/B testing• Analyze user behavior• Iterate• Get insights about your customers

Data Produced

Available for Analysis

Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares

Three Types of Data Analytics

Retrospectiveanalysis and reporting

Here-and-nowreal-time processing and dashboards

Predictionsto enable smart

apps

Amazon Kinesis Amazon EC2

Amazon DynamoDBAWS Lambda

Amazon Redshift Amazon RDS Amazon S3Amazon EMR

AWS Marketplace

Amazon Machine LearningAmazon EMRAWS Lambda

Fast, simple, petabyte-scale data warehousing for less than $1,000/TB/Year

Amazon Redshift

Amazon Redshift Architecture

• Leader Node– SQL endpoint– Stores metadata– Coordinates query execution

• Compute Nodes– Local, columnar storage– Execute queries in parallel– Load, backup, restore via

Amazon S3;; load from Amazon DynamoDB or SSH

• Two hardware platforms– Optimized for data processing– DW1: HDD;; scale from 2TB to 1.6PB– DW2: SSD;; scale from 160GB to 256TB

10 GigE(HPC)

IngestionBackupRestore

SQL Clients/BI Tools

128GB RAM

16TB disk

16 cores

Amazon S3 / DynamoDB / SSH

JDBC/ODBC

128GB RAM

16TB disk

16 coresCompute Node

128GB RAM

16TB disk


128GB RAM

16TB disk


LeaderNode

Amazon Redshift Node Types

Dense Storage

DW1.XL

15 GB RAM

2 TB disk

2 vcpus

Single Node (2 TB)

Cluster 2-32 Nodes (4 TB – 64 TB)

XL XL XL XL XL XL XL XL




XL


Dense Storage

DW1.8XL

120 GB RAM

16 vcpus

16 TB disk

Cluster 2-100 Nodes (32 TB – 1.6 PB)8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL

8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL










Dense Compute

DW2.L

15 GB RAM

160 GB SSD

2 vpus

Single Node (160 GB)

Cluster 2-32 Nodes (320 GB – 5 TB)

L

L L L L L L L L

L L L L L L L L

L L L L L L L L

L L L L L L L L


Dense Compute

DW2.8XL

244 GB RAM

32 vcpus

2.5 TB SSD

Cluster 2-100 Nodes (5 TB – 256 TB)8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL










AWS bootcamp: Building a Big Data Platform on AWS

Free Training at the AWS Pop Up Loft

2nd October 10.00

Stay Lean

It is cheap to get started on AWSHow do successful startups save more as they

grow and extend their runway?

Stay Lean!

http://aws.amazon.com/solutions/case-studies/flipboard

Amazon EBS standard volume prices are lowered up to 50% for both storage and I/O requests.

January 21, 2014

50%Amazon is reducing prices for Amazon ElasticCache cache nodes by an average of 34%.March 26, 2014

34%Amazon S3: We are reducing prices for Standard and Reduced Redundancy Storage, by an average of 51%.

March 26, 2014

51%

We’ve announced price reductions 42* times since our inception in 2006. Recent price drops included…

*as of April 1, 2014

Main cost-saving principles

1. Turn off the lights2. Use Auto Scaling3. Use the newer instance types4. Use the right instances for your workload5. Use Reserved Instances for predictable workloads6. Use Spot Instances for async workloads7. Leverage Amazon S3 storage classes8. Use Glacier for archival9. Serve content through Cloudfront10. Offload your architecture

Tools: Cost Explorer, Detailed Billing, Alerts, Usage Reports

Summary

Amazon Route 53DNS serviceNo limit

Availability Zone a

RDS DB instance

ElastiCachenode 2

Availability Zone b


www.example.com


RDS DBstandby

ElastiCachenode 3

RDS read replica

RDS read replica

DynamoDB

RDS read replica

ElastiCachenode 4

RDS read replica

ElastiCachenode 1

CloudSearchLambdaSES SQS

A quick review• Keep it simple and stateless• Make use of managed self-scaling services• Multi-AZ and AutoScale your EC2 infrastructure• Use the right DB for each workload • Cache data at multiple levels• Simplify operations with deployment tools

Next steps?READ! • aws.amazon.com/documentation• aws.amazon.com/architecture• aws.amazon.com/start-ups• aws.amazon.com/training

ASK FOR HELP!• forums.aws.amazon.com• aws.amazon.com/support

AWS for Startups

Upcoming Sessions at the AWS Pop Up Loft

11th September, 13.0016th September, 17.0029th September, 17.0014th October, 18.00

scaling the platform for your startup

Technology