aws pop-up loft berlin: cache is king - running lean architectures: optimizing for cost efficiency

Cache is KingScale Your Application whileImproving Performance and Lowering Costs

Constantin Gonzalez

Principal Solutions Architect

Amazon Web Services Germany GmbH

[email protected] | @zalez

Cash is KingCash is King

Cache is Cash

Httparchive

Content Breakdown

87% of Your Site Consists Of Static/Re-usable Content

52% of The Internet’s Content is

NOT Cached Today

But Wait…

Cache is King

Source: httparchive.org300K Tracked Websites Oct 2014-Oct 2015

~52%{

CACHE EVERYTHING

Browser Cache

Edge Cache

Web Cache

Application Cache

Database Cache

Image Source: cakeormistake.wordpress.com

http://cakeormistake.wordpress.com/2012/01/10/top-ten-2012-cake-trends/

Speed of Light ~300,000 Km/s

Image Source: wordlesstech.com

http://wordlesstech.com/2011/09/24/cern-scientists-break-the-speed-of-light/

~200,000 Km/s through fiber

Image Source: www2.wiringandcabling.com

http://www2.wiringandcabling.com/wiring-cabling-companies-in-cochran-fiber-optics-cat5-cat-6-and-structured-cabling-quotes.php

Download Performance

Round Trip: 13,026 km



Theoretical: 65 ms



Theoretical: 65 ms

Measured: 68 ms

Time to First Byte68 ms

Client Server


68 ms

Client Server

SYN

SYN/ACK


68 ms

68 ms

Client Server136 ms

SYN

SYN/ACK

ACK,

GET /image.jpg HTTP/1.1

Browser Cache

Browser Caching

Set max-age or expiry date in the headers.

HTML5 Application Cache.

Helps eliminate network latency.

But… Browser cache size is limited.(e.g. IE is 8-50M, Chrome is < 80M, Firefox is 50MB, etc.)

Browser Cache

Edge Cache


68 ms

68 ms

Client Server136 ms

SYN

SYN/ACK

ACK,


Bring the Bytes Closer to Your

Users 68 ms

Client Origin

SYN

10 ms

SYN/ACK

GET /image.jpg HTTP/1.1ACK,


CloudFront

Saves 68 ms

RTT

SYN

SYN/ACK

Time to first byte: 20 ms vs. 136 ms

Amazon CloudFront: 53 Global Locations

Edge Cache: Amazon CloudFront

Amazon S3

Elastic Load

Balancing

Mobile Clients

Browsers/Desktop

Clients

Tablets/Devices

Amazon CloudFront

Edge Location

Edge Cache

How do you decide what to cache?

• Static or Re-Usable Content

• Customized Content

• On-Demand and Live Video

• Dynamic or Unique Content

Cache Static or Re-Usable Content

HTTP/1.0 200 OK

Date: Mon, 19 Mar 201 12:51:28 GMT

Server: Apache

Last-Modified: Mon, 19 Mar 2012 07:15:25 GMT

Accept-Ranges: bytes

Content-Length: 1918

Cache-Control: max-age=86400

Content-Type: image/jpegVary: Accept-Encoding

Age: 16

X-Cache: Hit from cloudfront

Connection: keep-alive

Cache Customized Content

Customized Content

Cache Video Content

Caching Your Live Stream?

Source

HTTP

Cache

Edge

Paris

New York

Encoder

HLS, HDS,

Smooth Streaming

Hong Kong

Dynamic Content?

Zero TTL – cannot be cached! CloudFront can still help…

Dynamic Content?Zero TTL – cannot be cached!

CloudFront can still help…

TCP/IP optimizations for the network path

Keep-Alive Connections to reduce RTT

SSL Termination close to viewers

POST/PUT upload optimizations

Latency Based Routing

Low prices, same as static content delivery!

AZAZ

CloudFront

EU APUS

ELB

US

Browser Cache

Edge Cache

Web Cache

You can’t have dessert, until

you have your dinner!

-An Experiment.https://secure.flickr.com/photos/stephen-oung/6319155216/

Experiment Premise

• Start with basic infrastructure.– Single m1.large EC2 instance running Amazon Linux

– Single non-Multi-AZ m1.xlarge RDS MySQL 5.6 instance

– Apache httpd-2.2.25

– PHP 5.3.27

– Drupal Commerce Kickstart 7.2

– EIP on Instance

• Throw a ton of traffic at it.– 8,000,000 queries, 40 per second, from 4 other instances.

• Profile it.– New Relic PHP agent

– webpagetest.org

Experiment Infrastructure

Web/App

Instance

Elastic IPAmazon RDS

DB InstanceClients

Results? (not pretty)

14.7 second responses

Mostly Static Data

Low RPM

Web Cache

Webserver or proxy caches would live between

your CDN/Users and your web tier and can offer

up increased cost performance via reducing

internal application and database load. Can also

offer up increased edge to origin speed for lots of

content.

Web Cache

Popular solutions:– Varnish

– Nginx

– Apache with mod_cache/mod_proxy

– Squid

– Perlbal

– Language/framework caches (i.e., APC, Zend)

Web Cache

Availability Zone

VPC Subnet VPC Subnet

Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load

Balancing

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load

Balancing

Internet

Gateway

Amazon

Route 53

Amazon

CloudFront

Customer

Traffic

Web Cache• Opt for in-memory caching when possible.

• Pay attention to your cache hit/miss ratios. It could be a sign that you need to re-size the instances or re-size the number of nodes in your cache pool.

• Set smart TTLs so that you don’t affect new deploys or cache content for too long.

• Be smart about what cookies can burst cache and what cookies can’t. Don’t serve up other people’s content or stale dynamic pages.

Web Cache

How do you decide what to cache?– All logged out user pages

– Any completely static pages

– Traffic/log analysis• Look at your web logs/CDN logs

• Find heavily hit pages

• Figure out how often they actually change

• Apply a TTL to that page to be cached

– Even 60 second TTLs could help drastically!

Browser Cache

Edge Cache

Web Cache

Application Cache

Application Cache

Application level caches for information such

as session data, temporary application data

such as cart information, and live aggregation

of data feeds.

Application Cache

Popular solutions:– Memcached

– Redis

– Cassandra

– Amazon DynamoDB

Application Cache

Availability Zone

VPC Subnet VPC SubnetVPC Subnet

Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load

BalancingElastiCache

Cache Node

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load


Cache Node

Internet

Gateway

Amazon DynamoDB Session Handler for PHP

https://aws.amazon.com/sdkforphp/

Application Cache

Internet

Gateway

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load

Balancing Squid Proxy

Instances Amazon

DynamoDB

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load

Balancing Squid Proxy

Instances

Application Cache

Use Cases:– Session information

– Temporary data• Cart info, metadata

– Rate limiting• Fight abuse of APIs, spamming, functionality

abuse

– Counters• Views, Scores, Leader Boards

Application Cache

How do you decide what to cache?

– Ideally, you have to treat data you cache at this tier as loss tolerant if working with in-memory caches.

– Session information that wouldn’t make sense in cookies or in a true DB.

– Look at the kind of data your application is generating and storing in a DB that it might not need to.

Browser Cache

Edge Cache

Web Cache

Application Cache

Database Cache

Database Cache

Reduce workload on database servers by

caching commonly requested information, or

any information that might not change

frequently (i.e., user info, listing info, product

info).

Database Cache

Popular solutions:– In-engine query caches

– Memcached

• On dedicated host

• On DB host (built in w/ MySQL 5.6)

– Redis

• On dedicated host

Database Cache

A word of caution:In-engine DB caches are often not recommended for

many use cases, as they can significantly impact the

performance of many databases. Depending on the

workload and dataset you have, an in-engine query

cache might not be a good idea for you. We recommend

off DB caches where possible.

Database Cache

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load


Cache Node

Availability Zone


Elastic Load

BalancingCache

Instances

Web/App

Instances

Elastic Load


Cache Node

Internet

Gateway

RDS Instance

Primary (M-AZ)

RDS Instance

Standby (MAZ)

Memcached – Code Samplefunction retrieveValue($query)

{

$queryId = md5($query);

if ($myValue = $memcache->get($queryId))

{

return $myValue;

}

else

{

$myValue = dbfetch($query);

$memcache->set($queryId, $myValue);

return $myValue;

}

}

MySQL 5.6 + MemcachedRDS MySQL supports version

5.6 with integrated Memcached

on the instance:– Part of the InnoDB engine

– Memcached running as part of MySQL

talks directly to data in InnoDB tables,

essentially turning MySQL into a fast

“key-value store”

– From the opposite view point, adds

persistence to Memcached

– Same Memcached API as standalone

https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-intro.html

Cache everything

MongoDB DynamoDB Amazon RDS

MongoDB DynamoDB Amazon RDS

ElastiCache

MongoDB Queries

• Shut down 8 machines

Caching saves money

DynamoDB Reads

• Saved 3k reads per second

• (>20k reads per second in total)

Without negative caching

Cache really everything!

With negative caching

• Cache hit ratio 25-30% • Cache hit ratio 89-95%

Hit

Miss

AWS Marketplace & Partners Can Help

• Customers can find, research, buy

software.

• Simple pricing, aligns with Amazon

EC2 usage model.

• Launch in minutes!

• Marketplace billing integrated into your

AWS account.

• 1100+ products across 24+ categories.

Learn more at: aws.amazon.com/marketplace

Back To Our

Experiment!

https://secure.flickr.com/photos/toyochin/1382531438/

Experiment Infrastructure (original)

Web/App

Instance

Elastic IPAmazon RDS

DB InstanceClients

Results? (not pretty)

14.7 second responses

Mostly Static Data

Low RPM

Experiment Infrastructure (with cake)

• Added in ElastiCache Memcached

cache.m1.large

• Added in Amazon CloudFront for static content

– Tuned expires and cache headers in Apache

• Added in APC for PHP caching

– Increased memory to 128Mb, no other changes

• Did nothing to the DB

• Drupal memcached & CDN modules

CloudFront

Experiment Infrastructure (with cake)

Web/App

Instance

Elastic IP

Amazon RDS

DB Instance

Clients

ElastiCache

Cache Node

Results? (AWESOME!)

response time ~1/2

~2x RPM

Repeat Page view from 15.8sec to 2.75sec

Easy as Pie!

https://secure.flickr.com/photos/linecon0/2654865842/

Thank you!

Constantin Gonzalez

[email protected]

@zalez

aws pop-up loft berlin: cache is king - running lean architectures: optimizing for cost efficiency

Technology