aws pop-up loft berlin: cache is king - running lean architectures: optimizing for cost efficiency
Post on 14-Jan-2017
1.105 Views
Preview:
TRANSCRIPT
Cache is KingScale Your Application whileImproving Performance and Lowering Costs
Constantin Gonzalez
Principal Solutions Architect
Amazon Web Services Germany GmbH
glez@amazon.de | @zalez
Cash is KingCash is King
Cache is Cash
Httparchive
Content Breakdown
87% of Your Site Consists Of Static/Re-usable Content
52% of The Internet’s Content is
NOT Cached Today
But Wait…
Cache is King
Source: httparchive.org300K Tracked Websites Oct 2014-Oct 2015
~52%{
CACHE EVERYTHING
Browser Cache
Edge Cache
Web Cache
Application Cache
Database Cache
Image Source: cakeormistake.wordpress.com
Speed of Light ~300,000 Km/s
Image Source: wordlesstech.com
~200,000 Km/s through fiber
Image Source: www2.wiringandcabling.com
Download Performance
Round Trip: 13,026 km
Download Performance
Round Trip: 13,026 km
Theoretical: 65 ms
Download Performance
Round Trip: 13,026 km
Theoretical: 65 ms
Measured: 68 ms
Time to First Byte68 ms
Client Server
Time to First Byte68 ms
68 ms
Client Server
SYN
SYN/ACK
Time to First Byte68 ms
68 ms
68 ms
Client Server136 ms
SYN
SYN/ACK
ACK,
GET /image.jpg HTTP/1.1
Browser Cache
Browser Caching
Set max-age or expiry date in the headers.
HTML5 Application Cache.
Helps eliminate network latency.
But… Browser cache size is limited.(e.g. IE is 8-50M, Chrome is < 80M, Firefox is 50MB, etc.)
Browser Cache
Edge Cache
Time to First Byte68 ms
68 ms
68 ms
Client Server136 ms
SYN
SYN/ACK
ACK,
GET /image.jpg HTTP/1.1
Bring the Bytes Closer to Your
Users 68 ms
Client Origin
SYN
10 ms
SYN/ACK
GET /image.jpg HTTP/1.1ACK,
GET /image.jpg HTTP/1.1
CloudFront
Saves 68 ms
RTT
SYN
SYN/ACK
Time to first byte: 20 ms vs. 136 ms
Amazon CloudFront: 53 Global Locations
Edge Cache: Amazon CloudFront
Amazon S3
Elastic Load
Balancing
Mobile Clients
Browsers/Desktop
Clients
Tablets/Devices
Amazon CloudFront
Edge Location
Edge Cache
How do you decide what to cache?
• Static or Re-Usable Content
• Customized Content
• On-Demand and Live Video
• Dynamic or Unique Content
Cache Static or Re-Usable Content
HTTP/1.0 200 OK
Date: Mon, 19 Mar 201 12:51:28 GMT
Server: Apache
Last-Modified: Mon, 19 Mar 2012 07:15:25 GMT
Accept-Ranges: bytes
Content-Length: 1918
Cache-Control: max-age=86400
Content-Type: image/jpegVary: Accept-Encoding
Age: 16
X-Cache: Hit from cloudfront
Connection: keep-alive
Cache Customized Content
Customized Content
Cache Video Content
Caching Your Live Stream?
Source
HTTP
Cache
Edge
Paris
New York
Encoder
HLS, HDS,
Smooth Streaming
Hong Kong
Dynamic Content?
Zero TTL – cannot be cached! CloudFront can still help…
Dynamic Content?Zero TTL – cannot be cached!
CloudFront can still help…
TCP/IP optimizations for the network path
Keep-Alive Connections to reduce RTT
SSL Termination close to viewers
POST/PUT upload optimizations
Latency Based Routing
Low prices, same as static content delivery!
AZAZ
CloudFront
EU APUS
ELB
US
Browser Cache
Edge Cache
Web Cache
You can’t have dessert, until
you have your dinner!
-An Experiment.https://secure.flickr.com/photos/stephen-oung/6319155216/
Experiment Premise
• Start with basic infrastructure.– Single m1.large EC2 instance running Amazon Linux
– Single non-Multi-AZ m1.xlarge RDS MySQL 5.6 instance
– Apache httpd-2.2.25
– PHP 5.3.27
– Drupal Commerce Kickstart 7.2
– EIP on Instance
• Throw a ton of traffic at it.– 8,000,000 queries, 40 per second, from 4 other instances.
• Profile it.– New Relic PHP agent
– webpagetest.org
Experiment Infrastructure
Web/App
Instance
Elastic IPAmazon RDS
DB InstanceClients
Results? (not pretty)
14.7 second responses
Mostly Static Data
Low RPM
Web Cache
Webserver or proxy caches would live between
your CDN/Users and your web tier and can offer
up increased cost performance via reducing
internal application and database load. Can also
offer up increased edge to origin speed for lots of
content.
Web Cache
Popular solutions:– Varnish
– Nginx
– Apache with mod_cache/mod_proxy
– Squid
– Perlbal
– Language/framework caches (i.e., APC, Zend)
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing
Internet
Gateway
Amazon
Route 53
Amazon
CloudFront
Customer
Traffic
Web Cache• Opt for in-memory caching when possible.
• Pay attention to your cache hit/miss ratios. It could be a sign that you need to re-size the instances or re-size the number of nodes in your cache pool.
• Set smart TTLs so that you don’t affect new deploys or cache content for too long.
• Be smart about what cookies can burst cache and what cookies can’t. Don’t serve up other people’s content or stale dynamic pages.
Web Cache
How do you decide what to cache?– All logged out user pages
– Any completely static pages
– Traffic/log analysis• Look at your web logs/CDN logs
• Find heavily hit pages
• Figure out how often they actually change
• Apply a TTL to that page to be cached
– Even 60 second TTLs could help drastically!
Browser Cache
Edge Cache
Web Cache
Application Cache
Application Cache
Application level caches for information such
as session data, temporary application data
such as cart information, and live aggregation
of data feeds.
Application Cache
Popular solutions:– Memcached
– Redis
– Cassandra
– Amazon DynamoDB
Application Cache
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Internet
Gateway
Application Cache
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Internet
Gateway
Amazon DynamoDB Session Handler for PHP
https://aws.amazon.com/sdkforphp/
Application Cache
Internet
Gateway
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing Squid Proxy
Instances Amazon
DynamoDB
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing Squid Proxy
Instances
Application Cache
Internet
Gateway
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing Squid Proxy
Instances Amazon
DynamoDB
Availability Zone
VPC Subnet VPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
Balancing Squid Proxy
Instances
Application Cache
Use Cases:– Session information
– Temporary data• Cart info, metadata
– Rate limiting• Fight abuse of APIs, spamming, functionality
abuse
– Counters• Views, Scores, Leader Boards
Application Cache
How do you decide what to cache?
– Ideally, you have to treat data you cache at this tier as loss tolerant if working with in-memory caches.
– Session information that wouldn’t make sense in cookies or in a true DB.
– Look at the kind of data your application is generating and storing in a DB that it might not need to.
Browser Cache
Edge Cache
Web Cache
Application Cache
Database Cache
Database Cache
Reduce workload on database servers by
caching commonly requested information, or
any information that might not change
frequently (i.e., user info, listing info, product
info).
Database Cache
Popular solutions:– In-engine query caches
– Memcached
• On dedicated host
• On DB host (built in w/ MySQL 5.6)
– Redis
• On dedicated host
Database Cache
A word of caution:In-engine DB caches are often not recommended for
many use cases, as they can significantly impact the
performance of many databases. Depending on the
workload and dataset you have, an in-engine query
cache might not be a good idea for you. We recommend
off DB caches where possible.
Database Cache
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Internet
Gateway
RDS Instance
Primary (M-AZ)
RDS Instance
Standby (MAZ)
Database Cache
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Availability Zone
VPC Subnet VPC SubnetVPC Subnet
Elastic Load
BalancingCache
Instances
Web/App
Instances
Elastic Load
BalancingElastiCache
Cache Node
Internet
Gateway
RDS Instance
Primary (M-AZ)
RDS Instance
Standby (MAZ)
Memcached – Code Samplefunction retrieveValue($query)
{
$queryId = md5($query);
if ($myValue = $memcache->get($queryId))
{
return $myValue;
}
else
{
$myValue = dbfetch($query);
$memcache->set($queryId, $myValue);
return $myValue;
}
}
MySQL 5.6 + MemcachedRDS MySQL supports version
5.6 with integrated Memcached
on the instance:– Part of the InnoDB engine
– Memcached running as part of MySQL
talks directly to data in InnoDB tables,
essentially turning MySQL into a fast
“key-value store”
– From the opposite view point, adds
persistence to Memcached
– Same Memcached API as standalone
https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-intro.html
Cache everything
MongoDB DynamoDB Amazon RDS
MongoDB DynamoDB Amazon RDS
ElastiCache
MongoDB Queries
• Shut down 8 machines
Caching saves money
DynamoDB Reads
• Saved 3k reads per second
• (>20k reads per second in total)
Without negative caching
Cache really everything!
With negative caching
• Cache hit ratio 25-30% • Cache hit ratio 89-95%
Hit
Miss
AWS Marketplace & Partners Can Help
• Customers can find, research, buy
software.
• Simple pricing, aligns with Amazon
EC2 usage model.
• Launch in minutes!
• Marketplace billing integrated into your
AWS account.
• 1100+ products across 24+ categories.
Learn more at: aws.amazon.com/marketplace
Back To Our
Experiment!
https://secure.flickr.com/photos/toyochin/1382531438/
Experiment Infrastructure (original)
Web/App
Instance
Elastic IPAmazon RDS
DB InstanceClients
Results? (not pretty)
14.7 second responses
Mostly Static Data
Low RPM
Experiment Infrastructure (with cake)
• Added in ElastiCache Memcached
cache.m1.large
• Added in Amazon CloudFront for static content
– Tuned expires and cache headers in Apache
• Added in APC for PHP caching
– Increased memory to 128Mb, no other changes
• Did nothing to the DB
• Drupal memcached & CDN modules
CloudFront
Experiment Infrastructure (with cake)
Web/App
Instance
Elastic IP
Amazon RDS
DB Instance
Clients
ElastiCache
Cache Node
Results? (AWESOME!)
response time ~1/2
~2x RPM
Repeat Page view from 15.8sec to 2.75sec
Easy as Pie!
https://secure.flickr.com/photos/linecon0/2654865842/
Thank you!
Constantin Gonzalez
glez@amazon.de
@zalez
top related