extending your applications to the edge with cdns

Extending  Applica/ons  To  The  Edge  With  CDNs  

Hooman  Behesh/,  VP  Technology  

Webperf  meetup  NYC  –  May  12  2015  

What  is  a  CDN  

•  Global  network  of  HTTP  caches  – Get  content  closer  to  users  

•  Performance  benefits  •  offload  

In  the  past…  

Classic  CDNs  

•  Black  box  service  •  Making  changes  sucked!  •  LiRle  to  no  real-­‐/me  interfaces  – APIs  or  configura/on  management  

•  Lots  of  professional  services  •  Decent  caching,  terrible  uncaching  

Cacheable  

Uncacheable  Sta3c  Dynamic  

How  classic  CDNs  look  at  your  content  

Cache-Control: max-age=31536000

Classic  CDNs:  dynamic  content  

•  Focused  on  delivery:  transport  op/miza/on  

But  some3mes…  

I  know  you’re  calling  it  dynamic,  but  it’s  kind  of  sta3c  some3mes  

I  know  you’re  calling  it  dynamic,  but  it’s  kind  of  sta3c  some3mes  

That’s  cool,  how  long  is  it  sta3c  for?  

That’s  cool,  how  long  is  it  sta3c  for?  Some3mes  a  liFle  bit,  

some3mes  a  lot  

Some3mes  a  liFle  bit,  some3mes  a  lot   Neat!  

Some3mes  a  liFle  bit,  some3mes  a  lot   You  look  great  today!  

You  look  great  today!  Don’t  change  the  subject!    How  do  I  

cache  it?  

Don’t  change  the  subject!    How  do  I  

cache  it?  Cache  it  for  half  an  

hour?  

Cache  it  for  half  an  hour?  Some3mes  it  changes  

much  faster  

Some3mes  it  changes  much  faster  

Cache  it  for  10  minutes?  

Cache  it  for  10  minutes?  Are  you  just  guessing  

right  now?  

Are  you  just  guessing  right  now?  

Just  trying  to  figure  out  your  comfort  

level  with  mistakes!  

Expiry  guessing!  

Terrible  Caching  

Terrible  Caching  

Terrible  Anxiety  

or  

(ex:  news  story)  

Things  are  different  today  

Today…  

•  Influenced  by  clouds  •  Devops  requires  programmability  and  control  •  CDNs  should  be  part  of  the  applica/on  •  Real-­‐/me  integra/on  –  Changes  and  APIs  

•  Unpredictable  content  expiry  •  Your  content  à  you  need  control    

What  does  this  all  mean?  

Control  

Visibility  

Cacheability  

Control  

Visibility  

Cacheability  

Cacheability  

•  Granularity  in  caching  rules  (i.e.  cache  the  way  you  want)  –  Customizable  cache  keys  –  Ex:  geo-­‐ip  caching  –  Ex:  normalizing  user  agents  

•  Cache  everything  –  Expect  high  cache-­‐hit  ra/os  – No  cache-­‐control  anxiety!  

Cacheability  

•  Granularity  in  caching  rules  (i.e.  cache  the  way  you  want)  –  Customizable  cache  keys  –  Ex:  geo-­‐ip  caching  –  Ex:  normalizing  user  agents  

•  Cache  everything  –  Expect  high  cache-­‐hit  ra/os  – No  cache-­‐control  anxiety!  

Cache  hit  ra3os  

Cache  Hit  Ra3o:  Tradi3onal  calcula3on  

1  -­‐     Requests  to  Origin    Total  Requests  

Origin  

Origin  

Cache          

TCP  

Origin  

Cache          

HTTP  

Origin  

Cache          

Origin  

Cache          

HTTP  

Origin  

Cache          

HTTP  

Origin  

Cache          

HTTP  

Origin  

Cache          

HOT   COLD  

Origin  

Cache          

cache  “hit”  

Cache  Hit  Ra3o:  Tradi3onal  calcula3on  

1  -­‐     Requests  to  Origin    Total  Requests  

Isn’t  this  beFer?  

Hits  Total  Requests  

@edge  

Isn’t  this  beFer?  

Hits  Hits  +  Misses  

@edge  

Cache  Hit  Ra3o  

vs.  1  -­‐     Requests  to  Origin    Total  Requests  

Hits  

Hits  +  Misses  @edge  

Cache  Hit  Ra3o  

vs.  1  -­‐     Requests  to  Origin    Total  Requests  

Hits  

Hits  +  Misses  @edge  

Offload  

Cache  Hit  Ra3o  

vs.  1  -­‐     Requests  to  Origin    Total  Requests  

Hits  

Hits  +  Misses  @edge  

Offload   Performance  

Effect  on  long  tail  content  

Long  Tail:  Cacheable  but  seldom  fetched  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Connect  (median)  

Popular   14msec  

1hr  Tail   15msec  

6hr  Tail   16msec  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Connect  (median)  

Popular   14msec  

1hr  Tail   15msec  

6hr  Tail   16msec  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Connect  (median)   Wait  (median)  

Popular   14msec   19msec  

1hr  Tail   15msec   26msec  

6hr  Tail   16msec   32msec  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Connect  (median)   Wait  (median)  

Popular   14msec   19msec  

1hr  Tail   15msec   26msec  

6hr  Tail   16msec   32msec   6,400+  measurements  

77,000+  measurements  

38,000+  measurements  

Popular   Medium  Tail  (1hr)   Long  tail    (6hr)  

Isn’t  this  beFer?  

“Dynamic”  Content  

We’ve  been  told:  

Sta3c  =  Cacheable  

Dynamic  =  Uncacheable  

Cacheable  

Uncacheable  Sta3c  

Dynamic  

How  our  sites  used  to  look  

Cacheable  

Uncacheable  Sta3c  Dynamic  

How  we’re  told  our  sites  look  

Cacheable  

Uncacheable  Sta3c  

Dynamic  

How  our  sites  actually  look  

Event  Driven  

Event-­‐Driven  content:  

Sta3c  for  unpredictable  periods  of  3me    

Three  types  of  content  •  Sta/c  

–  Infrequent  changes,  can  cache  for  a  long  /me  –  Cache-­‐control  headers  are  good  enough  –  Examples:  Images,  javascript  ,  css,  etc  

•  Dynamic  –  Can’t  cache  at  all    -­‐  must  go  to  origin  –  DSA  and  transport  is  key  for  performance  –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc  

•  Event-­‐driven  –  Sta/c,  but  unpredictably  –  Cache-­‐control  headers  are  not  good  enough  –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  

Three  types  of  content  •  Sta/c  

–  Infrequent  changes,  can  cache  for  a  long  /me  –  Cache-­‐control  headers  are  good  enough  –  Examples:  Images,  javascript  ,  css,  etc  

•  Dynamic  –  Can’t  cache  at  all    -­‐  must  go  to  origin  –  DSA  and  transport  is  key  for  performance  –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc  

•  Event-­‐driven  –  Sta/c,  but  unpredictably  –  Cache-­‐control  headers  are  not  good  enough  –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  

Three  types  of  content  •  Sta/c  

–  Infrequent  changes,  can  cache  for  a  long  /me  –  Cache-­‐control  headers  are  good  enough  –  Examples:  Images,  javascript  ,  css,  etc  

•  Dynamic  –  Can’t  cache  at  all    -­‐  must  go  to  origin  –  DSA  and  transport  is  key  for  performance  –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc  

•  Event-­‐driven  –  Sta/c,  but  unpredictably  –  Cache-­‐control  headers  are  not  good  enough  –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  

Caching  event-­‐driven  content  

•  Not  easy  – Classic  CDNs  con/nue  to  consider  it  the  same  as  dynamic  (Bad  caching  or  panic!)  

– Need  the  right  tools  to  cache  •  You  may  not  know  expiry  ahead  of  /me,  but  your  app  knows  when  things  change  

Cache  normally  and  then    

 “uncache”  when  it  changes  

Uncaching!  

•  Purging  – Accessible  through  API  – Granular  •  Dependencies  through  labels,  tags,  or  surrogate  keys  

– Must  be  instantaneous  •  Ex:  news  story,  part  2  

Instant  Purge  

•  Enables  caching  of  event-­‐driven  content  –  Can  deal  with  unpredictable  invalida/on  –  As  pages  update/publish,  purge  command  is  sent  to  CDN  

•  Instant?  –  Instant  ≠    12  minutes!  –  Instant  ≠    quick  acknowledgement!  –  <1sec  –  Determinis/c  and  predictable  

Example:  CMS  +  Purge  

WordPress:  Before  

CDN  Node  

WordPress:  Before  

CDN  Node  

WordPress:  Before  

CDN  Node  

WordPress:  Before  

CDN  Node  

WordPress:  Before  

CDN  Node  

Cache  

WordPress:  Ajer  

CDN  Node  

WordPress:  Ajer  

CDN  Node  

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Ajer  

CDN  Node  

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Ajer  

CDN  Node  

WordPress:  Ajer  

CDN  Node  

WordPress:  Ajer  

CDN  Node  PURGE  

WordPress:  Ajer  

CDN  Node  PURGE  

WordPress:  Ajer  

CDN  Node  

(Has  to  be  instantaneous!)  

PURGE  

WordPress:  Ajer  

CDN  Node  

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress:  Ajer  

CDN  Node  

Example:  customer1.js  

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

(Referer: www.customer1.com)

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

Before  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Before  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Before  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

(Ajer  1  min)  

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js VALIDATION

If-Modified-Since: Tue, 12 May 2015 19:51:30 GMT

(Ajer  1  min)  

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

304 Not Modified

(Ajer  1  min)  

Before  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

304 Not Modified

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

(Ajer  1  min)  

This  happens  many  many  3mes!  (many  many  happy  visitors!)  

Ajer  

CDN  Node  

Origin  

Customer1  changes  config  

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  PURGE  customer1.js  

Customer1  changes  config  

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  PURGE  customer1.js  

Customer1  changes  config  

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  http://www.3rdparty.com/customer1.js

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

(Ajer  1  min)  

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

(Ajer  1  min)  

Control  

Visibility  

Cacheability  

Programmability  

•  Granular  Invalida/on  API  •  Configura/on  API  •  Logic  at  the  Edge  •  All  must  be  instantaneous  and  real  /me  

Control  at  the  Edge  

•  Moving  app  logic  to  the  edge  •  Ex:  VCL  (Varnish  Configura/on  Language)  – Script-­‐like  configura/on  for  func/onality  at  the  edge  

•  Any  mechanism  offered  by  CDN  to  let  you  do  stuff  at  the  edge  is  good  

Logic  at  the  Edge:  Examples    •  Generate  content  at  the  edge  •  HTTP  header  manipula/on  •  Origin  selec/on  •  Caching  rules  •  Geo-­‐IP  rules  •  Forcing  SSL  •  Serving  stale  content    

–  stale-­‐while-­‐revalidate,  stale-­‐while-­‐error  •  Etc…  

Example:  Edge-­‐generated  Content  

JSON  Data  Center  ID  

CDN  Node  

Origin  http://www.site.com/which_datacenter.js

JSON  Data  Center  ID  

CDN  Node  

Origin  

{ ‘datacenter’ : ‘LAX’ }

http://www.site.com/which_datacenter.js

VCL  Snippet  

JSONP  Geo  IP  

CDN  Node  

Origin  

myFunction({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});

http://www.site.com/geo_ip.js?jsonp=myFunction

JSONP  Geo  IP  

CDN  Node  

Origin  

myFunction({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});

http://www.site.com/geo_ip.js?jsonp=myFunction

JSONP  Geo  IP  

CDN  Node  

Origin  

somethingElse({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});

http://www.site.com/geo_ip.js?jsonp=somethingElse

Control  

Visibility  

Cacheability  

Real-­‐3me  Analy3cs  

•  Stats  API  •  Network  stats  •  HTTP  stats  (status  codes,  etc)  •  Caching  stats  (hits,  misses,  errors,  etc)  •  All  have  to  be  real-­‐/me  – Historic  data  is  also  a  must  for  long  term  stats  analysis  

Logging  

•  Real  /me!  •  Log  streaming  – To  any  logging  des/na/on  endpoint  – Ex:  S3,  FTP,  syslog,  etc  

Example:  Beacon  Termina3on  at  the  Edge  

Before  

CDN  Node  

Origin  

Log  Analysis  

http://collector.site.com/beacon.img?a=1&b=2&c=3

Before  

CDN  Node  

Origin  

Log  Analysis  

http://collector.site.com/beacon.img?a=1&b=2&c=3

Before  

CDN  Node  

Origin  

Log  Analysis  

HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Before  

CDN  Node  

Origin  

Log  Analysis  

HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Ajer  

CDN  Node  

Origin  http://collector.site.com/beacon.img?a=1&b=2&c=3

Ajer  

CDN  Node  

Origin  

HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35

http://collector.site.com/beacon.img?a=1&b=2&c=3

Ajer  

CDN  Node  

Origin  

HTTP/1.1 204 No ContentDate: Sat, 21 Jun 2014 23:21:12 GMTServer: Awesome ServerContent-Length: 0

http://collector.site.com/beacon.img?a=1&b=2&c=3

Ajer  

CDN  Node  

Origin  

Syslog  /  S3  /  FTP/etc  

http://collector.site.com/beacon.img?a=1&b=2&c=3

Summary  

•  CDNs  should  be  an  extension  of  your  applica/ons:  –  Flexible  caching,  predictable  edge  caching,  and  instant  purging  to  un-­‐cache!  

–  Control  at  the  edge  –  Real-­‐/me  visibility  and  analy/cs  

•  Full  real-­‐/me  programma/c  interface  is  a  must    

Thank  you!  

hooman@fastly.com