caching on the edge
DESCRIPTION
TRANSCRIPT
Speaker | Company
Title // Presentation Name
Caching on the Edge Fabien Potencier
Who are you?
May 1996
RFC1945 – HTTP/1.0
March 1999
RFC2616 – HTTP/1.1
http://www.ietf.org/rfc/rfc2616.txt
HTTP Bis – Work in progress
http://tools.ietf.org/wg/httpbis/
http://www.flickr.com/photos/visualpanic/233508614
p1: Messaging: Low-level message parsing and connection management p2: Semantics: Methods, status codes and headers p3: Payload: Dealing with content, message-specific headers p4: Conditional Requests: e.g., If-Modified-Since p5: Range Requests: Getting partial content p6: Caching: Browser and intermediary caches p7: Authentication: HTTP authentication framework
HTTP
The Client sends a Request to the Server The Server sends back a Response to the Client
A Client (browser, bot, WS, curl, …) A Server (Apache, nginx, …)
The Request and the Response are HTTP messages
Let’s play with HTTP and HTTP headers
GET / HTTP/1.1 Host: http.trainings.sensiolabs.com
« The request-header fields allow the client to pass additional information about the request, and about the client itself, to the
server. These fields act as request modifiers, with semantics equivalent to the parameters on a programming language method
invocation. »
« Each HTTP header field consists of a case-insensitive field name followed by a colon (":"), optional whitespace, and the field value »
HTTP/1.1 200 OK Date: Wed, 15 Oct 2005 07:07:07 GMT Server: Apache Content-Length: 14 Content-Type: text/html
Hello World!
Live HTTP headers Firefox extension
http://livehttpheaders.mozdev.org/
$ curl -i http://http.trainings.sensiolabs.com/
HTTP/1.1 200 OK Date: Wed, 15 Oct 2005 07:07:07 GMT Server: Apache Content-Length: 14 Content-Type: text/html
Hello World!
$ telnet localhost 80 Trying ::1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 Host: http.trainings.sensiolabs.com
HTTP/1.1 200 OK Date: Wed, 15 Oct 2005 07:07:07 GMT Server: Apache Content-Length: 14 Content-Type: text/html
Hello World!
Connection closed by foreign host.
HTTP Headers with PHP
header('Content-Type: text/plain');
header('content-type: text/plain');
Caching in the HTTP Specification
p1: Messaging: Low-level message parsing and connection management p2: Semantics: Methods, status codes and headers p3: Payload: Dealing with content, message-specific headers p4: Conditional Requests: e.g., If-Modified-Since p5: Range Requests: Getting partial content p6: Caching: Browser and intermediary caches p7: Authentication: HTTP authentication framework
HTTP Expiration HTTP Validation
Fresh vs Stale
HTTP Headers for Expiration
Cache-Control
Expires
HTTP Headers for Validation
Last-Modified / If-Modified-Since
ETag / If-None-Match
HTTP Cache headers only work with “safe” HTTP methods
(like GET & HEAD)
Never change the state of the server when serving a GET request
(we are talking about the application’s state of course, you can log, cache, …)
HTTP Expiration
Expires
« The "Expires" header field gives the date/time after which
the response is considered stale. »
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
GET /foo HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
Before expira4on
Your applica4on is not called
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
GET /foo HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
HTTP/1.1 200 OK Expires: Thu, 01 Dec …
Hello
A<er expira4on
Not fresh
Expires: Thu, 01 Dec 2010 16:00:00 GMT
Date in RFC1123 format, not RFC2822 (timezone always GMT, which is ~ UTC)
$expires = gmdate('D, j M Y H:i:s T', time() + 5);
header('Expires: '.$expires);
$date = new DateTime(null, new DateTimeZone('UTC')); $date->modify('+5 seconds');
$expires = $date->format('D, d M Y H:i:s').' GMT';
$date = gmdate('D, j M Y H:i:s T', time() + 5);
header('Expires: '.$expires);
$date->setTimezone(new DateTimeZone('UTC'));
/expires_with_expires_in_5s.php
WARNING
The clocks on the Web server AND the cache (aka browser)
MUST be synchronised
WARNING
« HTTP/1.1 servers SHOULD NOT send Expires dates
more than one year in the future. »
Mostly useful to make static assets (images, css, js, …) cacheable with an extremely long expiry time
But there is a better way!
Use Cache-Control instead
header('Cache-Control: max-age=5');
http.trainings.sensiolabs.com/expires_in_5s.php
Don’t use Expires Use Cache-Control
…except if you want to set a date, which should be pretty rare
HTTP Validation
“304 Not Modified” is your friend
HTTP Headers for Validation
Etag / If-None-Match
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK ETag: abcdef …
Hello
HTTP/1.1 200 OK ETag: abcdef
Hello
HTTP/1.1 200 OK ETag: abcdef
Hello
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-None-Match: abcdef
HTTP/1.1 304 Not Modified
GET /foo HTTP/1.1 200 OK ETag: abcdef …
Hello
HTTP/1.1 200 OK ETag: abcdef
Hello
If the resource has not changed
Some
Cach
e
GET /foo HTTP/1.1 200 OK ETag: abcdef …
Hello
Brow
ser
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-None-Match: abcdef
If the resource has changed
GET /foo HTTP/1.1 200 OK ETag: 123456 …
Hello
HTTP/1.1 200 OK ETag: 123456
Hello
HTTP/1.1 200 OK ETag: 123456
Hello
// compute ETag value $etag = '...';
if ( isset($_SERVER['HTTP_IF_NONE_MATCH']) && $_SERVER['HTTP_IF_NONE_MATCH'] == $etag ) { header('HTTP/1.1 304 Not Modified'); } else { header('ETag: '.$etag); echo 'Hello'; }
This is a simple but naïve implementation…
HTTP Headers for Validation
Last-Modified / If-Modified-Since
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK Last-Modified: Thu, …
Hello
HTTP/1.1 200 OK Last-Modified: Thu, …
Hello
HTTP/1.1 200 OK Last-Modified: Thu, …
Hello
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-Modified-Since: Thu,
HTTP/1.1 304 Not Modified
GET /foo HTTP/1.1 200 OK Last-Modified: Thu, …
Hello
HTTP/1.1 200 OK Last-Modified: Thu, …
Hello
If the resource has not changed
Some
Cach
e
GET /foo HTTP/1.1 200 OK Last-Modified: Thu,…
Hello
GET /foo HTTP/1.1 200 OK Last-Modified: Sun,…
Hello
Brow
ser
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-Modified-Since: Thu,
If the resource has changed
HTTP/1.1 200 OK Last-Modified: Sun, …
Hello
HTTP/1.1 200 OK Last-Modified: Sun, …
Hello
Expiration & Validation
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK Etag: abcdef Cache-‐Control: max-‐age=10 …
Hello
HTTP/1.1 200 OK ETag: abcdef Cache-‐Control: max-‐age=10
Hello
HTTP/1.1 200 OK ETag: abcdef Cache-‐Control: max-‐age=10
Hello
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
Before expira4on
Your applica4on is not called
GET /foo HTTP/1.1 200 OK Etag: abcdef Cache-‐Control: max-‐age=10 …
Hello
HTTP/1.1 200 OK ETag: abcdef Cache-‐Control: max-‐age=10
Hello
Brow
ser
Your PHP application
Some
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-None-Match: abcdef
HTTP/1.1 304 Not Modified Cache-‐Control: max-‐age=10
A<er expira4on but resource s4ll valid
GET /foo HTTP/1.1 200 OK Etag: abcdef Cache-‐Control: max-‐age=10 …
Hello
HTTP/1.1 200 OK ETag: abcdef Cache-‐Control: max-‐age=10
Hello
You can combine HTTP headers the way you want
Expiration wins over Validation
Expiration allows you to scale as less requests hit your server (and client speed is better too)
Validation saves bandwidth
The goal is to never generate the same response twice
PHP and Cache HTTP headers
session_start(); $_SESSION['foo'] = 'bar';
echo 'Hello';
http.trainings.sensiolabs.com/index.php
http.trainings.sensiolabs.com/cookie.php
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Kinds of caches
Browser Cache
On the server side
Brow
ser
Your PHP application
Brow
ser C
ache
With
in a Com
pany
On the server side
Brow
ser
Your PHP application Br
owse
r Br
owse
r
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Proxy Cache
On the server side
With
in a Com
pany
Brow
ser
Your PHP application Br
owse
r Br
owse
r
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Prox
y Cac
he
On the server side
With
in a Com
pany
Brow
ser
Your PHP application Br
owse
r Br
owse
r
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Prox
y Cac
he
With
in a Com
pany
Brow
ser
Brow
ser
Brow
ser
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Prox
y Cac
he
Gateway Cache
On the server side
Your PHP application Br
owse
r
Brow
ser C
ache
Gatew
ay Ca
che
With
in a Com
pany
On the server side
Brow
ser
Your PHP application Br
owse
r Br
owse
r
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Gatew
ay Ca
che
On the server side W
ithin a Com
pany
Brow
ser
Your PHP application Br
owse
r Br
owse
r
Brow
ser C
ache
Br
owse
r Cac
he
Brow
ser C
ache
Prox
y Cac
he
Brow
ser
Brow
ser
Brow
ser C
ache
Br
owse
r Cac
he
Gatew
ay Ca
che
Your PHP application
Gateway Cache Reverse Proxy Cache
Surrogate Cache HTTP Accelerator
Browser
Browser Cache
Proxy Cache
Browser Cache
Local cache for when you hit “back” or
when images are reused throughout a website
Proxy Cache
A shared cache Many people behind a single proxy
Installed by large corporations and ISPs Reduce latency and network traffic
Gateway Cache
A shared cache on the server side Installed by network administrators
Make websites more scalable, reliable and performing better
CDNs like Akaïma are gateway caches
Today, we will mainly talk about Gateway caches
Edge Caching
App Browser
Browser
Gateway Cache
Gateway Cache
HTTP 1.1 allows caching anything by default unless explicit Cache-Control header
In practice, most caches avoid anything with
Cache-Control Cookie / Set-Cookie
WWW-Authenticate / Authorization POST / PUT
302 / 307 status codes
Cache-Control: private
This is the default with PHP when you have a session Browser cache will still work fine here
(public means shared caches, private means browser cache)
A gateway cache won't cache anything "private" or carrying a cookie
In a real-world: tracking cookies (Google Analytics)
All proxies do caching based on the same HTTP headers
We need several clients to understand how it works and how the cached is shared
HTTP Expiration
Gatew
ay Ca
che
Bob
Your PHP application
Bob’s
Cach
e
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
HTTP/1.1 200 OK C-C: max-age=600
Hello
HTTP/1.1 200 OK C-C: max-age=600
Hello
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
HTTP/1.1 200 OK C-C: max-age=600
Hello
Bob’s
Cach
e
Gatew
ay Ca
che
Bob
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
HTTP/1.1 200 OK C-C: max-age=600
Hello
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
Brow
ser C
ache
Alice
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
HTTP/1.1 200 OK C-C: max-age=600
Hello
HTTP/1.1 200 OK C-C: max-age=600
Hello
Gatew
ay Ca
che
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
GET /foo HTTP/1.1 200 OK C-C: max-age=600 …
Hello
HTTP Validation
Bob’s
Cach
e
Gatew
ay Ca
che
Bob
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 200 OK Etag: abcde …
Hello
HTTP/1.1 200 OK Etag: abcde
Hello
HTTP/1.1 200 OK Etag: abcde
Hello
GET /foo HTTP/1.1 200 OK Etag: abcde …
Hello
HTTP/1.1 200 OK Etag: abcde
Hello
Bob’s
Cach
e
Gatew
ay Ca
che
Bob
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-None-Match: ab
GET /foo HTTP/1.1 Host: foo.org If-None-Match: ab
GET /foo HTTP/1.1 200 OK Etag: ab …
Hello
HTTP/1.1 200 OK Etag: ab
Hello
GET /foo HTTP/1.1 200 OK Etag: ab …
Hello
304 Not Modified 304 Not Modified
Gatew
ay Ca
che
Your PHP application
Brow
ser C
ache
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org If-None-Match: ab
GET /foo HTTP/1.1 200 OK Etag: ab …
Hello
HTTP/1.1 200 OK Etag: ab
Hello
304 Not Modified
Alice
HTTP/1.1 200 OK Etag: ab
Hello
GET /foo HTTP/1.1 200 OK Etag: ab …
Hello
Gateway caches
Varnish (only does that and tries to do it well) Squid (just one way to use it)
mod_cache (Apache)
Gateway caches are a great way to make your website performs better
But what if you cannot cache whole pages? What if a page has "more" dynamic parts?
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus, in pulvinar lectus rutrum sit amet. Duis gravida, metus in dictum eleifend, dolor risus 4ncidunt ligula, non volutpat nulla sapien in elit. Nulla rutrum erat id neque suscipit eu ultricies odio sollicitudin. Aliquam a mi vel eros placerat hendrerit. Phasellus porRtor, augue sit amet vulputate venena4s, dui leo commodo odio, a euismod turpis ligula in elit.
cacheable for 10 seconds cacheable for 5 seconds
ESI… or Edge Side Includes
http://www.w3.org/TR/esi-lang
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus, in pulvinar lectus rutrum sit amet. Duis gravida, metus in dictum eleifend, dolor risus 4ncidunt ligula, non volutpat nulla sapien in elit. Nulla rutrum erat id neque suscipit eu ultricies odio sollicitudin. Aliquam a mi vel eros placerat hendrerit. Phasellus porRtor, augue sit amet vulputate venena4s, dui leo commodo odio, a euismod turpis ligula in elit.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus, in pulvinar lectus rutrum sit amet. Duis gravida, metus in dictum eleifend, dolor risus 4ncidunt ligula, non volutpat nulla sapien in elit. Nulla rutrum erat id neque suscipit eu ultricies odio sollicitudin. Aliquam a mi vel eros placerat hendrerit. Phasellus porRtor, augue sit amet vulputate venena4s, dui leo commodo odio, a euismod turpis ligula in elit.
<esi:include src="..." />
<esi:include src="http://..." />
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /bar HTTP/1.1 Host: foo.org
Lorem ipsum dolor
<esi:include src="hYp.." />
HTTP/1.1 200 OK
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /bar HTTP/1.1 Host: foo.org
Lorem ipsum dolor
<esi:include src="hYp.." />
HTTP/1.1 200 OK C-C: max-age=10
Lorem ipsum dolor
HTTP/1.1 200 OK C-C: max-age=5
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: max-age=10
Lor <esi:include />
GET /bar C-C: max-age=5
Lorem
GET /foo C-C: max-age=10
Lor Lorem
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: max-age=10
Lor <esi:include />
GET /bar C-C: max-age=5
Lorem
GET /foo C-C: max-age=10
Lor Lorem 2 seconds later…
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: max-age=10
Lor <esi:include />
GET /bar C-C: max-age=5
Lorem
GET /foo C-C: max-age=10
Lor Lorem 7 seconds later…
WRONG
Cache-Control: max-age=10
Cache-Control: s-maxage=10
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /bar HTTP/1.1 Host: foo.org
Lorem ipsum dolor
<esi:include src="hYp.." />
HTTP/1.1 200 OK C-C: s-maxage=10
Lorem ipsum dolor
HTTP/1.1 200 OK C-C: s-maxage=5
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: s-maxage=10
Lor <esi:include />
GET /bar C-C: s-maxage=5
Lorem
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: s-maxage=10
Lor <esi:include />
GET /bar C-C: s-maxage=5
Lorem
2 seconds later…
Brow
ser C
ache
Gatew
ay Ca
che
Brow
ser
Your PHP application GET /foo HTTP/1.1 Host: foo.org
GET /foo HTTP/1.1 Host: foo.org
GET /bar HTTP/1.1 Host: foo.org
Lorem ipsum dolor
HTTP/1.1 200 OK C-C: s-maxage=5
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
Lorem ipsum dolor sit amet,
Lorem ipsum dolor
HTTP/1.1 200 OK
GET /foo C-C: s-maxage=10
Lor <esi:include />
GET /bar C-C: s-maxage=5
Lorem
7 seconds later…
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam. Mauris dignissim, urna quis iaculis tempus, justo libero porRtor est, nec eleifend est elit vitae ante. Curabitur interdum luctus metus, in pulvinar lectus rutrum sit amet. Duis gravida, metus in dictum eleifend, dolor risus 4ncidunt ligula, non volutpat nulla sapien in elit. Nulla rutrum erat id neque suscipit eu ultricies odio sollicitudin. Aliquam a mi vel eros placerat hendrerit. Phasellus porRtor, augue sit amet vulputate venena4s, dui leo commodo odio, a euismod turpis ligula in elit.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc. Integer semper turpis et enim porRtor iaculis. Nulla facilisi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris vehicula ves4bulum dictum. Aenean non velit tortor. Nullam adipiscing malesuada aliquam.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In vel nulla arcu, vitae cursus nunc.
main template
layout
base layout
included page included page
included page
sub vcl_fetch { esi; }
varnishlog -i TxURL
varnishlog –o VCL_call hit | grep RxURL
Make your application work when ESI is enabled and when it’s not
And only parse for ESIs when it’s needed
Surrogate-Capability
Surrogate-Control
http://www.w3.org/TR/edge-arch
Surrogate-Capability: abc="Surrogate/1.0 ESI/1.0"
Surrogate-Control: content="ESI/1.0"
RewriteEngine On RewriteRule ^(.*)$ index.php [QSA,L]
sub vcl_recv { set req.http.Surrogate-Capability = "abc=ESI/1.0"; }
sub vcl_fetch { if (beresp.http.Surrogate-Control ~ "ESI/1.0") { unset beresp.http.Surrogate-Control; esi; } }
function has_surrogate_ESI_capability() { return isset($_SERVER['HTTP_SURROGATE_CAPABILITY']) && preg_match('#ESI/1.0#', $_SERVER['HTTP_SURROGATE_CAPABILITY']); }
function add_surrogate_control() { header('Surrogate-Control: content="ESI/1.0"'); }
function include($path) { if (has_surrogate_ESI_capability()) { $path = get_absolute_path($path);
echo '<esi:include src="'.$path.'" />';
add_surrogate_control(); } else { include dirname(__FILE__).'/'.$path; } }
Imagine the power when you combine ESI, expiration, validation, max-age, s-maxage, …
Why would you want to reinvent the wheel? and implement your own caching system?
Because you cannot afford to use Varnish? Because you use a shared hosting company?
<?php
// With the Symfony2 HTTP accelerator instead of Varnish // won't work if you use exit() for instance... // won't work if you have "global" state // This is just a skeleton to get you started
// Of course, this is native with Symfony2 ;)
use Symfony\Component\HttpFoundation\Response; use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpKernel\HttpKernelInterface; use Symfony\Component\HttpKernel\Cache\Cache; use Symfony\Component\HttpKernel\Cache\Esi; use Symfony\Component\HttpKernel\Cache\Store;
class AppKernel implements HttpKernelInterface { public function handle(Request $request = null, $type = HttpKernelInterface::MASTER_REQUEST, $raw = false) { return new Response($content, 200, $headers); }
public function getRequest() {} }
$_SERVER['HTTP_SURROGATE_CAPABILITY'] = 'symfony2="ESI/1.0"';
// quick hack... not secure at all! $base = str_replace('index.php', '', $request->getScriptName()); $script = str_replace($base, '', $request->getRequestUri());
ob_start();
include __DIR__.'/'.$script;
$content = ob_get_clean(); $headers = array(); foreach (headers_list() as $header) { $elements = explode(':', $header, 2); $headers[$elements[0]] = trim($elements[1]); }
// do not deal with response others than 200 // implementation depends on your code return new Response($content, 200, $headers);
$kernel = new AppKernel(); $store = new Store('/path/to/http_cache'); $esi = new Esi();
$cache = new Cache($kernel, $store, $esi); $cache->handle()->send();
error_log($cache->getLog());
Goal Be as dynamic as needed
Hit the application as less as possible
The power is even bigger than what you think…
<esi:include src="http://...” alt="http://...” onerror="continue" />
stale-while-revalidate - rfc5861
stale-if-error - rfc5861
There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
http://martinfowler.com/bliki/TwoHardThings.html
PURGE: varnishadm -T localhost:6082 purgeurl "^/images/ »
acl purge_acl { "localhost"; "1.2.3.4"; }
sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge_acl) { error 405 "Not allowed."; } purge_url(req.url); error 200 "Purged."; }
return(lookup); }
curl -X PURGE http://...
WARNING
Don't use GZIP with ESI as Varnish won't be able to assemble the page