a flexible and efficient api for a customizable proxy cache
DESCRIPTION
A Flexible and Efficient API for a Customizable Proxy Cache. Vivek S. Pai, Alan L. Cox, Vijay S. Pai, and Willy Zwaenepoel. iMimic Networking, Inc. http://www.imimic.com. Motivation. More features moving into proxy caches The ubiquitous layer 7 device - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/1.jpg)
A Flexible and Efficient API for a Customizable Proxy Cache
Vivek S. Pai, Alan L. Cox, Vijay S. Pai, and Willy Zwaenepoel
iMimic Networking, Inc.http://www.imimic.com
![Page 2: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/2.jpg)
2
Motivation
More features moving into proxy caches– The ubiquitous layer 7 device
– Filtering, reporting, CDN support, transformation
– Lots of this being done one-off, ad hoc
– Can’t know everything at deployment
Some approaches for generalization– ICAP/OPES, proprietary mechanisms
– But design considerations shifting
Goal: new approach for modern environments
![Page 3: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/3.jpg)
3
Contributions
Designed event-friendly proxy API
Implemented on iMimic DataReactor cache
Imposes negligible performance overhead
Demo modules– High performance
– Low interference
![Page 4: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/4.jpg)
4
Outline
Background
API Design
API Functions
Implementation and Performance
Conclusions
![Page 5: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/5.jpg)
5
Proxy Cache Concepts
clients
proxy cache
LANorigin
servers
WAN
![Page 6: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/6.jpg)
6
Why Program a Proxy?
It’s at the right point in network– Sees all client-side and server-side HTTP traffic
– Can react to both LAN and WAN conditions
Already examines layer 7
Groundwork in place for value-adds– Content filtering, access control, etc.
![Page 7: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/7.jpg)
7
Enabling Technologies
Moore’s Law– CPU speeds outstripping all other components
– Lots of cycles to burn…
Proxy software– Increasing efficiency in managing connections,
disk storage, etc.
Commodity OS/hardware improvements– No longer need specialized systems to run
efficient proxy caches
![Page 8: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/8.jpg)
8
Commodity System Improvements
1997: Appliances 4x faster than software running on a 2-processor UltraSparc
[Source: Danzig, “NetCache Architecture and Deployment”]
![Page 9: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/9.jpg)
9
Commodity System Improvements
1997: Appliances 4x faster than software running on a 2-processor UltraSparc
[Source: Danzig, “NetCache Architecture and Deployment”]
1st NLANR cacheoff (April ’99): gap only 2.5 x– 600 req/sec (Peregrine) vs. 1500 (InfoLibria)
![Page 10: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/10.jpg)
10
Commodity System Improvements
1997: Appliances 4x faster than software running on a 2-processor UltraSparc
[Source: Danzig, “NetCache Architecture and Deployment”]
1st NLANR cacheoff (April ’99): gap only 2.5 x
2nd cacheoff (Jan ’00): gap only 1.7x– 1450 req/sec (iMimic) vs. 2400 (Compaq)
![Page 11: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/11.jpg)
11
Commodity System Improvements
1997: Appliances 4x faster than software running on a 2-processor UltraSparc
[Source: Danzig, “NetCache Architecture and Deployment”]
1st NLANR cacheoff (April ’99): gap only 2.5 x
2nd cacheoff (Jan ’00): gap only 1.7x
3rd cacheoff (Oct ’00): gap only 15%– 2083 req/sec (Microsoft) vs. 2400 (Compaq)
![Page 12: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/12.jpg)
12
Commodity System Improvements
1997: Appliances 4x faster than software running on a 2-processor UltraSparc
[Source: Danzig, “NetCache Architecture and Deployment”]
1st NLANR cacheoff (April ’99): gap only 2.5 x
2nd cacheoff (Jan ’00): gap only 1.7x
3rd cacheoff (Oct ’00): gap only 10%
4th cacheoff (Dec ’01): commodity system best– Performance record: 2700 req/sec (Cintel/iMimic)
![Page 13: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/13.jpg)
13
How free is the CPU?
Stratacache Dart-10, with Nokia phone
120 req/sec (7 Mbps) with 300 MHz CPU– CPU mostly idle; performance disk-limited
![Page 14: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/14.jpg)
14
Outline
Background
API Design
API Functions
Implementation and Performance
Conclusions
![Page 15: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/15.jpg)
15
Previous Customization Approaches
Write your own proxy or modify Squid– Huge code, changes likely to conflict with updates
ICAP: TCP-based offload– Proxy redirects requests/responses to a separate
server for modification
Filter-style processes– Plugins where proxy designers anticipated a need
(e.g., content filtering)
Kernel modules– Difficult programming model, but needed for
kernel-integrated proxies
![Page 16: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/16.jpg)
16
Reasons for a New Approach
Scalability needed to > 10,000 flows– Filter processes may not scale
Limitations of ICAP-style offloading– Offloading small requests adds latency
– Need for separate ICAP server with own CPU
Programmers want flexibility– Program in C using standard OS and libraries
– Avoid problems from later code conflicts
![Page 17: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/17.jpg)
17
Design of the Proxy API
Event-aware– Modules notified as requests/responses arrive
– Maps well to implementation of modern proxies
HTTP-Complete– Capture all key interactions in HTTP request-
response protocol for full flexibility
Support various programming models– Events, threads, processes
– Communication via function call or socket
![Page 18: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/18.jpg)
18
HTTP Data Flows
Client Proxy Cache
Server
Storage System
Requests
Responses
CacheMisses
NewContent
CachedContent
CacheHits
![Page 19: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/19.jpg)
19
HTTP Data Flows and the API
Client Proxy Cache
Server
Storage System
modify
modify
modify
modify
modify
![Page 20: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/20.jpg)
20
HTTP Request-Response Structure
Requested URLRequest header line 1Request header line 2...Request header line N<blank terminating line>
Optional request “body"used in POST requestsfor forms, etc.
Header block – special first line
followed bymore detail aboutrequest/response
Body data
Response Status Code Response header line 1Response header line 2...Response header line N<blank terminating line>
Actual response “body,"containing HTML file,image binary data, etc.
![Page 21: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/21.jpg)
21
typedef struct DR_FuncPtrs { DR_InitFunc *dfp_init; // on module load DR_ReconfigureFunc *dfp_reconfig; // on config change DR_FiniFunc *dfp_fini; // on module unload
DR_ReqHeaderFunc *dfp_reqHeader; // when req hdr done DR_ReqBodyFunc *dfp_reqBody; // on each piece of req body DR_ReqOutFunc *dfp_reqOut; // before req to remote srv
DR_DNSResolvFunc *dfp_dnsResolv; // when DNS resolution needed
DR_RespHeaderFunc *dfp_respHeader; // when resp hdr done DR_RespBodyFunc *dfp_respBody; // on each piece of resp body DR_RespReturnFunc *dfp_respReturn; // when resp returned to clt
DR_TransferLogFunc *dfp_logging; // log entry after req done DR_OpaqueFreeFunc *dfp_opaqueFree; // when each resp completes DR_TimerFunc *dfp_timer; // periodic maintenance int dfp_timerFreq; // timer period (sec) } DR_FuncPtrs;
Design of API Notifications
![Page 22: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/22.jpg)
22
Outline
Background
API Design
API Functions
Implementation and Performance
Conclusions
![Page 23: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/23.jpg)
23
API Functions
Content Adaptation
Content Management
Customized Administration
Utility Functions
![Page 24: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/24.jpg)
24
Content Adaptation
Functions to allow modules to inspect and modify requests and replies through cache
Client Proxy Cache
Server
Storage System
modify
modify
modify
modify
modify
![Page 25: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/25.jpg)
25
Content Adaptation (cont’d)
Example uses– Integration into a CDN based on URL rewriting
– Transcoding for mobile devices
Special features of cache integration– Store modified content
– Return multiple versions using HTTP Vary header
![Page 26: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/26.jpg)
26
Content Management
Fine-grained control over cacheability– Content-freshness modification/eviction
– Content preloading
– Content querying
Example uses– News CDN needs new home page on major event
– Premium services
![Page 27: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/27.jpg)
27
Customized Administration
Notifications on logging
Example uses– Aggregation at network operation centers
– Detection of high error rates indicates bad links
![Page 28: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/28.jpg)
28
Utility Functions
Interfaces to underlying OS event-notification– Module may register or clear interest on FD events
– API will automatically call back module
– Independent of underlying OS mechanisms (e.g., poll, select, /dev/poll, kevent)
Configuration options processing
![Page 29: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/29.jpg)
29
Outline
Background
API Design
API Functions
Implementation and Performance
Conclusions
![Page 30: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/30.jpg)
30
Implementation in DataReactor
Commercial proxy server– Portable (x86, Alpha, Sparc), and
(FreeBSD, Linux, Solaris)
– Fast (exposes overheads)
– Independently measured at Proxy Cache-Offs (alone or via OEMs)
Support requires < 1000 lines of code
Implementation < 6 person-months
![Page 31: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/31.jpg)
31
Sample Modules
Ad Remover– Matches ad patterns in Hostname, URI
Dynamic Compressor– Uses zlib to compress, store, & serve object
Image Transcoder– Color stripping via NetPBM & ijpeg helpers
Text Injector– Finds <head> tag, asks helper what to insert
Content Manager– Local telnet, then query, fetch, inject, evict objects
ICAP client– Implements ICAP 1.0 draft to use external server
![Page 32: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/32.jpg)
32
Web Surfing Now
![Page 33: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/33.jpg)
33
Web Surfing Without Ads
![Page 34: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/34.jpg)
34
Sample Module Implementation
Module Name
Total Lines
Code Lines
Semi-colons
# API call sites
Ad Remover 175 115 51 4
Compressor 387 280 126 11
Transcoder + helper
391
+166
309
+118
148
+54
10
Text Injector
+ helper
473
+56
367
+32
170
+8
12
Manager 675 556 289 56
ICAP Client 1024 719 321 15
![Page 35: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/35.jpg)
35
Measurement
Polygraph and PolyMix-3, Measurement Factory– De facto standard for proxy testing
Scales with load– Number of clients– Number of servers– Data set size– Working set size
Very long test time– Fill phase (~14 hours)– Test phase (~10 hours)
![Page 36: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/36.jpg)
36
PolyGraph Test Phases
Fill Phase
1st L
oad
Pha
se
2nd L
oad
Pha
se 0 5 10 15 20 25 30
Time (hours)
![Page 37: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/37.jpg)
37
PolyGraph Hit Rates
Cacheable
Offered
Actual
![Page 38: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/38.jpg)
38
Our Test Environment
Proxy - 1.4GHz Athlon, 2GB memory 5 SCSI disks, GigE, FreeBSD
Harness– 10 Polygraph client/server machines– Target load: 1450 reqs/sec– 16000 simultaneous connections
Pmix-3: Modified Polymix-3– Single fill phase for all tests– Load phase time cut in half– Slight increase in hit rate
![Page 39: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/39.jpg)
39
API Performance
Throughput
req/sec
Response Time, ms
Miss Time, ms
Hit Time, ms
Hit Ratio, %
Baseline 1452.87 1248.99 2742.53 19.82 57.81
API Enabled 1452.75 1248.95 2743.18 19.86 57.81
Empty Callback 1452.89 1251.25 2744.33 20.87 57.76
Add Headers 1452.62 1251.98 2745.07 20.85 57.74
Body + Headers 1452.84 1250.14 2746.98 22.10 57.85
![Page 40: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/40.jpg)
40
Module Performance
Throughput
req/sec
Response Time, ms
Miss Time, ms
Hit Time, ms
Hit Ratio, %
Baseline 1452.87 1248.99 2742.53 19.82 57.81
Ad Remover 1452.72 1248.87 2743.55 20.42 57.81
Images 25 Trans/s 1452.65 1256.60 2753.47 23.21 57.74
Images Max Trans 1452.73 1277.76 2778.09 43.30 57.80
Max Trans Nice 19 1452.68 1250.69 2744.60 20.15 57.78
Compress 75 obj/s 1452.73 1252.24 2745.63 23.44 57.81
Compress 95 obj/s 1452.88 1258.34 2752.63 28.69 57.78
![Page 41: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/41.jpg)
41
Outline
Background
API Design
API Functions
Implementation and Performance
Conclusions
![Page 42: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/42.jpg)
42
Summary
CPUs getting more idle
Commodity OS suitable choices
High-concurrency servers needed
Customizable, efficient event-friendly API
Implemented with low overhead
Sample results, deployments promising
![Page 43: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/43.jpg)
43
Ongoing Work
CoDeeN – a CDN system on PlanetLab– Uses a customized version of DataReactor
– Being built at Princeton
– Prototype: 1 week reading + 1 week reading
– Currently: ~42 nodes (one per site)
Lessons– API easy enough for busy grad students
– Logging infrastructure would be nice
– Want to mask non-HTTP failures
![Page 45: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/45.jpg)
45
Cacheoff-3 Hit Times
![Page 46: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/46.jpg)
46
Cacheoff-3 Miss Times
![Page 47: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/47.jpg)
47
Cacheoff-3 Improvements
![Page 48: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/48.jpg)
48
Cacheoff-3 Price/Performance
![Page 49: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/49.jpg)
49
CacheOff-3 Results
![Page 50: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/50.jpg)
50
CacheOff-3 Results
![Page 51: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/51.jpg)
51
Cacheoff-4 Hit Times
![Page 52: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/52.jpg)
52
Cacheoff-4 Miss Times
![Page 53: A Flexible and Efficient API for a Customizable Proxy Cache](https://reader035.vdocuments.mx/reader035/viewer/2022062422/56812daf550346895d92d747/html5/thumbnails/53.jpg)
53
CacheOff-4 Results