web proxy acceleration

Cluster Computing 4, 307–317, 2001 2001 Kluwer Academic Publishers. Manufactured in The Netherlands.

Web Proxy Acceleration

DANIELA ROSU, ARUN IYENGAR and DANIEL DIASIBM T.J. Watson Research Center, PO Box 704, Yorktown Heights, NY 10598, USA

Abstract. Numerous studies show that miss ratios at forward proxies are typically at least 40–50%. This paper proposes and evaluatesa new approach for improving the throughput of Web proxy systems by reducing the overhead of handling cache misses. Namely,we propose to front-end a Web proxy with a high performance node that filters the requests, processing the misses and forwardingthe hits and the new cacheable content to the proxy. Requests are filtered based on hints of the proxy cache content. This system,called Proxy Accelerator, achieves significantly better communications performance than a traditional proxy system. For instance, anaccelerator can be built as an embedded system optimized for communication and HTTP processing, or as a kernel-mode HTTP server.Scalability with the Web proxy cluster size is achieved by using several accelerators. We use analytical models, trace-based simulations,and a real implementation to study the benefits and the implementation tradeoffs of this new approach. Our results show that a singleproxy accelerator node in front of a 4-node Web proxy can improve the cost-performance ratio by about 40%. Hint-based request filterimplementation choices that do not affect the overall hit ratio are available. An implementation of the hint management module integratedin Web proxy software is presented. Experimental evaluation of the implementation demonstrates that the associated overheads are verysmall.

Keywords: proxy cache, hints, Web performance

1. Introduction

Proxy caches for large Internet Service Providers can receivehigh request volumes. Numerous studies show that miss ra-tios are often at least 40–50%, even for large proxy caches[7,9,12]. Consequently, a reduction of the cache miss over-heads can result in significant throughput improvements forproxy caches experiencing high request volumes.

Based on this insight and on our experience with Webserver acceleration [15], this paper proposes and evaluates anew approach to improving the throughput of Web proxysystems. Namely, we propose to front-end a Web proxysystem with a high performance node that filters the re-quests, processing the misses and forwarding the hits andnew cacheable content to the proxy. The Web proxy’s miss-related overheads are reduced to receiving the cacheable ob-jects pushed by the accelerator over permanent connections.Consequently, the proxy can use the released resources toservice an increased number of hits. Figure 1 illustrates theinteractions of the proxy accelerator with Web proxy nodes,clients, and Web servers.

The accelerator is an extended HTTP server running un-der an embedded operating system optimized for commu-nication, like the Web server accelerator described [15], orunder a general-purpose operating system in kernel-modeat service-interrupt level, like the Netfinity Web Server Ac-celerator [16]. The accelerator has a main memory cache,mainly used to store content to be forwarded to the Webproxy.

The accelerator filters the requests based on a summaryof the proxy cache content, called hints. The hints are main-tained based on information provided by proxy nodes and in-formation derived from the accelerator’s own decisions. The

hint representation and the update mechanism are importantfactors for the performance gains enabled by the proposedproxy acceleration method. This is due to the related re-quirements for system and network resources and to the in-trinsic tradeoff between resource usage and information ac-curacy.

Previous research has considered the use of location hintsin the context of cooperative Web proxies [8,10,19,22]. Oursystem applies this approach to the management of a cluster-based Web proxy. In addition, we experiment with new hintmaintenance mechanisms, specifically designed for the in-teraction between a front end and a cluster of homogeneousnodes.

Our work extends the set of solutions to proxy cache redi-rection [1,13,14,17]. Most relevant is the comparison withthe proposal in [14], which has the front end identifying andprocessing the requests for non-cacheable content. Extend-

Figure 1. Accelerated Web Proxy Architecture.

308 ROSU, IYENGAR AND DIAS

ing this functionality, a proxy accelerator processes all of therequests for content not cached at the Web proxy, includingthe cacheable content not currently in the proxy cache.

Building on top of previous research on cluster-based net-work services [2,4], our proxy acceleration method over-comes the intrinsic limitations of content-based routing com-ponents by integrating several accelerator nodes in front of aWeb proxy cluster.

In this paper, we use analytical models in conjunctionwith simulations to study the potential for performance im-provement of our proxy acceleration scheme. We evaluatethe impact of several hint representation and maintenancemethods on throughput, hit ratio, and resource usage. In ad-dition, we present and evaluate the performance impact ofan actual implementation of the hint-management compo-nent in Web proxy application.

Our analytical study reveals that the throughput improve-ment can be significant. For instance, when the acceleratorcan achieve about an order of magnitude better throughputthan a Web proxy node for communications operations, suchas the embedded system considered in [15], a single acceler-ator node can boost the throughput of a four-node Web proxyby about 100%. Our simulations validate that, with appropri-ate hint representations and update protocols, the proposedacceleration method can offload more than 50% of the Webproxy load with no impact on the observed hit ratio. In ad-dition, the evaluation of our implementation shows that theoverhead incurred with hint management has no significantimpact on the client-perceived response times and through-put.

Paper overview. The rest of the paper is structured as fol-lows. Section 2 describes the architecture of an acceleratedWeb proxy. Section 3 analyzes the expected throughput im-provements with an analytical model. Section 4 evaluatesthe impact of the hint management on several Web proxyperformance metrics with a trace-driven simulation. Sec-tion 5 presents the implementation of the hint-managementmodule in a Web proxy application and evaluates its perfor-mance impact. Section 6 discusses related work, and sec-tion 7 summarizes our results.

2. Accelerated Web proxy

Typical high-performance Web proxy systems consist of acluster of nodes running a Web proxy application and afront-end node that distributes the requests to the applica-tion nodes. Front end-based solutions to improving the per-formance of Web proxy systems are restricted mainly to bal-ancing the load based on criteria like number of active con-nections and CPU utilization. Because of the inherently un-predictable content mixtures in Web proxy caches, typicalcontent-based routing policies that distinguish origin serversand/or object types are less beneficial. However, a content-based scheme distinguishing between cacheable and non-cacheable content has been proved to bring significant per-

formance improvements to Web proxy systems when com-bined with off-loading the processing of requests for non-cacheable content from proxy nodes to the front end [14].

In this paper, we propose to further offload proxy nodeoverheads by extending the functionality of the front-end.Namely, the extended front-end, called proxy accelerator(PA), is a content-based router that can distinguish betweencache hits and misses based on information about proxycache content. In the case of a cache hit, the PA forwardsthe request to a Web proxy (WP) node that contains the ob-ject either by TCP splicing or handoff [17,21]. In the case ofa cache miss, the PA sends the request to the origin Web site,bypassing the WP. When it receives the object from the Website, the PA replies to the client, and if appropriate, pushesthe object to a WP node which is selected based on someload balancing criteria.

The enabling element for the new front-end functionalityis the PA’s information about the content of the WP cache,henceforth called hints. Possible solutions for hint represen-tation and update mechanism will be discussed later in thissection and evaluated by trace-based simulation in section 4.

The PA includes a main memory cache for storing contentwaiting to be pushed to the WP. In addition, the PA cache canstore duplicates of objects in the WP cache. Requests satis-fied from a PA cache never reach the WP, further reducingthe load of the WP.

Figure 1 illustrates the interactions of a PA in a 4-nodeWP cluster. PA and WP nodes are connected by permanentTCP connections for request handoff [17], hint updates, andcontent push. Notice that WP nodes still interact with Websites. This occurs when the PA filter is not accurate, or cacheobjects have to be revalidated.

For scalability, the system may include several PAs (fig-ure 2) and a router that balances the load among them. EachPA has hints about each of the WP nodes. Therefore, a PAcan appropriately forward any of the incoming hits. The pol-icy for new content distribution may be as simple as round-robin.

A PA can be built from stock hardware which runs un-der an embedded operating system optimized for commu-nication. Alternatively, the PA can be built as an extendedkernel-mode HTTP server. Either way, a PA can achieve anorder of magnitude better throughput than a WP node forcommunications operations.

Figure 2. Interactions in a Web Proxy (WP) cluster with several ProxyAccelerators (PA).

WEB PROXY ACCELERATION 309

A WP node would typically run software extended forhint maintenance and distribution, for handoff of TCP/IPconnections and URL requests, and for pushing objects toPA caches.

2.1. Hint representation

Previous research has considered several hint representa-tions and consistency protocols. Hint representations varyfrom schemes that provide accurate information [10,22] toschemes that trade accuracy for reduced costs (e.g., memoryand communication requirements, look-up and update over-heads) [8,19].

In this study, we consider two representations – one thatprovides accurate information and one that provides approx-imate (inaccurate) information. The accurate-informationscheme, called the directory scheme, uses a list of objectidentifiers including an entry for each object known to existin the WP cache. In our implementation, the list is repre-sented as a hash table [10,22].

The approximate-information scheme is based on BloomFilters [5]. The filter is represented by a bitmap, henceforthcalled hint space (see figure 3), and several hash functions.To register an object in the hint space, the hash functionsare applied to the object identifier, and the correspondingbits, which are generically called hint entries, are set in thehint space. A hint look-up is positive if all of the entries as-sociated with the object of interest are set. A hint entry iscleared when no object is left in the cache whose represen-tation includes the entry. Therefore, the cache owner (i.e.,the WP node) maintains a “collision” counter for each en-try, which is updated whenever a related object is added orremoved from the cache. When a collision counter reachesits maximum value, it is no longer updated. The entire hintspace is recomputed when the number of overflowed coun-ters reaches a threshold. The PA maintains a Bloom Filterfor each of the WP nodes.

2.2. Hint consistency

The hint consistency protocol ensures that the informationat the PA site reflects the content of the WP cache. Pro-tocol choices vary in several dimensions: the entity thatidentifies the updates, the protocol initiator, the rate of up-date messages, and the exchanged information. In the pre-viously proposed solutions, updates are identified either bythe WP (henceforth called WP-only updates) [8,19] or by thecontent-based router (PA) [17]. Both methods have a signif-icant likelihood of wrong request redirection decisions. The

Figure 3. Hint representation based on Bloom Filters.

first method may result in false misses because of the in-herent delay of updates. The second method may result infalse hits because information about objects removed fromthe cache is not propagated to the router. False misses arealso possible because entries in the router directory are dis-carded periodically due to space limitations.

In this paper, we propose and evaluate a method, calledeager hint registration, which attempts to address the draw-backs of previous methods. Namely, both PA and WP iden-tify and record hints. When the PA pushes an object to aWP node, it “eagerly” registers that the object exists in thecorresponding cache. As a side effect, the method enables areduction of the update traffic sent by a WP node. Namely,a WP node does not send update notifications for objectsreceived from the PA (a small exception, due to the asyn-chronous communication between PA and WP, is discussedin section 5).

With respect to the protocol initiator, previously proposedsolutions address only WP-only updates. Updates are pulledby the hint owner (i.e., PA) [19] or pushed by the cacheowner (i.e., WP) [8,10,22]. With respect to the update rate,previous proposals consider update messages sent at fixedtime intervals [19] or after a fixed number of cache up-dates [8]. The exchanged information may be incremental,addressing only the updates occurred since the previous mes-sage [8,10,19], or a copy of the entire hint space [8].

2.3. Proxy accelerator cache

The PA cache may include objects received from Web sitesor pushed by the associated WP nodes. In selecting thePA cache replacement policy, we have to consider that allobjects in the PA cache also exist in the WP cache. Con-sequently, the PA cache replacement policy should not befocused on minimizing the network-related costs, as is ap-propriate for WP caches. Policies that maximize the hit ra-tio based on hotness statistics and object sizes are expectedto enable better PA performance than policies like Greedy-Dual-Size [6].

In the following sections, we evaluate the throughput po-tential of the proposed proxy acceleration method and theimpact of several hint implementation choices.

3. Expected throughput improvements

The major benefit of the proposed hint-based accelerationof Web proxies is throughput improvement. We analyze thethroughput properties of this scheme with an analytic modelby varying the number of WP and PA nodes, the PA func-tionality and cache size, and the WP’s and PA’s CPU speeds.

The system is modeled as a network of M/G/1 queues.The servers represent PAs, WP CPUs and disks, and a Website. The serviced events represent (1) stages in the process-ing of a client request (e.g., PA Miss, WP Hit, Disk Access)and (2) hint and cache management operations (e.g., hint up-date, object push). Event arrival rates derive from the rate of


Figure 4. Throughput with number of WP and PA nodes (no service fromPA cache).

client requests and the likelihood of a request to switch fromone processing stage to another. The likelihood of variousevents, such as cache hits and disk operation, are derivedfrom recent studies on cache performance and Web contentmodels [6,14,18,23].

The overhead associated with each event includes a ba-sic compute overhead, I/O overheads (e.g., connect, re-ceive message, handoff), and operating system overheads(e.g., context switch). The basic overheads are identicalfor the PA and WP, but the I/O and operating system over-heads are different. Namely, the PA overheads correspondto the embedded system used for Web server accelerationin [15] – a 200 MHz Power PC PA which incurs a 514 µsmiss overhead (i.e., 1945 misses/s) and 209 µs hit overhead(4783 hits/s). The WP overheads are derived from [17] –a 300 MHz Pentium II which incurs a 2518 µs miss overhead(i.e., 397 misses/s) and 1392 µs hit overhead (718 hits/s).These overheads assume 8 KByte objects.

The following parameters are fixed for this study: (1) 40%non-cacheable content ratio and 25% miss ratio, (2) TCPhandoff is used by the PA for client connections upon pos-itive hint look-up; (3) directory-based hints with 1 h incre-mental updates; (4) disk I/O is required for 75% of the WPhits; (5) one disk I/O per object; (6) four network I/Os perobject transmission; (7) 25% of WP hits are pushed to thePA, when the PA caches hot objects; (8) 30 µs WP contextswitch overhead. Unless otherwise specified, the PA nodeshave 450 MHz CPUs, and the WP nodes have 300 MHzCPUs. The costs of the hint-related operations are derivedfrom our implementation and the simulation-based study(see section 4).

3.1. Hint-based acceleration improves throughput andcost-performance ratio

Figure 4 presents the expected performance for a traditionalWP-cluster and for clusters enhanced with up to four PAswhen the PAs do not service objects from their cache. Theplot illustrates that the hint-based acceleration enables sig-nificant WP throughput improvements. For instance, a sin-

Figure 5. Cost-performance improvement with number and speed of WPand PA nodes (no service from PA cache).

gle PA node can increase the performance of a 2-node WPabout three times and that of a 4-node WP about two times.With two PAs, a 4-node WP can achieve higher throughputthan a 16-node WP. However, consistent with previous stud-ies on content-based routing [2,4,21], the plot illustrates thata PA becomes a bottleneck for large WP clusters.

The cost-performance improvement enabled by the pro-posed scheme is relevant. For instance, figure 5 illustratesthat if we assume that the cost of a PA is almost the same asfor a WP node, the cost-performance improvement achievedwith a single PA is ∼60% in a 2-node WP and ∼40% in a4-node WP. The performance benefits of the scheme increasewith the difference between WP and PA’s CPU speeds, as il-lustrated by figure 5 for the case of 700 MHz PA CPU’s.Figure 5 also illustrates that a 2-node accelerator benefits rel-atively large WP configurations more than small WP config-urations. For instance, the cost-performance improvementwith 2 PAs is lower than with 1 PA for a 2-node WP.

3.2. Hint-based acceleration can benefit more than theredirection of non-cacheable objects

Figure 6 illustrates that, for appropriately sized WP clusters,the proposed hint-based acceleration can enable better per-formance than the previously proposed policy of redirectingonly the non-cacheable objects [14]. For a workload withonly 25% miss ratio, a 2-node WP cluster can attain morethan double throughput levels if the front-end can identifyand process the request for cache misses along with the re-quests for non-cacheable content.

3.3. PA-level caches benefit low-performance WPs

Figure 7 illustrates that the effect of a PA cache is marginalin particular when the ratio of non-cacheable content is sig-nificant, like in our selection of experiment parameters.

To summarize, the hint-based Web Proxy accelerationmay lead to significant performance and cost-performanceimprovements when the PA nodes can achieve a much higherthroughput for network I/O than the WP nodes. The new


Figure 6. Throughput of hint-based acceleration and non-cacheable redi-rection (no service from PA cache).

Figure 7. Throughput when PA services objects from its cache.

scheme improves upon the previously proposed schemes forredirection of non-cacheable content for appropriately sizedWP clusters.

4. Impact of hint management

In this section, we evaluate how the hint representation andthe consistency protocol can affect the performance of anaccelerated Web proxy. We focus on performance met-rics, such as hit ratio and throughput, and on cost metrics,such as memory, computation, and communication require-ments.

These performance and cost metrics depend on variouscharacteristics of the hint management mechanism (see ta-ble 1). For instance, the look-up overhead affects the long-term PA throughput. Also, the false miss decisions of the PAfilter cause undue network loads and response delays. Ta-ble 2 summarizes some of the differences between the rep-resentation methods and the update protocols introduced insection 2. For instance, the look-up overhead is constantfor the Bloom Filter-based scheme while for the directoryscheme it depends on the context, namely on how well the

Table 1Impact of hint management choices.

Characteristic Affected Performance

Memory at WP WP memory hit ratioMemory at PA PA cache hit ratioUpdate period Long-term throughputUpdate period WP/PA short-term throughputLook-up time PA long-term throughputFalse hits WP loadFalse misses Network usage

Table 2Selected method characteristics.

Characteristic RepresentationDirectory Bloom Filters

Memory at WP None O(hint space)Memory at PA O(objs WP) O(hint space)Look-up overhead Context-dependent constantFalse hits None O(objs WP)

ProtocolWP-only Eager Reg.

False misses Period-dependent NonePA computation On-off, large bursts Smaller bursts

hash table is balanced. Similarly, eager registration results inno false misses, while for WP-only registration the amountof false misses increases with the update period.

In this study, we identify and compare relevant trendsof these representation methods and update protocols. Thestudy is conducted with a trace-driven simulator and a seg-ment of the Home IP Web traces collected at UC Berkeleyin 1996 [11]. The traces include about 5.57 million clientrequests for cacheable objects over a period of 12.67 days.The number of objects is about 1.68 million with a total sizeof about 17 GBytes.

4.1. Factors

The factors considered in this study are the following: (1) PAand WP memory, (2) hint representation, (3) hint update pro-tocol, and (4) update period. Namely, we consider configu-rations with 0.25–1 GByte PA memory and 1–10 GByte WPmemory and disk space. For the Bloom Filter-based scheme,we vary the size of the hint space (0.5–10 MBytes) and thenumber of hash functions; each hash function returns the re-mainder of the hint space size divided by an integer obtainedas a combination of four bytes in the object identifier. Dueto limitations associated with trace encoding, object iden-tifiers are composed of the 16-byte MD5 representation ofthe URL and a 4-byte encoding of the server name (this re-striction makes impossible an accurate evaluation of the re-source requirements associated with the Bloom Filter-basedscheme). Finally, we experiment with update periods in therange of 1 s to 2 h.


4.2. Fixed parameters

We experiment only with “push” updates; in comparison to“pull” updates, “push” updates allow better control of hintaccuracy [8] and lower I/O overheads. In addition, we ex-periment only with incremental updates, as we consider itmore appropriate than “entire copy” updates in the contextof the hint representations and the system parameters consid-ered in this study. The WP cache replacement policy is LRU.The PA cache replacement is “LRU with bounded replace-ment burst”. According to this policy, an incoming objectis not cached if this would cause more than a given num-ber of objects to be removed from the cache. This policyis expected to perform well based on studies that show thatsmall objects have a large contribution to the hit ratio [3,18].For both caches, garbage collection is invoked when a newobject has to be accommodated.

4.3. Cost metrics

4.3.1. Memory requirementsThe main memory requirements of the hint mechanism de-rive from the hint meta-data and the message buffers usedfor the hint consistency protocol.

Hint meta-data. For the directory scheme, no meta-data ismaintained at the WP, and at the PA, the requirements in-crease linearly with the population of the WP cache. Forinstance, in our implementation, with 68-byte entries, themeta-data is ∼70 MBytes for a 10 GByte WP cache. Bycontrast, for the Bloom Filter-based scheme, meta-data ismaintained at both PA and WP, and is independent of thecache size. The PA meta-data is as large as the hint spacemultiplied by the number of WP nodes, and the WP meta-data is as large as the hint space multiplied by the size of thecollision counter (e.g., 8 bits).

Hint consistency protocol. Message buffer storage spacerequired at the WP site for the hint consistency protocol in-creases with the rate of WP cache updates and the period ofthe consistency protocol. For the directory scheme, an up-date entry has unpredictable length, as it includes the fullname of the object. For the Bloom Filter-based scheme, anupdate corresponds to a hint entry update which is repre-sented by a 5-byte data structure; a cache update can resultin as many hint entry updates as hash functions in the filter.

4.3.2. Computation requirementsThe computation requirements associated with our acceler-ation scheme result mainly from request look-up and theprocessing of update messages. In addition, minor compu-tation overheads may be incurred at WP and PA upon eachcache update.

Look-up overhead. For the directory scheme, the look-upoverhead depends on the distribution of objects to hash tablebuckets. For the Bloom Filter-based scheme, the look-up

overhead is more predictable, depending mainly on the fil-ter parameter. More specifically, the overhead includes thetransformation of the object name into one or more inte-ger values, the computation of the hash functions, the se-lection, and the test of the corresponding bitmap entries.On a 332 MHz PowerPC, the transformation of the objectname into an MD5 representation takes ∼9 µs per 55 bytestring segment. For the four simple hash functions used inour Bloom Filter implementation, the overhead of hash func-tion computation and bitmap look-up is ∼1 µs. Overall, thelook-up overhead is lower than 10% of the expected cachehit overhead on a 200 MHz PA (see section 3).

Hint consistency protocol. The amount of computationtriggered by the receipt of a hint consistency message de-pends on the number of update entries included in the mes-sage. Because of hint entry collisions, the Bloom Filter-based scheme results in about half the load of the directoryscheme (with 160-byte object identifiers). The variation ofmaximum and average message sizes presented in figures 8and 9 illustrate this trend. These figures also illustrate the

Figure 8. Variation of maximum number of entries per update message.1 h period, 256 MByte PA, 4 MByte hint space, 4 entries per object.

Figure 9. Variation of total number of update entries. 1 h update period,256 MByte PA, 4 MByte hint space, 4 entries per object.


Figure 10. Variation of overall hit ratio with WP configuration. 512 MBytePA, Bloom Filter with 10 MByte hint space.

Figure 11. Variation of WP requests with update protocol and period.256 MByte PA, 4 GByte WP, 4 MByte hint space.

benefit of the eager registration approach. For both hintschemes, on average, the eager hint registration reduces byhalf the computation loads. Similar trends are observed forthe communication requirements of the consistency proto-col. Note that the very large maximums observed in figure 8are due to a few very large WP cache replacement bursts(∼2300 objects).

4.4. Performance metrics

4.4.1. Hit ratioThe characteristic of the hint mechanism that most affectsthe (observed) hit ratio is the likelihood of false misses. Typ-ically, false misses are due to hint representation and updatedelays. In our experiments, because of the selected hint rep-resentations, false misses are only the result of update de-lays.

As expected, the likelihood of false misses increases withthe update period. Figure 11 illustrates this trend indirectly:the ratio of requests directed to the WP with a traditionalupdate mechanism decreases with the increase of the update

Figure 12. Variation of false hit ratio with hint space and WP cache.512 MByte PA.

period. Eager hint registration reduces the number of ob-served false misses, with less impact on the Bloom-Filterscheme because of collisions.

The amount of hint meta-data at the WP can have a rele-vant impact on the overall hit ratio (i.e., hits in both PA andWP caches) when the ratio of the meta-data in the WP mem-ory is large, as for small WP cache sizes (see figure 10). PAmeta-data has no significant impact because objects removedfrom the PA cache are pushed to the WP rather than beingdiscarded.

4.4.2. ThroughputThe extent of throughput improvement depends on how wellthe PA identifies the WP cache misses. This characteristic,reflected by the likelihood of false hits, is affected by therepresentation method and the consistency protocol.

Hint representation. While the directory scheme has nofalse hits, for the Bloom Filter-based scheme, the likelihoodof false hits increases with the likelihood of hint-space col-lision. As illustrated in figure 12, this occurs when the hintspace decreases or the number of objects in the WP cacheincreases. We notice that the false-hit ratio is constant whenthe hint space increases beyond a threshold. This thresholddepends on the WP cache size, and the effect is due to thecharacteristics of the selected hash functions.

The likelihood of false hits also depends on the num-ber of hint entries per object. Figure 13 illustrates thevariation observed in our experiments with filters with 3–5hash functions. While the ratio of false hits typically de-creases with the number of hint entries per object, when hintspace occupancy is high, inversions may occur due to colli-sions.

Hint consistency protocol. The impact of update periodon false hit ratio is more significant in configurations withfrequent WP cache updates (e.g., systems with small WPcaches). Figure 14 illustrates this trend for both hintschemes. For instance, for Bloom Filters, the difference be-


Figure 13. Variation of false hit ratio with hint entries per-object.512 MByte PA.

Figure 14. Variation of false hits with update period. 256 MByte PA,4 MByte hint space.

tween the false-hit ratios observed for 1-h and for 1-s up-dates is 0.07% for a 4 GByte cache and 0.04% for a 6 GBytecache. Comparing the two schemes, the relative impact ofthe update period is almost identical.

4.4.3. Request response timeBesides the overhead of hint look-up and update, the ac-celerated WP architecture can affect the average request re-sponse time due by the reduction of hit ratios in the WP andPA main-memory caches. Besides the amount of meta-datain the available main memory, the hit ratios depend on thecache replacement policies. Considering the PA cache, fig-ure 15 illustrates that the bounded replacement policy en-ables better PA hit ratios without reducing the overall hit ra-tio. For instance, limiting the replacement burst to 10 objectsenables more than 5% larger PA hit ratios than with unlim-ited burst (see bound = −1). This is because the averagereplacement burst is significantly smaller than the maximum(e.g., 3 vs. 2300 in our experiments).

Figure 15. Variation of hit rate with replacement bound. “−1” for un-bounded replacements. Bloom Filter, 6 GByte WP, 4 MByte hint

space.

4.4.4. SummaryBy trading off accuracy, the Bloom Filter-based schemeexhibits lower computation and communication overheads.However, the meta-data at the WP node may reduce thememory-hit ratio. While, in general, the hint look-up over-heads are small with respect to the typical request processingoverheads, the overheads are more predictable for the BloomFilter-based scheme. Eager hint registration reduces update-related overheads and prevents hit ratio reductions caused byupdate delays. False hit ratio (and, consequently, through-put) is more significantly affected by hint representation thanby update period. For Bloom Filter-based schemes, this ratioincreases exponentially with hint space contention.

5. Implementation of hint management in Web proxynodes

The implementation of the hint-based accelerated Webproxy infrastructure is built by extending a thread-basedWeb proxy application and a kernel-mode HTTP server withfunctionality for hint management and content push. Theimplementation of the hint mechanism is based on BloomFilters with eager registration and periodic updates. The im-plementation makes possible a flexible system configuration.Namely, the Web proxy node can interact with multiple ac-celerators, and the accelerator node can interact with multi-ple proxy nodes.

5.1. Hint representation

The size of the hint space is customizable at initializationtime. The collision counters take up one byte. The BloomFilter has six hash functions, four applied to the object nameand two to the origin server IP address. The object name,s0s1 . . . , is transformed into an integer value by evaluat-ing the polynomial

∑i siX

i for a fixed prime number. Theoverhead of name transformation is ∼0.95 µs for the first


Figure 16. Hint representation as interleaved Bloom Filters for 4 WP nodes.

10 bytes and ∼0.7 µs for each additional 10 bytes on a332 MHz PowerPC. Note that, although less effective thanMD5 in avoiding collisions, this solution is characterizedby lower computation overheads, which clearly benefit theproxy accelerator performance.

The hash functions are equal to mod(HintSpace) appliedto the resulting 4-byte integer and to three permutations ofits bytes. Similarly, the mod(HintSpace) function is appliedto the 4-byte integer representing the IP address and to onepermutation of its bytes. The overhead of computing thesehash functions is ∼5.5 µs.

When the WP has several nodes, the correspondingBloom Filters are represented at the PA by interleavedbitmaps. In this representation, the corresponding bits ofall of the filters are placed in consecutive memory locations(see figure 16). In comparison to the original representa-tion in which each Bloom Filter is represented in a separatememory segment, the interleaved representation is character-ized by predictable look-up overheads, independent of thecluster size. With the interleaved representation, the look-up overhead for a 4-bit entry is constant at ∼1.28 µs, ona 332 MHz PowerPC. For the original representation, theoverhead varies from ∼1.19 µs for a single node look-up to∼4.39 µs for four node look-up.

5.2. Hint update protocol

The hint management implementation uses the eager hintregistration protocol introduced in section 2. Namely, an ac-celerator registers hints as it pushes objects towards a proxynode. A proxy node collects the hint updates and periodi-cally sends them to each accelerator such that an accelera-tor will not receive updates resulting from cache updates forcontent it has pushed to the WP node. An exception is madewhen a hint entry is set by a push operation when the pre-vious clear has not reached the accelerator before it pushedthe object. If not appropriately handled, this situation maylead to false misses because the bit would remain cleared atthe PA. To identify this type of situation, proxy updates arelabeled, and the pushed content sent by an accelerator carrythe label of the most recent update enacted by the proxy.

5.3. Web proxy software extension

The extension of the Web proxy software has three compo-nents. The first component represents the update interfaceinvoked by the proxy threads handling client requests whenobjects are added or permanently removed from the cache(cache operations that remove objects only for accommodat-ing their newer versions should not result in hint updates).

The calling thread indicates the object name, origin server,and the corresponding cache operation. The resulting hintupdates, if any, are recorded in update buffers. When an up-date buffer gets full, it is released for transmission. Updatebuffers are organized in an active set and a waiting set. Up-dates are recorded in buffers of the active set. When sucha buffer gets full, it is replaced with one in the waiting set.The active set includes a buffer for each active acceleratorand the local proxy node. Updates are registered in the ac-tive buffer associated with the node that originated the op-eration. More specifically, updates resulting from a cache-add operation are registered in the buffer of the proxy nodewhen it has received the content from the origin Web server;otherwise, the update is registered in the buffer of the accel-erator that pushed the object to the proxy cache. Updatesfor cache-remove operations are registered in the buffer ofthe proxy node. This scheme prevents replication of updaterepresentations when the proxy interacts with several accel-erators and permits the use of multicast if available.

The second component represents the proxy acceleratorinterface. This component consists of three threads: onethread listens for incoming connections from accelerators,one thread handles accelerator requests for transmission ofthe entire hint space, and the third thread handles the pe-riodic updates. Accelerators may request a complete copyof the hint space at their initialization. The periodic updatethread disseminates to the registered accelerators the updatebuffers released for transmission. If no buffer is released, thethread transmits the currently active batch update buffer.

The third component of the Web proxy extension is thepush handler. This component receives the new content andstores it in the cache. The content is sent in HTTP formatas a POST request. While this requires the proxy node toduplicate the header processing that was already done in theaccelerator, it decouples the implementations of the proxycache and the accelerator. In order to hide the latency ofthe related I/O operations, multiple push connections can beconcurrently active between accelerator and proxy.

5.4. Experimental evaluation

In the remainder of this section, we present an experimen-tal evaluation of the overhead of hint management. Theexperimental evaluation is done with the Web Polygraphproxy performance benchmark [20]. Each component of thebenchmark, polyclt, polysrv, Web Proxy, and proxy accel-erators run on separate nodes of a switched Ethernet-basedLAN.

The selected traffic model is meant to stress the hint-related component of the Web proxy application. Namely,the request model is characterized by a 75% miss ratio. Thecontent size is fixed to 2 bytes, which results, with the head-ers, in about 370 bytes per request. Requests are issued bythe polyclt node in best-effort mode, meaning that a new re-quest is sent as soon as the reply for the previous one is re-ceived. Each experiment consists of 100,000 requests.


During experiments, the Web proxy configuration is var-ied from original (i.e., no hint collection and updates), to hintcollection but no updates, and to both hint collection and up-dates to one and two accelerators. During these experiments,the role of accelerators is reduced to receiving hint updates;requests are received directly by the Web proxy and, there-fore, they do not pass through the accelerator. The updateperiod is 5 min.

The Web Polygraph statistics about the response timesobserved by the client show that the distributions of hitand miss response times are almost identical up to the 98-percentile for all the tested configurations. For the 99-percentile, the miss response times experienced with theconfigurations with active hint management differ by lessthan 1% from the performance of the original configuration.For hit response times, the 99-percentile difference is lessthan 10% of the performance in the original configuration.The throughput observed by the client is unchanged for theexperiment with no active accelerators and ∼1.8% lower forthe experiments with active accelerators.

With Web Proxy internal monitoring instrumentation, wecollected statistics about the overhead of hint collectionwhen no proxy accelerator was active. This overhead in-cludes hint computation and collection into batch updatebuffers. The statistics show a minimum of 31 µs, a me-dian of 62 µs, and a 95-percentile of 123 µs. The aboveresults demonstrate that hint collection and update protocolhave negligible impact on the performance of a Web proxynode.

6. Related work

Previous research has considered the use of location hintsin the context of cooperative Web Proxies [8,10,19,22]. Inthese papers, the hints help reduce network traffic by moreefficient Web Proxy cache cooperation. Extending these ap-proaches, we use hints to improve the throughput of an in-dividual Web proxy. In addition, we propose and evaluatenew hint maintenance techniques that reduce overheads byexploiting the interactions between a cache redirector andthe associated cache.

Web traffic interception and cache redirection relates ourapproach to architectures such as Web server accelerators[15], ACEdirector (Alteon) [1], DynaCache (InfoLibria)[13], Content Smart Switch [14], and LARD [17]. The abil-ity to integrate a Proxy Accelerator-level cache renders ourapproach similar to architectures in which the central ele-ment is an embedded system-based cache, such as the hier-archical caching architecture in [13], or the front end of aWeb server in [15]. Our proxy accelerator extends the ap-proach to bypassing the Web proxy proposed in [14]; be-sides non-cacheable objects, our accelerator can identify themisses and process them locally. Trading the throughput ofthe routing component, which is not the bottleneck resource,our approach can boost even more the throughput of the bot-tleneck proxy nodes. Furthermore, our approach is similar

to the redirection methods used in architectures focused oncontent-based redirection of requests to Web Proxy nodes[1,17]. However, our approach enables dynamic rather thanfixed mappings of objects to WP nodes [1]. In contrast tothe method in [17], redirection does not cause caching ofmultiple object replicas and is independent of client requestpatterns.

7. Conclusions

Based on the observation that miss ratios at proxy caches areoften relatively high [7,9,12], we have developed a methodto improve the performance of a cluster-based Web proxy byshifting some of its cache miss-related functionality to be ex-ecuted on a Proxy Accelerator – an extended content-basedrouter implemented on an embedded system optimized forcommunication. Consequently, the Web proxy can servicea larger number of hits, and the Proxy Accelerator-basedsystem can achieve better throughput than traditional Webproxies [1,8,17,19] or systems in which the Web proxy isbypassed only for non-cacheable objects [14]. In addition,under moderate load, response time can be improved with aProxy Accelerator main memory cache [13,15].

Our study shows that a single Proxy Accelerator nodewith about an order of magnitude better throughput for com-munication operations than a proxy node [15,16] can im-prove the cost-performance ratio of a 4-node Web proxy by∼35%. Our study shows that eager registration is a “must”and that the Bloom Filter-based scheme is more appropri-ate than the directory scheme for large WP clusters or whenPA and WP nodes have comparable power. The implemen-tation of the accelerated Web proxy node demonstrates thatthe overhead added by hint management is not significant incomparison to typical proxy node overheads.

References

[1] Alteon Web Systems, ACEdirector Web Switch, http://www.alteonwebsystems.com/products/acedirector2-data.shtml (1999).

[2] M. Aron, D. Sanders, P. Druschel and W. Zwaenepoel, Scalablecontent-aware request distribution in cluster-based network servers,in: Proc. Annual Usenix Technical Conference (2000).

[3] P. Barford, A. Bestavros, A. Bradley and M. Crovella, Changes inWeb client access patterns. Characteristics and caching implications,World Wide Web Journal, Special Issue on Characterization and Per-formance Evaluation 2(1–2) (1999).

[4] A. Bestavros, M. Crovella, J. Liu and D. Martin, Distributed packetrewriting and its application to scalable server architectures, in: Proc.6th Int. Conf. on Network Protocols (1998).

[5] B. Bloom, Space/time trade-offs in hash coding with allowable errors,Communications of ACM 13(7) (1970) 422–426.

[6] P. Cao and S. Irani, Cost-aware WWW proxy caching algorithms, in:Proc. USENIX Symp. on Internet Technologies and Systems (1997).

[7] B.M. Duska, D. Marwood and M.J. Feeley, The measured access char-acteristics of World-Wide-Web client proxy caches, in: Proc. USENIXSymp. on Internet Technologies and Systems (1997).

[8] L. Fan, P. Cao, J. Almeida and A.Z. Broder, Summary cache: A scal-able wide-area Web cache sharing protocol, in: Proc. SIGCOMM’98(1998).


[9] A. Feldmann, R. Caceres, F. Douglis, G. Glass and M. Rabinovich,Performance of Web proxy caching in heterogeneous bandwidth en-vironments, in: Proc. IEEE INFOCOM’99 (1999).

[10] S. Gadde, J. Chase and M. Rabinovich, A taste of crispy squid, in:Proc. Workshop on Internet Server Performance (1998).

[11] S.D. Gribble, UC Berkeley Home IP HTTP Traces, http://www.acm.org/sigcomm/ITA (1997).

[12] S.D. Gribble and E.A. Brewer, System design issues for Internet mid-dleware services: Deductions from a large client trace, in: Proc.USENIX Symp. on Internet Technologies and Systems (1997).

[13] A. Heddaya, DynaCache: Weaving caching into the Internet, in: Proc.International Web Caching Workshop (1998).

[14] E. Johnson (ArrowPoint Communications), Increasing the perfor-mance of transparent caching with content-aware cache bypass, in:Proc. International Web Caching Workshop (1999).

[15] E. Levy, A. Iyengar, J. Song and D. Dias, Design and performance ofa Web server accelerator, in: Proc. IEEE INFOCOM’99 (1999).

[16] IBM Netfinity Web Server Accelerator V2.0, http://www.pc.ibm.com/us/solutions/netfinity/server_accelerator.html (2000).

[17] V. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel, W. Zwaenepoeland E. Nahum, Locality-aware request distribution in cluster-basednetwork servers, in: Proc. Int. Conf. on Architectural Support for Pro-gramming Languages and Operating Systems (1998).

[18] A. Rousskov and V. Soloviev, A performance study of the squid proxyon HTTP/1.0, WWW Journal 1(2) (1999).

[19] A. Rousskov and D. Wessels, Cache digests, http://squid.nlanr.net/Squid/CacheDigest (1998).

[20] A. Rousskov and D. Wessels, High performance benchmarkingwith Web polygraph, http://ircache.nlanr.net/Polygraph/doc/papers/paper01.ps.gz (1999).

[21] J. Song, E. Levy, A. Iyengar and D. Dias, Design alternatives for scal-able Web server accelerators, in: Proc. IEEE Int. Symp. on Perfor-mance Analysis of Systems and Software (2000).

[22] R. Tewari, M. Dahlin, H.M. Vin and J.S. Kay, Beyond hierarchies:Design considerations for distributed caching on the Internet, in:Proc. Int. Conf. on Distributed Computing Systems (1999).

[23] A. Wolman, G.M. Voelker, N. Sharma, N. Cardwell, A. Karlin andH.M. Levy, On the scale and performance of cooperative Web proxycaching, in: Proc. ACM Symp. on Operating Systems Principles(1999).

Daniela Rosu is a Research Staff Member at IBM’s T.J. Watson ResearchCenter. She earned Ph.D. and M.S. degrees in computer science, fromGeorgia Institute of Technology in 1999 and 1995, respectively. Also, she

earned a M.S. degree in theoretical computer science from the Universityof Bucharest, Romania, School of Mathematics, in 1987. At IBM, her workis focused on Web server performance. At Georgia Institute of Technology,her work focused on real-time systems and adaptive systems.E-mail: [email protected]

Arun Iyengar does research into Web performance, electronic commerce,and distributed computing at IBM’s T.J. Watson Research Center. He hasPh.D. and Master’s degrees, both in computer science, from the Massa-chusetts Institute of Technology where he was a National Science Founda-tion Graduate Fellow and a BA Summa Cum Laude and with Distinction inChemistry from the University of Pennsylvania. His work on Web perfor-mance and caching has improved performance at some of the most highlyaccessed sporting and event Web sites hosted by IBM and been incorpo-rated into several IBM products. He is the National Delegate representingthe IEEE Computer Society to IFIP Technical Committee 6 on Communica-tion Systems, the Chair of IFIP Working Group 6.4 on Internet ApplicationsEngineering, and the Vice Chair for the IEEE Computer Society’s TechnicalCommittee on the Internet. He has served on program committees for sev-eral important conferences related to the Internet and is Program Co-Chairfor the 2002 International World Wide Web Conference.E-mail: [email protected]

Daniel M. Dias received the B. Tech. degree from the Indian Institute ofTechnology, Bombay, India, in 1976, and the M.S. and Ph.D. degrees fromRice University, Houston, Texas, in 1978 and 1981 respectively, all in elec-trical engineering. Dr. Dias has been with the IBM T.J. Research Center,Yorktown Heights, NY, since 1983. He manages the Parallel CommercialSystem department, which currently has projects focusing on scalable andhigh performance Internet servers, business-to-business e-Commerce, andperformance management. His recent work includes scalable and highlyavailable Web servers, high performance scalable Web caches, Web col-laboratory systems, scalable Video servers, frameworks for business-to-business e-Commerce, highly available clustered systems, and performanceanalysis. Technologies developed in these projects have been used forSporting and Event Web sites hosted by IBM, large Customer sites, andsome are now available as IBM products such as Network Dispatcher, WebCache Accelerator, HACMP ES, among others. Dr. Dias has publishedmore that 100 papers in refereed journals and conferences. He has won twobest paper awards, IBM outstanding innovation and technical achievementawards, nine invention achievement awards and Research Division awards.He holds 16 US patents, with 12 additional patents pending. He is a SeniorMember of the IEEE.E-mail: [email protected]

web proxy acceleration

Documents