Web Transfer Latency Study

Presented by Ye Xia

WebTP Presentation, Aug. 28, 2000

Paper Presented:

Paul Barford and Mark Crovella, “Critical Path Analysis of TCP Transactions”, SIGCOMM 2000

Overview

• Objective: To answer why my web transfer is slow.– Careful assignment of end-to-end delay to either network

or server.– The focus is web transaction. Can be extended to other

Internet applications.

• Methodologies:– Developed a tool, tcpeval, to analyze tcpdump

traces taken at the client and server.– Use Critical Path Analysis

• Experimental study

Critical Path for Unidirectional TCP Flow

• Data flow from server to client

• Draw Packet Dependence Graph (PDG), a weighted digraph (a tree). See figure 1.– A node represents a packet arrival or departure

at either endpoint.– An arc represents dependence relation

“happened-before”. Weight of the arc is the time difference.

Critical Path for Unidirectional TCP Flow- Possible Nodes

• Server side– a. arrival of an ACK packet; – b. departure of a data packet

• Client side– c. arrival of an data packet; – d. departure of an ACK packet

Critical Path for Unidirectional TCP Flow- Possible Arcs

• If packet not lost– Type 1: For each packet that is not dropped, draw an

arc from its departure at one endpoint to its arrival at the other endpoint. (server-to-client, client-to-server)

b c d aor

– Type 2: For each ACK packet, find the data packet containing the last byte being acked. Construct an arc from the arrival of that data packet to the departure of that ACK packet. (at client side)

c d

Critical Path for Unidirectional TCP Flow- Possible Arcs (cont)

– Type 3: For each ACK packet, determine the data packets it “liberated” and construct an arc from the arrival of the ACK to the departure of each liberated data packets. (at server side, one-to-many)

• If the packet is lost– Type 4: Construct an arc from the departure of the

dropped packet to the departure of the earliest subsequent retransmission. (at server side)

a b

Critical Path Definition

• The longest path in the tree starting from the root.• A weighted chain of dependence relations running

from the connection’s first data packet to its last data packet sent.

• Motivation: reducing the execution times of activities on the critical path will certainly reduce overall response time, while this is not necessarily true for activities off the critical path.

• Events on the critical path are the “right” ones for examining the sources of delay (rather than all events during the entire transfer).

Critical Path for an HTTP Transaction

• Dependencies among events also exist at application level.• For HTTP/1.0, dependency only exists between the last

packet comprising the HTTP GET and the first data packet sent in response.

• For generality, the critical path of the entire transfer is constructed in stages.

• Four stages for HTTP/1.0 (see figure 2.): 1. connection establishments; 2. packets sequence delivering HTTP GET;3. data transfer in response to GET (file size dependent);4. connection tear-down.

• Only 3 is file size dependent. We focus on 3.

Critical Path Discovery Algorithm

• Forward pass over the packet trace to construct rounds, one for each ACK packet. (See figure 1)

• Each round consists of the ACK and the data packets liberated by that ACK.

• Backward trace on the tree-of-rounds finds the critical path, starting from the last data packet. (See figure 3.)

Profiling Critical Path

• Identify sources of delay for the critical path by categorizing arcs– Data-data or ACK-ACK arc between two endpoints (type

1): network delay• Propagation delay• Network variation (queueing) delay

– Data-ACK arc (type 2): client delay– ACK-data arc (type 3): server delay– Data-data on the same endpoint (type 4): packet loss delay

• Fast retransmission delay• Coarse-grained timeout delay

Experimental Setup

• Server site: Boston Univ.– Apache server on Linux or FreeBSD– Server load controlled by SURGE Web load generator:

high or low load– One PC collect TCP packet traces– One PC measures network conditions

• Client sites: Univ. of Denver and Harvard Univ.• Network load: high or low depending on time of day• File sizes requested by clients: small (1KB), medium

(20KB) and large (500KB)

Results – File Sizes

• Propagation delay is often the most important contributor to transaction delay (See figure 4). It is determined by the network bandwidth and congestion control scheme.

• Small files: in high server load, server delays can dominate.

• Medium files: in high server load, propagation delay and server delay are comparable.

• Large files: network delay dominates overall transaction delay regardless of server load.

Results – High Server Load

• For small and medium files, nearly all server delay occurs between the receipt of the HTTP GET at the server and the generation of the first data packet. (See figure 5)

• For large files, server delay after the connection startup is also important. (See figure 4)

Results – Network Load

• For small and medium files, propagation delay is independent of network load or server load. (strange)

• For large files, propagation delay are higher under high network load. There are more round-trips in the critical path due to more coarse-grained timeouts.

• Network queueing delay is less important than propagation delay.

Results – Coarse-Grained Timeouts

• Coarse-grained timeouts occur more frequently than fast retransmission.

Results – Delay Variability

• For large files and heavy network load, the variability of propagation delay contributes the most to the overall variability of delay.

• This means the number of packets on the critical path shows high variability under the above situation. (See figure 8)

Highlights

• Idea: attribute the overall transaction delay to different sources.

• Tool developed: tcpeval• Methodologies

– Theoretical: critical path analysis– Experimental: tcpdump trace adequate? WAN

setup representative? Server load or network load control adequate?

Possible Limitations

• Client application is not considered. – The time from the receipt of response data at

the transport to the its receipt at the application is not considered.

– Other delays in the browser?

Previous Works By the Authors

• Mark E. Crovella and Azer Bestavros, ``Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes,'' in IEEE/ACM Transactions on Networking, 5(6):835--846, December 1997.

• P. Barford and M. E. Crovella, "Generating Representative Web Workloads for Network and Server Performance Evaluation," in Proceedings of Performance '98/ACM SIGMETRICS '98, pp. 151-160, Madison WI. Slightly expanded version appears as BUCS-TR-1997-006, November 4, 1997.

• M. E. Crovella, R. Frangioso, and M. Harchol-Balter, "Connection Scheduling in Web Servers," in Proceedings of the 1999 USENIX Symposium on Internet Technologies and Systems (USITS '99), Boulder, Colorado, October 1999.

Web Site: http://www.cs.bu.edu/fac/crovella/

Other Sources of Web Transaction Latency

• Cohen and Kaplan shows– DNS lookups exceed 3 seconds for over 10% of

servers.

– Cold server delay: response times for start-of-session HTTP requests are significantly longer than subsequent requests. (They exceed 1 second for 10% of servers and 4 second for 6% of servers.)

– TCP connection establishment delay is significant

– Cold route delay: first IP datagram is more likely to travel for a longer time than subsequent datagrams.

Their Solutions

• Pre-resolving

• Pre-connecting

• Pre-warming