jigsaw performance analysis ------ potential bottlenecks

Jigsaw Performance Analysis

------ Potential Bottlenecks

June 28, 2000CHEN Ge CSIS HKU

Core Components of Jigsaw

httpd

SocketClientFactory(The Thread Pool)

serverpor:8001

Incoming Requests

ResourceStoreManage(Resource Cache)

Jigsaw



The HTTP Daemon– The HTTP Daemon in Jigsaw is an instance of the

httpd Class– The httpd Object contains a Thread Pool called

SocketClientFactory, which actually handle the incoming requests

– The httpd Object also contains a resource cache called ResourceStoreManager



The SocketClientFactory– The SocketClientFactory contains a pool of threads

(default is 45 threads) to be bound to incoming request

– SocketClientFactory is also responsible for maintaining the server’s load information



The ResourceStoreManager– Jigsaw use the concept resource refer to all the objects

accessible through the Web Server– A resource maybe as simple as file in the file system. It also

may represent some dynamic content generating program like CGI scripts, servlets, etc.

– ResourceStoreManage stores referred resource in StoreEntry objects for caching purpose. When the server is shutdown, it will deploy the resources to persistent storage for reloading at the next startup time


Request Handling Stages

SocketClient Thread Bind– The httpd runs in an infinite loop waiting for

incoming socket connection– When a socket connect comes, httpd will pass the

incoming SocketClient to SocketClientFactory to find out a thread in the thread pool to handle the request

– The httpd thread will block until the SocketClientFactory either find a thread to handle the request or reject the request



public void run () {…while ( ( ! finishing) && ( socket != null ) ) { Socket ns = null ; try {

ns = socket.accept() ;ns.setTcpNoDelay(true);

} catch (IOException e) { … } if ( (socket != null) && (ns != null) && (factory != null) )

factory.handleConnection (ns) ;}// Our socket has been closed, perform associated cleanup.cleanup(restarting) ;

}

httpd

factory

manager



SoketClient Thread Bind

– The SocketClientFactory will try to assign an thread from the pool to the incoming SocketClient. if server exceeds the predefined maximum load, the incoming request will be rejected(No response)

– During the binding procedure, the SocketClientFactory will update many shared variables in the pool to refresh the server’s status through synchronized methods



Resource Lookup– When the SocketClientFactory successfully binds a

thread to a incoming socket connection, the bound thread will proceed to handle the request

– When it get the request content, for example, a URI for a html document, the thread will begin a complex procedure of ‘Resource Lookup’

– The previous requested URIs are stored in the ResourceStoreManager



Resource Emitting– After getting the requested resource information

from the lookup stages, the thread gets all the necessary information about the resource requested

– After it gets the necessary information, the thread emit the reply via socket


Major Methods Called

org.w3c.jigsaw.httpd.run() --- the main thread public void run () {

…while ( ( ! finishing) && ( socket != null ) ) { Socket ns = null ; try {

ns = socket.accept() ;ns.setTcpNoDelay(true);

} catch (IOException e) {…

} if ( (socket != null) && (ns != null) && (factory != null) )

factory.handleConnection (ns) ; //This the Thread Binding process }

…. }



org.w3c.jigsaw.http.socket.SocketClientFacotory.handleConnection() --- Thread Binding

public void handleConnection (Socket socket) {…//Try to adjust client pool status according to the server’s loadswitch(loadavg) { case AVG_LIGHT: // Free list is non empty, be fast: if ( decrFreeCount() )

cs = (SocketClientState) freeList.removeTail(); break; case AVG_NORMAL: case AVG_HIGH: // Free list is non empty, but we try killing a client: killSomeClients(); if ( decrFreeCount() )

cs = (SocketClientState) freeList.removeTail(); break; case AVG_DEAD: break;}


org.w3c.jigsaw.http.socket.SocketClientFacotory.handleConnection() --- Thread Binding

…

if ( cs != null ) {

…

cs.status = SocketClientState.C_BUSY;

cs.client.bind(socket);

} else {

…

try {

socket.close();

} catch (IOException ex) {

}

server.errlog(socket.getInetAddress()+" refused (overloaded).");

}

return;

}



Org.w3c.jigsaw.http.Client.startconnection() --- the abstract class define the basic request handling interface

protected boolean startConnection(InputStream in, DataOutputStream out) throws ClientException

{ // Process request, and time it:

tstart = System.currentTimeMillis() ;reply = processRequest (request) ; tend = System.currentTimeMillis() ;

//By Robin - 19/06/2000 lookupTime += tend-tstart; numOfLookupRequest ++; //End of By Robin

…sent = emitReply(reply) ;…

}



org.w3c.jigsaw.httpd.perform() --- looking up the resource– This method is where Jigsaw try to find out the

requested resource through a serials of lookup process

– The previous introduced method Client.startconnection() calls this method through Client.processRequest() method


org.w3c.jigsaw.httpd.perform() --- looking up the resource

public ReplyInterface perform(RequestInterface req) throws ProtocolException, ResourceException{

…

// Create a lookup state, and a lookup result:

LookupState ls = new LookupState(request);

LookupResult lr = new lookupResult(root.getResourceReference());

// Run the lookup algorithm of root resource:

try {

if ( root.lookup(ls, lr) ) {

if (lr.hasReply())

return lr.getReply();

}

} catch (ProtocolException ex) {

…

}

…


org.w3c.jigsaw.httpd.perform() --- looking up the resource

// Let the target resource perform the method

ResourceReference target = lr.getTarget();

Reply reply = null;

…

//perform the request:

if ((error == null) && (target != null)) {

request.setFilters(filters, infilter);

request.setTargetResource(target);

try {

FramedResource res = (FramedResource)target.lock();

reply = (Reply)res.perform(request);

…

}

return reply;

}


How do we test the Performance Breakdown

Insert some System.currentTimeMillis() call in the methods introdueced above

Divide the whole request process time into three stages: thread binding time, resource lookup time, reply emitting time


Thread Binding Time

In httpd.run() //By Robin - 19/06/2000 {

long tstart = 0; long tend = 0; tstart = System.currentTimeMillis(); factory.handleConnection (ns) ; tend = System.currentTimeMillis(); requestNum ++; totalHandleTime = totalHandleTime + (tend-tstart); } /* Original Code Before By Robin - 19/06/2000 factory.handleConnection (ns) ; */ //end of By Robin - 19/06/2000


Resource Lookup Time

In order to avoid the synchronization overhead during the timing process, all the threads first calculate their own values, then, the server will collect all the values of each thread every 6 (predefined, changeable) minutes



For individual thread, in Client.startConnect()

// Process request, and time it:

tstart = System.currentTimeMillis() ;

reply = processRequest (request) ;

tend = System.currentTimeMillis() ;

//By Robin - 19/06/2000

lookupTime += tend-tstart;

numOfLookupRequest ++;

//End of By Robin



For the whole value, in SocketClientFactory.doBreakdownLog(), it get each value of the individual thread, and calculate out the average Lookup Time by divide the sum of all the threads lookup time by the number of total lookup processed


SocketClientFactory.doBreakdownLog()

SocketClientState cs = csList;

while (cs != null)

{

…

totalHandleTime += cs.totalHandleTime;

totalLookupTime += cs.totalLookupTime;

totalEmitTime += cs.totalEmitTime;

totalLookupRequest += cs.numOfLookupRequest;

totalRequest += cs.numOfRequest;

totalEmittedRequest += cs.numOfEmittedRequest;

cs = cs.csnext;

}


SocketClientFactory.doBreakdownLog()

if (totalRequest!= 0) { aveHandleTime = totalHandleTime/totalRequest; } if (totalLookupRequest != 0) { aveLookupTime = totalLookupTime/totalLookupRequest; aveHandleTimeByLookupNum = totalHandleTime/totalLookupRequest; } if (totalEmittedRequest != 0) { aveEmitTime = totalEmitTime/totalEmittedRequest; }


Reply Emitting Time

The Reply Emitting Time is tested similarly with that of Lookup Time, in Client.startConnection() …

tend = System.currentTimeMillis() ;

…

// Inital keep alive check, emit the reply:

…

//By Robin - 20/06/2000

tendEmit = System.currentTimeMillis();

emitTime += tendEmit-tend;

numOfEmittedRequest ++;

//End of By Robin


Performance Breakdown

Average Bind Time and Lookup Time

0

100

200

300

400

500

600

700

1c-3t

4c-3t-a

4c-3t-b

4c-3t-c

8c-3t-a

8c-3t-b

8c-6t-a

8c-6t-b

8c-6t-c

Ave Bind Time

Ave LookupTime



Total Connection Replied / Total Connection

0

500

1000

1500

2000

2500

3000

3500

1c-3t

4c-3t-a

4c-3t-b

4c-3t-c

8c-3t-a

8c-3t-b

8c-6t-a

8c-6t-b

8c-6t-c

Total Connection

Handled Request

HR/TC %

0%

20%

40%

60%

80%

100%

120%

1 4 4 4 8 8 8 8 8

1c-3t 4c-3t-a

4c-3t-b

4c-3t-c

8c-3t-a

8c-3t-b

8c-6t-a

8c-6t-b

8c-6t-c

HR/TC %



Individual Process Stage Time/ Total Handle Time

0%

20%

40%

60%

80%

100%

120%

1c-3t

4c-3t-a

4c-3t-b

4c-3t-c

8c-3t-a

8c-3t-b

8c-6t-a

8c-6t-b

8c-6t-c

HR/TC %

AveBind/AveHand %

AveLookup/AveHand%

AveEmitTime/AveHand %



Individual Process Stage Time/ Total Handle Time

0% 50% 100%

1c-3t

4c-3t-a

4c-3t-b

4c-3t-c

8c-3t-a

8c-3t-b

8c-6t-a

8c-6t-b

8c-6t-c

Ave Bind Time

Ave LookupTime

Ave Emit Time



– The Bind Time and Lookup Time increases significantly as the load increases

– Except for the strange behavior of Jigsaw when the workload becomes very heavy, at which point the emit time suddenly increases dramatically, the Lookup Time counts for more than half of the total process time

– The bind time is very irregular


Problems

– Jigsaw sometimes hangs when the workload become heavy, even not that heavy as the most one in our test

– Traceinfo shows that the loopup stage have some problem, it generate a lot excepts and errors when the workload increase, and Jigsaw will unable to find some really exists resources

– The emit time become unexpectedly high when the work load very heavy, is it the problem of underlying socket problem? Still do no clear how Java implement the Socket interface?

– Our previous speculation that the thread pool may be a potential bottleneck seems partially correct, if the irregular behavior of Binding Time can be explained by this

– The loopup stage (including the cache) seems another bottleneck in Jigsaw from the tested data

jigsaw performance analysis ------ potential bottlenecks

Documents