cs423-cotter 1

Concurrency Issues in Client/Server Applications

Chapters 15,16, 28

cs423-cotter 2

Server Concurrency Control

• Must choose between Iterative or Concurrent– How many clients may query server?– How often will clients Query?– What mode of query will be used?– How long will service take?

cs423-cotter 3

Server Concurrency Control

• Must choose between Iterative or Concurrent– How many clients may query server?– How often will clients Query?– What mode of query will be used?– How long will service take?

• If concurrent, must choose the level of concurrency– Apparent or real concurrency– Variable number of client threads– Fixed Maximum number of threads

cs423-cotter 4

Server Concurrency Models

• Thread on demand– One thread per session– One thread per request

cs423-cotter 5

Server Concurrency Models

• Thread on demand– One thread per session– One thread per request

• Thread Pool– Create thread pool– Allocate threads to queries as needed.– When query has been served, return thread to pool– When all threads busy, queue or discard other

queries

cs423-cotter 6

Socket Pre-allocation Techniques

• Associate a thread (or a thread pool) with each different service

• For TCP, associate each thread with the service socket, such that when a query arrives, one (and only one) thread will have accept unblocked, will create a new socket for this query, handle the query, and then return to accept.

• For UDP, each request is a single datagram, which gets assigned to a single slave thread.

cs423-cotter 7

Concurrent Connection-oriented

Master

slave slave slave

Port

cs423-cotter 8

Concurrent Connectionless

Master

slave slave slave

Port

cs423-cotter 9

Delayed Concurrency in Servers

• Concurrency helpful when the cost of a new process (thread) is less than service time for the request.

• May vary by request. (Some database queries are short and others are long).

cs423-cotter 10

Delayed Concurrency in Servers

• Concurrency helpful when the cost of a new process (thread) is less than service time for the request.

• May vary by request. (Some database queries are short and others are long).

• Start service iteratively and set timer. If timer expires, create new thread to finish service.

• Start new threads based on service requested (where multiple services are available).

cs423-cotter 11

Concurrency in Clients

• Why?– Easier to program, because concurrency forces

modularity– Easier to maintain, also because of modularity– Can interact with several servers at once– Allows the user to interact with server, even when a

call is blocked.

cs423-cotter 12

Concurrency in ClientsExample

• Database query - “residents on Elm Street”• Response may be short (in Lone Jack)....

or very long (in KCMO)

cs423-cotter 13

Concurrency in ClientsExample

• Database query - “residents on Elm Street”• Response may be short (in Lone Jack)....

or very long (in KCMO)• How does the user know when to abort a “failed”

query?• Might query server for status...• Might terminate current query (if server is too

slow) and restart on another server...

cs423-cotter 14

Functional Concurrency in Clients

• Use multiple threads to perform different actions– One thread manages input (keyboard)– One thread manages input to socket– One thread manages output from socket– One thread manages control interaction with server– etc.

cs423-cotter 15

Concurrency in ClientsMultiple Servers

• Allows multiple accesses at the same time...so that performance measures (round trip delay, lost packets, etc.) are all taken against the same network environment.

• Faster than iterative model

cs423-cotter 16

Concurrency in ClientsIssues

• Concurrency Model: May use real or apparent concurrency (single threaded or multi-threaded).

• OS Support for concurrency: Some systems allow sharing memory between threads, while others may not.

• Functional Decomposition: What operations might be done in parallel, and which need to be done sequentially?

cs423-cotter 17

Concurrency in ClientsSingly threaded

• Allows memory sharing between connections• May (or may not) overload, depending on rate of

data exchange with multiple connections• May deadlock!!

cs423-cotter 18

Concurrent Client (TCPtecho) #include <stdio.h>, <string.h>, <time.h>, <winsock.h>#define BUFSIZE 4096 /* write buffer size*/#define CCOUNT 64*1024 /* default character count*/#define WSVERS MAKEWORD(2, 0)#define MIN(x, y) ((x)>(y) ? (y) : (x))#define USAGE "usage: TCPtecho [ -c count ] host1 host2...\n”struct hdat {

char *hd_name; /* host name */SOCKET hd_sock; /* host socket descriptor*/ unsigned hd_rc; /* recv character count */unsigned hd_wc; /* send character count */

} hdat[FD_SETSIZE]; /* fd to host name mapping*/char buf[BUFSIZE]; /* read/write data buffer */void TCPtecho(fd_set *, int);int reader(struct hdat *, fd_set *);void writer(struct hdat *, fd_set *);long mstime(u_long *);

cs423-cotter 19

Concurrent Client (TCPtecho)

void main(int argc, char *argv[]){

int ccount = CCOUNT, i, hcount = 0, fd;unsigned long one = 1;fd_set afds;WSADATA wsdata;if (WSAStartup(WSVERS, &wsdata))

errexit("WSAStartup failed\n");FD_ZERO(&afds);for (i=1; i<argc; ++i) {

if (strcmp(argv[i], "-c") == 0) {

if (++i < argc && (ccount = atoi(argv[i])))continue;

errexit(USAGE);} /* else, a host */

cs423-cotter 20

Concurrent Client (TCPtecho)

fd = connectTCP(argv[i], "echo");

if (ioctlsocket(fd, FIONBIO, &one))

{

fprintf(stderr,"can't mark nonblocking (host %s): %d\n", argv[i], GetLastError());

continue;

}

hdat[hcount].hd_name = argv[i];

hdat[hcount].hd_sock = fd;

hdat[hcount].hd_rc = hdat[hcount].hd_wc = ccount; ++hcount;

FD_SET(fd, &afds);

}

TCPtecho(&afds, hcount);

WSACleanup();

exit(0);}

cs423-cotter 21

Concurrent Client (TCPtecho) void TCPtecho(fd_set *pafds, int hcount)

{

fd_set rfds, wfds; /* read/write fd sets */

fd_set rcfds, wcfds; /* read/write fd sets (copy)*/

int fd, hndx, i;

for (i=0; i<BUFSIZE; ++i) /* echo data */

buf[i] = 'D';

memcpy(&rcfds, pafds, sizeof(rcfds));

memcpy(&wcfds, pafds, sizeof(wcfds));

(void) mstime((u_long *)0); /* set the epoch */

cs423-cotter 22

Concurrent Client (TCPtecho) while (hcount) {

memcpy(&rfds, &rcfds, sizeof(rfds));memcpy(&wfds, &wcfds, sizeof(wfds));if (select(FD_SETSIZE, &rfds, &wfds, (fd_set *)0,

(struct timeval *)0) == SOCKET_ERROR)errexit("select failed: error %d\n",GetLastError());

for (hndx=0; hndx<hcount; ++hndx) {fd = hdat[hndx].hd_sock;if (FD_ISSET(fd, &rfds))

if (reader(&hdat[hndx], &rcfds) == 0){/* host done*/for (i=hndx+1; i<hcount; ++i)

hdat[i-1]=hdat[i];hcount--;continue;

}if (FD_ISSET(fd, &wfds))

writer(&hdat[hndx], &wcfds);} }}

cs423-cotter 23

Concurrent Client (TCPtecho)

int reader(struct hdat *phd, fd_set *pfdset)

{

...

}

void writer(struct hdat *phd, fd_set *pfdset)

{

...

}

long mstime(u_long *pms)

{

...

}

cs423-cotter 24

Deadlock in Client/Server Systems

• A situation where computation cannot proceed because a set or 2 or more components in the system is blocked and each component is waiting on another component in the set.

• Cannot be released through an external input.• Often difficult to detect or prove, because it may

depend on a particular, unusual sequence of events.

cs423-cotter 25

Deadlock Between Client & Server

• May use request / response model to avoid deadlock.

• Must ensure that protocol design is “tight”. Remove ambiguities in the design.

• Assign responsibility for communication synchronization. (One side always starts)

cs423-cotter 26

Starvation in Client/Servers

• Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources.

• Results in unfair allocation of services

cs423-cotter 27

Starvation in Client/Servers

• Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources.

• Results in unfair allocation of services• Delays waiting for a client to transmit (blocking

on receive) can be managed through timeouts.• For busy / malicious clients, servers

multithreading or non-blocking can be used.

cs423-cotter 28

Livelock in Clients / Servers

• Like deadlock, livelock results from circular dependency among processes and resources.

• System is locked into an endless self-generating sequence of commands that ultimately consumes all system resources.

cs423-cotter 29

Livelock in Clients / Servers• Like deadlock, livelock results from circular dependency

among processes and resources. • System is locked into an endless self-generating

sequence of commands that ultimately consumes all system resources.

• Example:– File Server with time stamp logging.– Livelock occurs if a remote call to time is used– First file request comes in.

• Time stamp requires time (which generates an incoming message to file server).

• Incoming time response looks like a file request, which requires a new time stamp.....

cs423-cotter 30

Planning for Concurrency

• Consider what system resources should be consumed by the server

• Consider what the demand for the service(s) is• Provide limits (threads, processes, memory) that

are appropriate.• Incorporate logging into the server to track

program problems, congestion, etc.• Consider “livelock” problems when working with

multiple services.