concurrent programming - kaistina.kaist.ac.kr/ee324/f13/lectures/05-select.pdf · 4 concurrent...

32
CONCURRENT PROGRAMMING Lecture 5

Upload: others

Post on 03-Aug-2020

32 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

CONCURRENT PROGRAMMING

Lecture 5

Page 2: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

2

Last lecture

TCP I/O

RIO: buffered I/O. Unix IO

Question: difference between send() and write()? MSG_DONTWAIT, MSG_NOSIGNAL

Page 3: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

3

Concurrent Programming is Hard!

The human mind tends to be sequential The notion of time is often misleading Thinking about all possible sequences of events in a

computer system is at least error prone and frequently impossible

Page 4: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

4

Concurrent Programming is Hard!

Classical problem classes of concurrent programs: Races: outcome depends on arbitrary scheduling decisio

ns elsewhere in the system Example: who gets the last seat on the airplane?

Deadlock: improper resource allocation prevents forward progress Example: traffic gridlock

Livelock / Starvation / Fairness: external events and/or system scheduling decisions can prevent sub-task progress Example: people always jump in front of you in line

Page 5: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

5

Client / ServerSession

Iterative Echo ServerClient Serversocket socket

bind

listen

rio_readlineb

rio_writenrio_readlineb

rio_writen

Connectionrequest

rio_readlineb

close

close EOF

Await connectionrequest fromnext client

open_listenfd

open_clientfd

acceptconnect

Page 6: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

6

Iterative Servers

Iterative servers process one request at a timeclient 1 server client 2

connect

accept connect

write read

call read

close

accept

write

read

close

Wait for Client 1

call read

write

ret read

writeret read

Page 7: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

7

Where Does Second Client Block?

Second client attempts to connect to iterative server

Call to connect returns Even though connection not yet

accepted Server side TCP manager que

ues request Feature known as “TCP listen

backlog”

Call to rio_writen returns Server side TCP manager buff

ers input data

Call to rio_readlineb blocks Server hasn’t written anything

for it to read yet.

Clientsocket

rio_readlineb

rio_writen

Connectionrequest

open_clientfd

connect

Page 8: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

8

Fundamental Flaw of Iterative Servers

Solution: use concurrent servers instead Concurrent servers use multiple concurrent flows to serve multip

le clients at the same time

User goesout to lunch

Client 1 blockswaiting for userto type in data

Client 2 blockswaiting to read from server

Server blockswaiting fordata fromClient 1

client 1 server client 2connect

accept connect

write read

call readwrite

call readwriteret read

Page 9: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

9

Creating Concurrent Flows Allow server to handle multiple clients simultaneously

1. Processes Kernel automatically interleaves multiple logical flows Each flow has its own private address space

2. Threads Kernel automatically interleaves multiple logical flows Each flow shares the same address space

3. I/O multiplexing with select() Programmer manually interleaves multiple logical flows All flows share the same address space Relies on lower-level system abstractions

Page 10: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

10

Concurrent Servers: Multiple Processes Spawn separate process for each client

client 1 server client 2

call connect call accept

call read

ret connectret accept

call connect

call fgetsforkchild 1

User goesout to lunch

Client 1 blockswaiting for user to type in data

call acceptret connect

ret accept call fgets

writefork

call read

child 2

write

call read

end readclose

close

...

Page 11: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

11

Review: Iterative Echo Server

Accept a connection request Handle echo requests until client terminates

int main(int argc, char **argv) {

int listenfd, connfd;int port = atoi(argv[1]);struct sockaddr_in clientaddr;int clientlen = sizeof(clientaddr);

listenfd = Open_listenfd(port);while (1) {

connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen);echo(connfd);Close(connfd);

}exit(0);

}

Page 12: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

12

int main(int argc, char **argv) {

int listenfd, connfd;int port = atoi(argv[1]);struct sockaddr_in clientaddr;int clientlen=sizeof(clientaddr);

Signal(SIGCHLD, sigchld_handler);listenfd = Open_listenfd(port);while (1) {

connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen);if (Fork() == 0) {

Close(listenfd); /* Child closes its listening socket */echo(connfd); /* Child services client */Close(connfd); /* Child closes connection with client */exit(0); /* Child exits */

}Close(connfd); /* Parent closes connected socket (important!) */

}}

Process-Based Concurrent ServerFork separate process for

each clientDoes not allow any

communication between different client handlers

Page 13: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

13

Process-Based Concurrent Server(cont)

Reap all zombie children

void sigchld_handler(int sig) {

while (waitpid(-1, 0, WNOHANG) > 0);

return;}

Page 14: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

14

Process Execution Model

Each client handled by independent process No shared state between them Both parent & child have copies of listenfd and connfd Parent must close connfd Child must close listenfd

Client 1Server

Process

Client 2Server

Process

ListeningServer

Process

Connection Requests

Client 1 data Client 2 data

Page 15: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

15

Concurrent Server: accept Illustratedlistenfd(3)

Client1. Server blocks in accept, waiting for connection request on listening descriptor listenfd

clientfd

Server

listenfd(3)

Client

clientfd

Server2. Client makes connection request by calling and blocking in connect

Connectionrequest

listenfd(3)

Client

clientfd

Server3. Server returns connfd from accept. Forks child to handle client. Client returns from connect. Connection is now established between clientfd and connfd

ServerChild

connfd(4)

Page 16: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

16

Implementation Must-dos With Process-Based Designs Listening server process must reap zombie children

to avoid fatal memory leak

Listening server process must close its copy of connfd Kernel keeps reference for each socket/open file After fork, refcnt(connfd) = 2 Connection will not be closed until refcnt(connfd) == 0

Page 17: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

17

View from Server’s TCP ManagerClient 1 ServerClient 2

cl1> ./echoclient greatwhite.ics.cs.cmu.edu 15213

srv> ./echoserverp 15213

srv> connected to (128.2.192.34), port 50437

cl2> ./echoclient greatwhite.ics.cs.cmu.edu 15213

srv> connected to (128.2.205.225), port 41656

Connection Host Port Host Port

Listening --- --- 128.2.220.10 15213

cl1 128.2.192.34 50437 128.2.220.10 15213

cl2 128.2.205.225 41656 128.2.220.10 15213

Page 18: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

18

Pros and Cons of Process-Based Designs

+ Handle multiple connections concurrently + Clean sharing model

file tables (yes) global variables (no)

+ Simple and straightforward – Additional overhead for process control – Nontrivial to share data between processes

Requires IPC (interprocess communication) mechanisms FIFO’s (named pipes), System V shared memory and semaphores

Page 19: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

19

#2) Event-Based Concurrent Servers Using I/O Multiplexing

Use library functions to construct scheduler within single process

Server maintains set of active connections Array of connfd’s

Repeat: Determine which connections have pending inputs If listenfd has input, then accept connection Add new connfd to array

Service all connfd’s with pending inputs

Page 20: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

20

Adding Concurency: Step One

Start with allowing address re-use

Then we set the socket to non-blocking

int sock, opts;sock = socket(…);// getting the current optionssetsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts));

// getting current optionsif (0 > (opts = fcntl(sock, F_GETFL)))

printf(“Error…\n”);// modifying and applyingopts = (opts | O_NONBLOCK);if (fcntl(sock, F_SETFL, opts))

printf(“Error…\n”);bind(…);

Page 21: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

21

Adding Concurrency: Step Two

Monitor sockets with select() int select(int maxfd, fd_set *readfds, fd_set

*writefds, fd_set *exceptfds, const struct timespec *timeout);

So what’s an fd_set? Bit vector with FD_SETSIZE bits

maxfd – Max file descriptor + 1 readfs – Bit vector of read descriptors to monitor writefds – Bit vector of write descriptors to monitor exceptfds – Read the manpage, set to NULL timeout – How long to wait with no activity before retur

ning, NULL for eternity

Page 22: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

22

How does the code change?

if (listen(sockfd, 5) < 0) { // listen for incoming connectionsprintf(“Error listening\n”);

init_pool(sockfd, &pool);while (1) {

pool.ready_set = &pool.read_set;pool.nready = select(pool.maxfd+1, &pool.ready_set,

&pool.write_set, NULL, NULL);if (FD_ISSET(sockfd, &pool.ready_set)) {

if (0 > (isock = accept(sockfd,…))) printf(“Error accepting\n”);

add_client(isock, &caddr, &pool);}check_clients(&pool);

}// close it up down here

Page 23: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

23

What was pool?

A struct something like this:

typedef struct s_pool {int maxfd; // largest descriptor in setsfd_set read_set; // all active read descriptorsfd_set write_set; // all active write descriptorsfd_set ready_set; // descriptors ready for readingint nready; // return of select()int maxi; /* highwater index into client array */int clientfd[FD_SETSIZE]; /* set of active descriptors */rio_t clientrio[FD_SETSIZE]; /* set of active read buffers */

} pool;

Page 24: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

24

void init_pool(int sockfd, pool * p);{

int i;p->maxfd = -1;for (i=0;i<FD_SETSIZE;i++)

p->clientfd[i]=-1;p->maxfd = sockfd;FD_ZERO(&p->read_set);FD_SET(listenfd, &p->read_set);

}// close it up down here

Page 25: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

25

void add_client(int connfd, pool *p) {

int i;p->nready--;for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */

if (p->clientfd[i] < 0) { /* Add connected descriptor to the pool */p->clientfd[i] = connfd; Rio_readinitb(&p->clientrio[i], connfd);

/* Add the descriptor to descriptor set */FD_SET(connfd, &p->read_set);

/* Update max descriptor and pool highwater mark */if (connfd > p->maxfd)

p->maxfd = connfd; if (i > p->maxi)

p->maxi = i; break;

}if (i == FD_SETSIZE) /* Couldn't find an empty slot */

app_error("add_client error: Too many clients");}// close it up down here

Page 26: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

26

void check_clients(pool *p) {

int i, connfd, n;char buf[MAXLINE]; rio_t rio;

for (i = 0; (i <= p->maxi) && (p->nready > 0); i++) {connfd = p->clientfd[i];rio = p->clientrio[i];

/* If the descriptor is ready, echo a text line from it */if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) {

p->nready--;if ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) {

byte_cnt += n; //line:conc:echoservers:beginechoprintf("Server received %d (%d total) bytes on fd %d\n",

n, byte_cnt, connfd);Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho

}

/* EOF detected, remove descriptor from pool */else {

Close(connfd); //line:conc:echoservers:closeconnfdFD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremovep->clientfd[i] = -1; //line:conc:echoservers:endremove

}}

}}

Page 27: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

27

So what about bit vectors?

void FD_ZERO(fd_set *fdset);

Clears all the bits void FD_SET(int fd, fd_set *fdset);

Sets the bit for fd void FD_CLR(int fd, fd_set *fdset);

Clears the bit for fd int FD_ISSET(int fd, fd_set *fdset);

Checks whether fd’s bit is set

Page 28: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

28

What about checking clients?

The code only tests for new incoming connections But we have many more to test!

Store all your client file descriptors In pool is a good idea!

Several scenarios Clients are sending us data We may have pending data to write in a buffer

Keep the while(1) thin Delegate specifics to functions that access the appropriate d

ata Keep it orthogonal!

Page 29: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

29

Back to that lifecycle…

ServerClient(s)socket()

connect()

write()read()

close()

socket()

bind()

listen()

select()

write()

read()

close()

read()EOF

Connection Request

Client / Server Session(s)

FD_ISSET(sfd)

accept()

check_clients() main loop

Page 30: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

30

I/O Multiplexed Event Processing

10

clientfd

74-1-1125-1-1-1

0123456789

Active

Inactive

Active

Never Used

listenfd = 3

10

clientfd

74-1-1125-1-1-1

listenfd = 3 Active Descriptors Pending Inputs

Read

Page 31: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

31

Time out?

int select(int nfds, fd_set *readfds, fd_set *writefds,fd_set *exceptfds, struct timeval *timeout);

Page 32: CONCURRENT PROGRAMMING - KAISTina.kaist.ac.kr/ee324/F13/lectures/05-select.pdf · 4 Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome

32

Pros and Cons of I/O Multiplexing

+ One logical control flow. + Can single-step with a debugger. + No process or thread control overhead.

Design of choice for high-performance Web servers and search engines.

– Significantly more complex to code than process- or thread-based designs.

– Hard to provide fine-grained concurrency E.g., our example will hang up with partial lines.

– Cannot take advantage of multi-core Single thread of control