Non Blocking Socket

Hu Zi Ming

hzmangel@gmail.com

201l-08-28

1 / 16Non Blocking Socket

N

Outline

1 Normal ServerFunction FlowSimple Stream Server

2 The C10K Problem

3 High Performance Web Server

2 / 16Non Blocking Socket

N

Function Flow

S: Get available address

S: Create the socket file descriptor

S: Bind the descriptor to specified port

S: Listen on the port (address)

C: Create the socket

C: Connect to the server

S: Accept the connection

Data exchange

3 / 16Non Blocking Socket

N

Function Flow

S: Get available address

S: Create the socket file descriptor

S: Bind the descriptor to specified port

S: Listen on the port (address)

C: Create the socket

C: Connect to the server

S: Accept the connection

Data exchange

3 / 16Non Blocking Socket

N

Function Flow

S: Get available address

S: Create the socket file descriptor

S: Bind the descriptor to specified port

S: Listen on the port (address)

C: Create the socket

C: Connect to the server

S: Accept the connection

Data exchange

3 / 16Non Blocking Socket

N

Function Flow

S: Get available address

S: Create the socket file descriptor

S: Bind the descriptor to specified port

S: Listen on the port (address)

C: Create the socket

C: Connect to the server

S: Accept the connection

Data exchange

3 / 16Non Blocking Socket

N

Simple Stream Server

server.py

import socket

PORT = 8080HTTP_RESP = ’HTTP/1.1 200 OK\r\n’HTTP_RESP += ’Connection:close1\r\n’HTTP_RESP += ’Content-Type: text/html\r\n’HTTP_RESP += ’Content-Length: 21\r\n’HTTP_RESP += ’\r\n<h1>hello, world</h1>’

available_sock = socket.getaddrinfo(None, PORT, 0, socket.SOCK_STREAM)[0]s = socket.socket(available_sock[0], available_sock[1])s.bind((available_sock[3], PORT))print ’Binded at’, available_sock[4][0]s.listen(10)

while True:conn, addr = s.accept()print ’Connected by’, addrconn.send(HTTP_RESP)conn.close()

4 / 16Non Blocking Socket

N

Outline

1 Normal Server

2 The C10K ProblemThe C10K ProblemProgramming Model

3 High Performance Web Server

5 / 16Non Blocking Socket

N

Huston, We Have a Problem

As web developed in high speed

More servers need to handle 10 thousand clients simultaneously

Which names this problem: C10K

Classical server can not handle this scenario well

Here comes the new models

6 / 16Non Blocking Socket

N

Huston, We Have a Problem

As web developed in high speed

More servers need to handle 10 thousand clients simultaneously

Which names this problem: C10K

Classical server can not handle this scenario well

Here comes the new models

6 / 16Non Blocking Socket

N

Huston, We Have a Problem

As web developed in high speed

More servers need to handle 10 thousand clients simultaneously

Which names this problem: C10K

Classical server can not handle this scenario well

Here comes the new models

6 / 16Non Blocking Socket

N

Programming Model

One client with each server thread, synchronous blocking IO.

Classical model, used in Apache

Many clients with each thread

Asynchronous blocking IO and level-triggered readinessnotification

Many clients with each thread

Asynchronous blocking IO and edge-triggered readinessnotification

Many clients with each thread

Asynchronous non-blocking IO.

7 / 16Non Blocking Socket

N

Programming Model

One client with each server thread, synchronous blocking IO.

Classical model, used in Apache

Many clients with each thread

Asynchronous blocking IO and level-triggered readinessnotification

Many clients with each thread

Asynchronous blocking IO and edge-triggered readinessnotification

Many clients with each thread

Asynchronous non-blocking IO.

7 / 16Non Blocking Socket

N

Programming Model

One client with each server thread, synchronous blocking IO.

Classical model, used in Apache

Many clients with each thread

Asynchronous blocking IO and level-triggered readinessnotification

Many clients with each thread

Asynchronous blocking IO and edge-triggered readinessnotification

Many clients with each thread

Asynchronous non-blocking IO.

7 / 16Non Blocking Socket

N

Programming Model

One client with each server thread, synchronous blocking IO.

Classical model, used in Apache

Many clients with each thread

Asynchronous blocking IO and level-triggered readinessnotification

Many clients with each thread

Asynchronous blocking IO and edge-triggered readinessnotification

Many clients with each thread

Asynchronous non-blocking IO.

7 / 16Non Blocking Socket

N

Outline

1 Normal Server

2 The C10K Problem

3 High Performance Web ServerSynchronous Blocking IO with Multi Thread/ProcessLevel-triggered (LT) Readiness NotificationEdge-triggered (ET) Readiness NotificationAsynchronous Non-Blocking IO

8 / 16Non Blocking Socket

N

Synchronous Blocking IO w/MultiThread/Process

Easy for programming

Can be used for complex logic

Good for interactive and persistent connection

Consume too much memory

9 / 16Non Blocking Socket

N

Synchronous Blocking IO w/MultiThread/Process

Easy for programming

Can be used for complex logic

Good for interactive and persistent connection

Consume too much memory

9 / 16Non Blocking Socket

N

Level-triggered (LT) Readiness Notification

Triggered by state

Notifications will be constantly sent if state not change

select(2) and poll(2) are working in this mode

epoll(4) can be configured to work in this mode

Support both blocking and non-blocking socket

10 / 16Non Blocking Socket

N

Level-triggered (LT) Readiness Notification

Triggered by state

Notifications will be constantly sent if state not change

select(2) and poll(2) are working in this mode

epoll(4) can be configured to work in this mode

Support both blocking and non-blocking socket

10 / 16Non Blocking Socket

N

select() & poll()

Both are working in LT mode

Max acceptable fd num of select() is FD SETSIZE

Which is defined in the kernel

User defined array is used in poll() to break this limitation

The whole fd set will be scanned, and. . .

The whole fd set will be copied between user and kernel mode

So, the performance will be reduced as set increased

11 / 16Non Blocking Socket

N

select() & poll()

Both are working in LT mode

Max acceptable fd num of select() is FD SETSIZE

Which is defined in the kernel

User defined array is used in poll() to break this limitation

The whole fd set will be scanned, and. . .

The whole fd set will be copied between user and kernel mode

So, the performance will be reduced as set increased

11 / 16Non Blocking Socket

N

select() & poll()

Both are working in LT mode

Max acceptable fd num of select() is FD SETSIZE

Which is defined in the kernel

User defined array is used in poll() to break this limitation

The whole fd set will be scanned, and. . .

The whole fd set will be copied between user and kernel mode

So, the performance will be reduced as set increased

11 / 16Non Blocking Socket

N

select() & poll()

Both are working in LT mode

Max acceptable fd num of select() is FD SETSIZE

Which is defined in the kernel

User defined array is used in poll() to break this limitation

The whole fd set will be scanned, and. . .

The whole fd set will be copied between user and kernel mode

So, the performance will be reduced as set increased

11 / 16Non Blocking Socket

N

select() & poll()

Both are working in LT mode

Max acceptable fd num of select() is FD SETSIZE

Which is defined in the kernel

User defined array is used in poll() to break this limitation

The whole fd set will be scanned, and. . .

The whole fd set will be copied between user and kernel mode

So, the performance will be reduced as set increased

11 / 16Non Blocking Socket

N

Edge-triggered (ET) Readiness Notification

Triggered by state change

Notification will only be sent out when state change

Recommended mode for epoll(4)

Support only non-blocking socket

12 / 16Non Blocking Socket

N

Edge-triggered (ET) Readiness Notification

Triggered by state change

Notification will only be sent out when state change

Recommended mode for epoll(4)

Support only non-blocking socket

12 / 16Non Blocking Socket

N

epoll()

Default mode is LT, can be configured to ET

Maximum acceptable fd num is the up limit of opened file

Only handle activate connections

So performance is better in normal scenario

Return fd id while trigged by event

So no need to traverse the whole fd set

Shared memory is used to avoid copying

13 / 16Non Blocking Socket

N

epoll()

Default mode is LT, can be configured to ET

Maximum acceptable fd num is the up limit of opened file

Only handle activate connections

So performance is better in normal scenario

Return fd id while trigged by event

So no need to traverse the whole fd set

Shared memory is used to avoid copying

13 / 16Non Blocking Socket

N

epoll()

Default mode is LT, can be configured to ET

Maximum acceptable fd num is the up limit of opened file

Only handle activate connections

So performance is better in normal scenario

Return fd id while trigged by event

So no need to traverse the whole fd set

Shared memory is used to avoid copying

13 / 16Non Blocking Socket

N

epoll()

Default mode is LT, can be configured to ET

Maximum acceptable fd num is the up limit of opened file

Only handle activate connections

So performance is better in normal scenario

Return fd id while trigged by event

So no need to traverse the whole fd set

Shared memory is used to avoid copying

13 / 16Non Blocking Socket

N

Asynchronous Non-Blocking IO

Not yet become popular in Unix

The programming model is different with classical

Almost the ONLY and BEST solution on Windowz machine

But only file system IO is supported on Linux (?)

14 / 16Non Blocking Socket

N

Asynchronous Non-Blocking IO

Not yet become popular in Unix

The programming model is different with classical

Almost the ONLY and BEST solution on Windowz machine

But only file system IO is supported on Linux (?)

14 / 16Non Blocking Socket

N

AIO

Overlapping process with IO

The READ request return immediately

Which indicate the request has been initiated successfully

The operation will be performed in background

After READ response ready

A signal or thread-based callback will be generated

15 / 16Non Blocking Socket

N

AIO

Overlapping process with IO

The READ request return immediately

Which indicate the request has been initiated successfully

The operation will be performed in background

After READ response ready

A signal or thread-based callback will be generated

15 / 16Non Blocking Socket

N

AIO

Overlapping process with IO

The READ request return immediately

Which indicate the request has been initiated successfully

The operation will be performed in background

After READ response ready

A signal or thread-based callback will be generated

15 / 16Non Blocking Socket

N

AIO

Overlapping process with IO

The READ request return immediately

Which indicate the request has been initiated successfully

The operation will be performed in background

After READ response ready

A signal or thread-based callback will be generated

15 / 16Non Blocking Socket

N

Q AND A

16 / 16Non Blocking Socket

N