lecture 15: simple java web server

COMP 150-CCPCOMP 150-CCPConcurrent ProgrammingConcurrent Programming

Lecture 15:Simple Java Web Server

Dr. Richard S. Hall [email protected]

Concurrent programming – March 11, 2008

Web Server FeaturesWeb Server Features

● Targets Java Foundation profile i.e., doesn't use advanced Java features so it should be

usable on small devices

● File-based i.e., it does not support dynamic content, such as CGI

scripts, servlets, or JSPs Also supports directory content

▴ If possible, it will convert a directory request to “index.html”

● Configurable● Multi-threaded

Web Server General AlgorithmWeb Server General Algorithm

Step 1 – Listen for requests until told to stop

Step 2 – Upon connection receiptStep 2.1 – Parse requestStep 2.2 – Create and send response

Step 3 – Go to step 1

Web Server StartupWeb Server Startup

● Code for HttpServer.start():

public synchronized void start() throws IOException { if (m_state == INACTIVE_STATE) { m_serverSocket = new ServerSocket(m_port); m_serverThread = new Thread(new Runnable() { public void run() { acceptConnections(); } }, "HttpServer"); m_state = ACTIVE_STATE; m_serverThread.start(); } else if (m_state == STOPPING_STATE) { throw new IllegalStateException( "Server is in process of stopping."); }}


● What happens when start() is called?





First call will actually start the server listening

for connections.




If server is in the process of shutting down, an ex-

ception is thrown.




If the server is already active, then subsequent calls to start() are

ignored.


● Are you prevented from running multiple servers at the same time?public synchronized void start() throws IOException { if (m_state == INACTIVE_STATE) { m_serverSocket = new ServerSocket(m_port); m_serverThread = new Thread(new Runnable() { public void run() { acceptConnections(); } }, "HttpServer"); m_state = ACTIVE_STATE; m_serverThread.start(); } else if (m_state == STOPPING_STATE) { throw new IllegalStateException( "Server is in process of stopping."); }}


● Are you prevented from running multiple servers at the same time?public synchronized void start() throws IOException { if (m_state == INACTIVE_STATE) { m_serverSocket = new ServerSocket(m_port); m_serverThread = new Thread(new Runnable() { public void run() { acceptConnections(); } }, "HttpServer"); m_state = ACTIVE_STATE; m_serverThread.start(); } else if (m_state == STOPPING_STATE) { throw new IllegalStateException( "Server is in process of stopping."); }}

No. You can create as many server instances as you want, they just need different port numbers.

Web Server Request ProcessingWeb Server Request Processing

● HttpServer server thread executes the acceptConnections() method:

private void acceptConnections() { m_threadPool.start(); Socket socket; while (m_serverSocket.isBound() && !m_serverSocket.isClosed()) { try { socket = m_serverSocket.accept(); try { Connection connection = new Connection( this, socket, m_connectionTimeout, m_connectionRequestLimit); m_threadPool.addConnection(connection); } catch (IOException ex) { /* Log */ } } catch (IOException ex) { /* Log */ } } shutdown();}


● Does the server thread process incoming connections?





No, they are handed over to a thread pool. Why?




So the server can handle multiple connections at

the same time.


● Why is this method not synchronized?



● Why is this method not synchronized?


It is only ever called by the server thread; thus,

there is no possibility of interference.

Thread PoolThread Pool

● What is a thread pool ?


● What is a thread pool ? It is a way of managing a set of threads so that you

can reuse them, instead of creating new ones all the time




▴ Why?




▴ More efficient




▴ More efficient

Typically, a thread executes its run() method and dies when run() returns

▴ We want to avoid this

Instead, we make the thread wait so that we can assign it more work




▴ More efficient




▴ How?




▴ More efficient




▴ We use some shared object on which the thread can wait() and some sort of shared data structure for communicating (e.g., a queue)

Insert new work and notify() the waiting thread about it

Thread Pool StartupThread Pool Startup

● ThreadPool.start() works similarly to HttpServer.start(), but does even less

public synchronized void start() { if (m_state != STOPPING_STATE) { m_state = ACTIVE_STATE; } else { throw new IllegalStateException( "Thread pool is in process of stopping."); }}



public synchronized void start() { if (m_state != STOPPING_STATE) { m_state = ACTIVE_STATE; } else { throw new IllegalStateException( "Thread pool is in process of stopping."); }} All calls simply set the

state to active, unless...




Unless the thread pool is in the process of shutting down, then it throws an

exception.


● Why doesn't it create any threads?



● Why doesn't it create any threads?


Better to defer thread creation until they are needed; doesn't con-

sume resources or slow down startup.

Thread Pool Connection HandlingThread Pool Connection Handling

● addConnection() is how the server thread communicates with thread pool threads

public synchronized void addConnection(Connection connection) { if (m_state == ACTIVE_STATE) { m_connectionList.add(connection); notify(); if ((m_threadAvailable < m_connectionList.size()) && (m_threadCount < m_threadLimit)) { m_threadCount++; if (m_threadName == Integer.MAX_VALUE) { m_threadName = 1; } else { m_threadName++; } new Thread(m_group, new Runnable() { public void run() { processConnections(); } }, Integer.toString(m_threadName)).start(); } } else { throw new IllegalStateException(...) }}


● When are threads created?



● When are threads created?


When there are not enough idle threads to

service outstanding connections and the

thread limit hasn't been reached.


● Why do we limit the number of threads created?



● Why do we limit the number of threads created?


For efficiency and to prevent the server

from being overload-ed with threads.

Thread Pool Thread HandlingThread Pool Thread Handling

● ThreadPool.processConnections() is executed by all thread pool threads

// This is pseudo code for presentation purposes

private void processConnections() { Connection connection; while (true) { synchronized (this) { m_threadAvailable++; connection = waitForConnection(); m_threadAvailable--; if (shouldThreadDie(connection)) return; } try { connection.process(); } catch (SocketTimeoutException ex) { /* Log */ } } catch (IOException ex) { /* Log */ } }}


● If all thread pool threads execute this code, won't one thread's connection overwrite another?




● If all thread pool threads execute this code, won't one thread's connection overwrite another?



No, each thread will gets its own local

variable copies on its own stack.


● How do thread pool threads communicated with each other and/or coordinate?




● How do thread pool threads communicated with each other and/or coordinate?



They coordinate us-ing the thread pool's data structure and synchronize on the thread pool itself.


● What happens in the waitForConnection() pseudo code?




● This is where threads wait for the arrival of a new connection to processtry { long start = System.currentTimeMillis(); long current = start; while ((m_connectionList.size() == 0) && ((m_threadTimeout == 0) || ((current - start) < m_threadTimeout))) { wait(m_threadTimeout - (current - start)); current = System.currentTimeMillis(); }} catch (InterruptedException ex) { Thread.currentThread().interrupt();}if (m_connectionList.size() == 0) { connection = null;} else { connection = (Connection) m_connectionList.remove(0);}


● What is all of the timing code doing?

try { long start = System.currentTimeMillis(); long current = start; while ((m_connectionList.size() == 0) && ((m_threadTimeout == 0) || ((current - start) < m_threadTimeout))) { wait(m_threadTimeout - (current - start)); current = System.currentTimeMillis(); }} catch (InterruptedException ex) { Thread.currentThread().interrupt();}if (m_connectionList.size() == 0) { connection = null;} else { connection = (Connection) m_connectionList.remove(0);}




Verifies that the thread at least waits for the thread time-out to expire, unless there is no timeout...




...probably not strictly necessary.


● Why have threads timeout at all?



● Why have threads timeout at all?


Just another way to conserve resources,

by freeing them when they are no longer needed.


● What happens if the thread is interrupted?





We catch it, but reset the interrupted flag, which effectively disables waiting.Why do this?




It is related to how threads die when the

thread pool is stopping...


● Returning to the pseudo code, we can expand on when a thread dies.




● The pseudo code expands to the following...

if (connection == null) { m_threadCount--; if ((m_state == STOPPING_STATE) && (m_threadCount == 0)) { m_shutdownGate.open(); m_shutdownGate = null; m_state = INACTIVE_STATE; } return;}


● The pseudo code expands to the following...which means what?



● The pseudo code expands to the following...which means what?


When a thread's timeout expires and

there are no connections, then it

kills itself.


● So, how does that explain why we use Thread.interrupt() to effectively eliminate waiting?



● So, how does that explain why we want Thread.interrupt() to effectively eliminate waiting?


If the thread no longer waits, it will effectively timeout

immediately and kill itself.


● We will return to this when we discuss shutdown...


Thread Pool Connection ProcessingThread Pool Connection Processing

● Returning to the pseudo code, how does a thread pool thread process a connection?




● Returning to the pseudo code, how does a thread pool thread process a connection?



?


● Pseudo code for Connection.process():public void process() throws IOException { HttpRequest req = new HttpRequest(); HttpResponse res = new HttpResponse(m_server, req); try { // Loop until we close the connection. boolean close = false; while (!close) { // Read the next request. req.parseRequestLine(m_is); req.parseHeaderLines(m_is); req.parseBody(m_is); m_requestCount++; if (m_requestCount >= m_requestLimit) { close = true; } res.sendFileOrDirectory(m_os, close); } } finally { // Close socket and streams. }}


● How is this related to threading/concurrency?public void process() throws IOException { HttpRequest req = new HttpRequest(); HttpResponse res = new HttpResponse(m_server, req); try { // Loop until we close the connection. boolean close = false; while (!close) { // Read the next request. req.parseRequestLine(m_is); req.parseHeaderLines(m_is); req.parseBody(m_is); m_requestCount++; if (m_requestCount >= m_requestLimit) { close = true; } res.sendFileOrDirectory(m_os, close); } } finally { // Close socket and streams. }}


● How is this related to threading/concurrency?public void process() throws IOException { HttpRequest req = new HttpRequest(); HttpResponse res = new HttpResponse(m_server, req); try { // Loop until we close the connection. boolean close = false; while (!close) { // Read the next request. req.parseRequestLine(m_is); req.parseHeaderLines(m_is); req.parseBody(m_is); m_requestCount++; if (m_requestCount >= m_requestLimit) { close = true; } res.sendFileOrDirectory(m_os, close); } } finally { // Close socket and streams. }}

Since clients can send multiple

requests, we limit the number so they

cannot hog a thread.


● What happens if a client hangs?public void process() throws IOException { HttpRequest req = new HttpRequest(); HttpResponse res = new HttpResponse(m_server, req); try { // Loop until we close the connection. boolean close = false; while (!close) { // Read the next request. req.parseRequestLine(m_is); req.parseHeaderLines(m_is); req.parseBody(m_is); m_requestCount++; if (m_requestCount >= m_requestLimit) { close = true; } res.sendFileOrDirectory(m_os, close); } } finally { // Close socket and streams. }}


● What happens if a client hangs?public void process() throws IOException { HttpRequest req = new HttpRequest(); HttpResponse res = new HttpResponse(m_server, req); try { // Loop until we close the connection. boolean close = false; while (!close) { // Read the next request. req.parseRequestLine(m_is); req.parseHeaderLines(m_is); req.parseBody(m_is); m_requestCount++; if (m_requestCount >= m_requestLimit) { close = true; } res.sendFileOrDirectory(m_os, close); } } finally { // Close socket and streams. }}

We set a timeout on our client socket so our thread resumes if the client takes too long, which is why we

catch a timeout exception in ThreadPool.process

Connections().

Thread GateThread Gate

● For stopping both the server and thread pool we use a gate...what is it?

public class ThreadGate { private boolean m_open = false;

public synchronized void open() { m_open = true; notifyAll(); }

public synchronized void await() throws InterruptedException { while (!m_open) { wait(); } }}

Thread GateThread Gate

● For stopping both the server and thread pool we use a gate...what is it?

public class ThreadGate { private boolean m_open = false;

public synchronized void open() { m_open = true; notifyAll(); }

public synchronized void await() throws InterruptedException { while (!m_open) { wait(); } }}

Similar to a semaphore with an initial value of zero, but

where up() makes the value infinity, i.e, down()

will always succeed.

Web Server ShutdownWeb Server Shutdown

● What happens when we stop HttpServer?

public void stop() throws InterruptedException { ThreadGate gate = null; synchronized (this) { if (m_state != INACTIVE_STATE) { if (m_shutdownGate == null) { m_shutdownGate = new ThreadGate(); } gate = m_shutdownGate; try { m_serverSocket.close(); } catch (IOException ex) { } } } if (gate != null) { gate.await(); }}


● What happens when we stop HttpServer?


Simple case, if the server is already

stopped, then ignore...


● How do we actually tell the server to stop?





We close the socket it is using to listen for connections.

Why?




The server thread is waiting on ServerSocket.accept() for new connections, but this

is not interruptible.


● What happens if multiple threads try to stop the server at the same time?



● What happens if multiple threads try to stop the server at the same time?


Only the first succeeds and creates a gate on

which it and subsequent threads will wait for the

server to stop.


● Why is the gate captured in a local variable?





Because we need to wait on the gate, but we must

do that outside the synchronized block...




Additionally, since the server can be stopped and

restarted, we cannot assume that the value of

gate does not change.


● What happens if the calling thread is interrupted while waiting on the gate?



● What happens if the calling thread is interrupted while waiting on the gate?


It is released from the gate and exits the stop() method via an InterruptedException,

but the server continues shutting down.


● Returning to the server thread, what happens when the server socket is closed?



● Returning to the server thread, what happens when the server socket is closed?


It exits its loop and invokes shutdown() on the server instance.


● What does HttpServer.shutdown() do?

private void shutdown() { while (true) { try { m_threadPool.stop(); break; } catch (InterruptedException ex) { } }

synchronized (this) { m_shutdownGate.open(); m_shutdownGate = null; m_state = INACTIVE_STATE; }}





It tells the thread pool to stop and waits for it.





Once the thread pool has stopped, then it releases any threads

waiting for the HTTP server to stop.

Thread Pool ShutdownThread Pool Shutdown

● What happens when we stop ThreadPool?

public void stop() throws InterruptedException { ThreadGate gate = null; synchronized (this) { if (m_state != INACTIVE_STATE) { if ((m_shutdownGate == null) && (m_threadCount > 0)) { m_shutdownGate = new ThreadGate(); } gate = m_shutdownGate; m_state = STOPPING_STATE; m_group.interrupt(); } } if (gate != null) { gate.await(); }}




Pretty much identical to stopping the HTTP server,

except...




Except we interrupt all thread pool threads. Why?




We want to interrupt all thread pool threads to

disable waiting so they kill themselves quickly.




The calling thread then waits for the thread pool to

stop. Who opens the gate?




This is handled when thread pool threads die; the last thread to die opens the

gate.


● So when actually does the server stop?



● So when actually does the server stop?


After all thread pool threads have finished

processing any existing connections.

SummarySummary

● This simple web server illustrates many sophisticated threading and concurrency issues

● It only used what we have learned about so far

lecture 15: simple java web server

Documents