Download - OS Support for Building Distributed Applications: Multithreaded Programming using Java Threads
1
OS Support for Building Distributed Applications:
Multithreaded Programming using Java
Threads
2
Outline
Introduction Thread Applications Defining Threads Java Threads and States Architecture of Multithreaded servers Threads Synchronization Thread Concurrency Models Summary
3
Introduction
We discuss how Middleware is supported by the Operating System (OS) facilities at the nodes of a distributed system.
The OS facilitates: Encapsulation and protection of resources
inside servers; It supports invocation of mechanisms
required to access those resources including concurrent access/processing.
4
Middleware and Network Operating System (NOS)
Many DOS (Distributed OS) have been investigated, but there are none in general/wide use. But NOS are in wide use for various reasons both technical and non-technical.
Users have much invested in their application software; they will not adopt to new OS that will not run their applications.
The 2nd reason against the adoption of DOS is that users tend to prefer to have a degree of autonomy of their machines, even in a closely knit organisation.
A combination of middleware and NOSs provides an acceptable balance between the requirement of autonomy and network transparency.
NOS allows users to run their favorite word processor. Middleware enables users to take advantage of services
that become available in their distributed system.
5
Operating system layers and Middleware
Unix and Windows are two examples of Network Operating Systems – have a networking capability built into them and so can be used to access remote resources using basic services such as rlogin, telnet.
Applications, services
Computer &
Platform
Middleware
OS: kernel,libraries & servers
network hardware
OS1
Computer & network hardware
Node 1 Node 2
Processes, threads,communication, ...
OS2Processes, threads,communication, ...
6
Core OS components and functionality
Communication
manager
Thread manager Memory manager
Supervisor
Process manager
7
A single threaded program
class ABC{….
public void main(..){…..}
}
begin
body
end
8
A Multithreaded Program
Main Thread
Thread A Thread B Thread C
start startstart
Threads may switch or exchange data/results
9
Single and Multithreaded Processes
Single-threaded Process
Single instruction stream Multiple instruction stream
Multiplethreaded ProcessThreads of
Execution
CommonAddress Space
threads are light-weight processes within a process
10
Application
Application Application
Application
CPU
Better Response Times in Multiple Application Environments
Higher Throughput for Parallelizeable Applications
CPU
CPU
CPU CPU CPU
Threaded Libraries, Multi-threaded I/OThreaded Libraries, Multi-threaded I/O
Multi-Processing (clusters & grids) and Multi-Threaded
Computing
11
Web/Internet Applications:Serving Many Users
Simultaneously
Internet Server
PC client
Local Area Network
PDA
12
ServerThreads
Server ProcessClient 1 Process
Client 2 Process
Multithreaded Server: For Serving Multiple Clients Concurrently
Internet
13
Printing ThreadPrinting Thread
Editing Thread
Editing Thread
Modern Applications need Threads (ex1):Editing and Printing documents in
background.
14
Multithreaded/Parallel File Copy
reader()
{
- - - - - - - - - -
lock(buff[i]);
read(src,buff[i]);
unlock(buff[i]);
- - - - - - - - - -
}
reader()
{
- - - - - - - - - -
lock(buff[i]);
read(src,buff[i]);
unlock(buff[i]);
- - - - - - - - - -
}
writer()
{
- - - - - - - - - -
lock(buff[i]);
write(src,buff[i]);
unlock(buff[i]);
- - - - - - - - - -
}
writer()
{
- - - - - - - - - -
lock(buff[i]);
write(src,buff[i]);
unlock(buff[i]);
- - - - - - - - - -
}
buff[0]buff[0]
buff[1]buff[1]
Cooperative Parallel Synchronized Threads
Cooperative Parallel Synchronized Threads
15
Code-GranularityCode ItemLarge grain(task level)Program
Medium grain(control level)Function (thread)
Fine grain(data level)Loop (Compiler)
Very fine grain(multiple issue)With hardware
Code-GranularityCode ItemLarge grain(task level)Program
Medium grain(control level)Function (thread)
Fine grain(data level)Loop (Compiler)
Very fine grain(multiple issue)With hardware
Task i-lTask i-l Task iTask i Task i+1Task i+1
func1 ( ){........}
func1 ( ){........}
func2 ( ){........}
func2 ( ){........}
func3 ( ){........}
func3 ( ){........}
a ( 0 ) =..b ( 0 ) =..
a ( 0 ) =..b ( 0 ) =..
a ( 1 )=..b ( 1 )=..
a ( 1 )=..b ( 1 )=..
a ( 2 )=..b ( 2 )=..
a ( 2 )=..b ( 2 )=..
++ xx LoadLoad
Sockets/PVM/MPI
Threads
Compilers
CPU
Levels of Parallelism
16
Process Parallelism Process Parallelism Process Parallelism Process Parallelism
P1P1
P2P2
P3P3
time
time
No of executing process ≥ the number of CPUs
No of executing process ≥ the number of CPUs
CPU
CPU
CPU
Multithreading - Multiprocessors
17
Multithreading on Uni-processor
Concurrency Vs Parallelism Process ConcurrencyProcess Concurrency Process ConcurrencyProcess Concurrency
Number of Simultaneous execution units > number of CPUs
Number of Simultaneous execution units > number of CPUs
P1P1
P2P2
P3P3
time
time
CPU
18
What are Threads?
A piece of code that run in concurrent with other threads.
Each thread is a statically ordered sequence of instructions.
Threads are being extensively used to express concurrency on both single and multiprocessor machines.
Programming a task having multiple threads of control – Multithreading or Multithreaded Programming.
19
Java Threads
Java has built in thread support for Multithreading
Synchronization Thread Scheduling Inter-Thread Communication:
currentThread start setPriority yield run getPriority sleep stop suspend resume
Java Garbage Collector is a low-priority thread.
20
Threading Mechanisms... Create a class that extends the Thread
class Create a class that implements the
Runnable interface
21
1st method: Extending Thread class
Threads are implemented as objects that contains a method called run() class MyThread extends Thread {
public void run() { // thread body of execution } } Create a thread: MyThread thr1 = new MyThread(); Start Execution of threads: thr1.start(); Create and Execute: new MyThread().start();
22
An example
class MyThread extends Thread { // the thread public void run() { System.out.println(" this thread is running ... "); }} // end class MyThread
class ThreadEx1 { // a program that utilizes the thread public static void main(String [] args ) {
MyThread t = new MyThread();MyThread t2 = new MyThread();// due to extending the Thread class (above)// I can call start(), and this will call// run(). start() is a method in class Thread.t2.start();
t.start(); } // end main()} // end class ThreadEx1
23
threads programming
More about threads programming Using interfaces Multiple threads in a program
24
2nd method: Threads by implementing Runnable interfaceclass MyThread extends ABC implements Runnable{ ..... public void run() { // thread body of execution }} Creating Object: MyThread myObject = new MyThread(); Creating Thread Object: Thread thr1 = new Thread( myObject ); Start Execution: thr1.start();
25
An example
class MyThread implements Runnable { public void run() { System.out.println(" this thread is running ... "); }} // end class MyThread
class ThreadEx2 { public static void main(String [] args ) { Thread t = new Thread(new MyThread()); // due to implementing the Runnable
interface // I can call start(), and this will call run(). t.start(); } // end main()} // end class ThreadEx2
26
Life Cycle of Thread
new
runnable non-runnable
dead
wait()sleep()suspend()blocked
notify()sleptresume()unblocked
start()
stop()
27
A Program with Three Java Threads
Write a program that creates 3 threads
28
Three threads example
class A extends Thread { public void run() { for(int i=1;i<=5;i++) { System.out.println("\t From ThreadA: i= "+i); } System.out.println("Exit from A"); } }
class B extends Thread { public void run() { for(int j=1;j<=5;j++) { System.out.println("\t From ThreadB: j= "+j); } System.out.println("Exit from B"); } }
29
Three threads example
class C extends Thread { public void run() { for(int k=1;k<=5;k++) { System.out.println("\t From ThreadC: k= "+k); }
System.out.println("Exit from C"); } }
class ThreadTest { public static void main(String args[]) { new A().start(); new B().start(); new C().start(); } }
30
Run 1
[user@speedy] threads [1:76] java ThreadTest From ThreadA: i= 1 From ThreadA: i= 2 From ThreadA: i= 3 From ThreadA: i= 4 From ThreadA: i= 5Exit from A From ThreadC: k= 1 From ThreadC: k= 2 From ThreadC: k= 3 From ThreadC: k= 4 From ThreadC: k= 5Exit from C From ThreadB: j= 1 From ThreadB: j= 2 From ThreadB: j= 3 From ThreadB: j= 4 From ThreadB: j= 5Exit from B
31
Run2
[user@speedy] threads [1:77] java ThreadTest From ThreadA: i= 1 From ThreadA: i= 2 From ThreadA: i= 3 From ThreadA: i= 4 From ThreadA: i= 5 From ThreadC: k= 1 From ThreadC: k= 2 From ThreadC: k= 3 From ThreadC: k= 4 From ThreadC: k= 5Exit from C From ThreadB: j= 1 From ThreadB: j= 2 From ThreadB: j= 3 From ThreadB: j= 4 From ThreadB: j= 5Exit from BExit from A
32
ServerThreads
Message PassingFacility
Server ProcessClient Process
Client Process
User Mode
Kernel Mode
Multithreaded ServerMultithreaded Server
33
Architecture for Multithread Servers
Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second.
Threads may need to treat requests with varying priorities: A corporate server could prioritize request
processing according to class of customers. Architectures:
Worker pool Thread-per-request Thread-per-connection Thread-per-object
34
Client and server with threads
In worker-pool architecture, the server creates a fixed pool of worker threads to process requests.
The module “receipt and queuing” receives requests from sockets/ports and places them on a shared request queue for retrieval by the workers.
Server
N threads
Input-output
Client
Thread 2 makes
T1
Thread 1
requests to server
generates results
Requests
Receipt &queuing
35
Alternative server threading architectures
a. Thread-per-request b. Thread-per-connection c. Thread-per-object
remote
workers
I/O remoteremote I/O
per-connection threads per-object threads
objects objects objects
36
Application 1: Multithreaded Dictionary Server – Demonstrate the use of Sockets
and Threads
MultithreadedDictionary Server
(try, dictionary.com)
A Client ProgramWhat is the Meaning of (“Love”)?
A Client ProgramWhat is the Meaning of (“Love”)?
A Client Program in “C++”A Client
Program in “C”
“Love”
“A deep, tender, ineffable feeling of affection and solicitude toward a
person ”
“care or concern, as for the well-being of another”
“Solicitude”
37
Synchronisation & Priorities
Thread Synchronisation Thread Priorities
38
Accessing Shared Resources
Applications Access to Shared Resources need to be coordinated. Printer (two person jobs cannot be printed at
the same time) Simultaneous operations on your bank
account. Can the following operations be done at the
same time on the same account? Deposit() Withdraw() Enquire()
39
Online Bank: Serving Many Customers and Operations
Internet Bank Server
PC client
Local Area Network
PDABank
Database
40
Shared Resources
If one thread tries to read the data and other thread tries to update the same data, it leads to inconsistent state.
This can be prevented by synchronising access to the data.
Use “Synchronized” method: public synchronized void update() {
… }
41
the driver: 3 Threads sharing the same object
class InternetBankingSystem { public static void main(String [] args ) { Account accountObject = new Account (); Thread t1 = new Thread(new MyThread(accountObject)); Thread t2 = new Thread(new YourThread(accountObject)); Thread t3 = new Thread(new HerThread(accountObject));
t1.start(); t2.start(); t3.start(); // DO some other operation } // end main()}
42
Shared account object between 3 threads
class MyThread implements Runnable { Account account; public MyThread (Account s) { account = s;} public void run() { account.deposit(); }} // end class MyThread
class YourThread implements Runnable { Account account; public YourThread (Account s) { account = s;} public void run() { account.withdraw(); } } // end class YourThread
class HerThread implements Runnable { Account account; public HerThread (Account s) { account = s; } public void run() {account.enquire(); }} // end class HerThread
account(shared object)
43
Monitor (shared object access): serializes operation on shared
objectclass Account { // the 'monitor' int balance;
// if 'synchronized' is removed, the outcome is unpredictable public synchronized void deposit( ) { // METHOD BODY : balance += deposit_amount; }
public synchronized void withdraw( ) { // METHOD BODY: balance -= deposit_amount;
} public synchronized void enquire( ) {
// METHOD BODY: display balance. }
}
44
Thread Priority
In Java, each thread is assigned priority, which affects the order in which it is scheduled for running. The threads so far had same default priority (NORM_PRIORITY) and they are served using FCFS policy. Java allows users to change priority:
ThreadName.setPriority(intNumber) MIN_PRIORITY = 1 NORM_PRIORITY=5 MAX_PRIORITY=10
45
Thread Priority Example
class A extends Thread{ public void run() { System.out.println("Thread A started"); for(int i=1;i<=4;i++) { System.out.println("\t From ThreadA: i= "+i); } System.out.println("Exit from A"); }}class B extends Thread{ public void run() { System.out.println("Thread B started"); for(int j=1;j<=4;j++) { System.out.println("\t From ThreadB: j= "+j); } System.out.println("Exit from B"); }}
46
Thread Priority Example
class C extends Thread{ public void run() { System.out.println("Thread C started"); for(int k=1;k<=4;k++) { System.out.println("\t From ThreadC: k= "+k); } System.out.println("Exit from C"); }}class ThreadPriority{ public static void main(String args[]) { A threadA=new A(); B threadB=new B(); C threadC=new C(); threadC.setPriority(Thread.MAX_PRIORITY); threadB.setPriority(threadA.getPriority()+1); threadA.setPriority(Thread.MIN_PRIORITY); System.out.println("Started Thread A"); threadA.start(); System.out.println("Started Thread B"); threadB.start(); System.out.println("Started Thread C"); threadC.start(); System.out.println("End of main thread"); }}
47
Thread Programming models
Thread concurrency/operation models
The master/worker model The peer model A thread pipeline
48
taskXtaskX
taskYtaskY
taskZtaskZ
main ( )main ( )
WorkersProgram
Files
Resources
Databases
Disks
SpecialDevices
Master
Input (Stream)
The master/worker model
49
Example
main() /* the master */ { forever { get a request; switch( request ) case X: pthread_create(....,taskX); case Y: pthread_create(....,taskY); .... } } taskX() /* worker */ { perform the task, sync if accessing shared resources } taskY() /* worker */ { perform the task, sync if accessing shared resources } .... --Above runtime overhead of creating thread can be solved by thread pool * the master thread creates all worker thread at program initialization and each worker thread suspends itself immediately for a wakeup call from the master
50
The peer model
taskXtaskX
taskYtaskY
WorkersProgram
Files
Resources
Databases
Disks
SpecialDevices
taskZtaskZ
InputInput
51
Example
main() { pthread_create(....,thread1...taskX); pthread_create(....,thread2...taskY); .... signal all workers to start wait for all workers to finish do any cleanup } } taskX() /* worker */ { wait for start perform the task, sync if accessing shared resources } taskY() /* worker */ { wait for start perform the task, sync if accessing shared resources }
52
A thread pipeline
Resources Files
Databases
Disks
Special Devices
Files
Databases
Disks
Special Devices
Files
Databases
Disks
Special Devices
Stage 1Stage 1 Stage 2Stage 2 Stage 3Stage 3
Program Filter Threads
Input (Stream)
A thread pipeline
53
Examplemain()
{
pthread_create(....,stage1);
pthread_create(....,stage2);
....
wait for all pipeline threads to finish
do any cleanup
}
stage1() {
get next input for the program
do stage 1 processing of the input
pass result to next thread in pipeline
}
stage2(){
get input from previous thread in pipeline
do stage 2 processing of the input
pass result to next thread in pipeline
}
stageN()
{
get input from previous thread in pipeline
do stage N processing of the input
pass result to program output.
}
54
Java thread constructor and management methods
Thread(ThreadGroup group, Runnable target, String name) Creates a new thread in the SUSPENDED state, which will belong to group and be identified as name; the thread will execute the run() method of target.
setPriority(int newPriority), getPriority()Set and return the thread’s priority.
run()A thread executes the run() method of its target object, if it has one, and otherwise its own run() method (Thread implements Runnable).
start()Change the state of the thread from SUSPENDED to RUNNABLE.
sleep(int millisecs)Cause the thread to enter the SUSPENDED state for the specified time.
yield()Enter the READY state and invoke the scheduler.
destroy()Destroy the thread.
55
Java thread synchronization calls
thread.join(int millisecs)
Blocks the calling thread for up to the specified time until thread has terminated.
thread.interrupt()Interrupts thread: causes it to return from a blocking method call such as sleep().
object.wait(long millisecs, int nanosecs)Blocks the calling thread until a call made to notify() or notifyAll() on object wakes the thread, or the thread is interrupted, or the specified time has elapsed.
object.notify(), object.notifyAll()Wakes, respectively, one or all of any threads that have called wait() on object.
56
RPC delay against parameter size
1000 2000
RPC delay
Requested datasize (bytes)
Packetsize
0
57
A lightweight remote procedure call
1. Copy args
2. Trap to Kernel
4. Execute procedureand copy results
Client
User stub
Server
Kernel
stub
3. Upcall 5. Return (trap)
A A stack
58
Times for serialized and concurrent invocations
Client Server
execute request
Send
Receiveunmarshal
marshal
Receiveunmarshal
process results
marshalSend
process args
marshalSend
process args
transmission
Receiveunmarshal
process results
execute request
Send
Receiveunmarshal
marshal
marshalSend
process args
marshalSend
process args
execute request
Send
Receiveunmarshal
marshal
execute request
Send
Receiveunmarshal
marshalReceive
unmarshalprocess results
Receiveunmarshal
process resultstime
Client Server
Serialised invocations Concurrent invocations
59
Summary
Operating system provides various types of facilities to support middleware for distributed system:
encapsulation, protection, and concurrent access and management of node resources.
Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second.
Threads support treating of requests with varying priorities. Various types of architectures can be used in current
processing: Worker pool Thread-per-request Thread-per-connection Thread-per-object
Threads need to be synchronized when accessing and manipulating shared resources.
New OS designs provide flexibility in terms of separating mechanisms from policies.