multi threading in java ppt
DESCRIPTION
multi threading in javaTRANSCRIPT
Multi-threading in Java
Jeff HUANG
Software Engineering Group @HKUSTSoftware Engineering Group @HKUST
Do you use them?
2
Do u know their internals?
3
Let’s see
File
DBDBHow can they service so many li i l lclients simultaneously?
4
Multi-threading
A thread is a single sequential flow of controlIn a single process, there could be multiple threadsbe multiple threadsThreads share memory
• The server program spawns multiple threads• Each thread handles a client request
5
ExampleA single core CPU and a task
compute readfile compute readfile compute writefile compute
0 1 2 3 4 5 6 7 sec
2• 2 such tasks6
A simple accountingSingle thread
T=14T=14
Two threads T=8Two threads T=8
7
What’s moreMulti-core & multi-processor
4096 processor Cray 32 processor Dual-core AMD p yX1
pPentium XeonAthlon X2
8
More accountingNumber of cores = 1,000,000 = 1M
Multi-threading can do such tasks in 82MMulti-threading can do such tasks in 8 seconds!Wh t b t th i l th d d i ?
2M
What about the single-threaded version?
9
More benefits
PerformanceUser responsivenesspResource utilizationSi li it f d liSimplicity of modelingSimplified handling of asynchronous eventsp g y…
10
Multi-threads are everywhere
Server programs (web/database server)Client programs (Downloading tools/Browser)p g ( g )Editors (Word/Photoshop/Adobe Reader)S h iSearch engineIDE (Matlab/Eclipse/Visual studio)( p )JVMSwing and AWTSwing and AWT…
11
Where are we now?
How to program?p g
12
Various ways of multithreadingThreadRunnableE F kExectutor FrameworkTimerTimerFutureTask…
13
Two major waysThreadRunnableE F kExectutor FrameworkTimerTimerFutureTask…
14
Creating threads in JavaThread classpublic class Thread{
public void run(){…}; }
Runnable interfacepublic interface Runnable {public interface Runnable {
public void run(); // work ⇒ thread}}
15
Thread Class
public class Thread extends Object implements Runnable {
public Thread();public Thread();public Thread(String name); // Thread namepublic Thread(Runnable R); // Thread ⇒ R.run() p ( ); ()public Thread(Runnable R, String name);
bli id () // if R k f h dpublic void run(); // if no R, work for threadpublic void start(); // begin thread execution...
}
16
More Thread Class Methods
public class Thread extends Object i l t R bl {implements Runnable {
…public static Thread currentThread()public static Thread currentThread()public String getName()public void interrupt()p p ()public boolean isAlive()public void join()
bli id tD ()public void setDaemon()public void setName()public void setPriority()public void setPriority()public static void sleep()public static void yield()p y ()
}17
Creating Threads in Java
1. Thread classExtend Thread class and override the run method
Examplepublic class MyThread extends Thread {public class MyThread extends Thread {
public void run() {… // work for thread
}}M Th d T M Th d () // t th dMyThread T = new MyThread () ; // create thread
T.start(); // begin running threadT.start(); // begin running thread… // thread executing in parallel
18
Creating Threads in Java
2. Runnable interfaceCreate object implementing Runnable interfacePass it to Thread object via Thread constructor
E lExamplepublic class MyRunnable implements Runnable {
public void run() {public void run() {… // work for thread
}}}MyRunnable myR = new MyRunnable(); // create runnable object
Thread T = new Thread(myR); // create thread
T.start(); // begin running thread… // thread executing in parallel 19
Producer - Consumerclass Producer extends Thread {
class Consumer extends Thread {{
private final BlockingQueue queue;Producer(BlockingQueue q){queue = q;}public void run()
{ private final BlockingQueue queue;Consumer(BlockingQueue q) { queue = q; }public void run()
public static void main (String[] args) {
{while(true) {
queue put(produce());
{while(true) {
consume(queue take());
BlockingQueue q = new SomeQueue(); Producer p = new Producer(q);
queue.put(produce()); }
}private Message produce()
consume(queue.take()); }
}private void consume(Message mes)
Consumer c = new Consumer(q); p.start();
{ return new Message();
}}
{…
}}
c.start(); }
} }
Example 120
Example 1
Simple WebServerpublic class MyRunnable implements Runnable {
public void run() {public void run() {handleRequest(connection);
}}
Process request
public class SimpleWebServer{
}
public static void main (String[] args) throws IOException {ServerSocket socket = new ServerSocket(80);while (true) {
fi l S k t ti k t t()Accept client request
final Socket connection = socket.accept();Runnable task = new MyRunnable(connection);Thread t = new Thread(task);t start(); t t th d
Create threadCreate Runnable
t.start();}
}}
start thread
}21Example 2
Let’s look a bit deeperThread statesThread schedulerDiffi l i i M l i h d dDifficulties in Multithreaded programmingp g g
22
Thread StatesJava thread can be in one of these states
New – thread allocated & waiting for start()Runnable – thread can begin executionRunning – thread currently executingBlocked – thread waiting for event (I/O, etc.)Dead – thread finished
Transitions between states caused byyInvoking methods in class Thread
new(), start(), yield(), sleep(), wait(), notify()…(), (), y (), p(), (), y()
Other (external) eventsScheduler, I/O, returning from run()…, , g ()
23
Thread States
runnablenewnew start
notify notifyAllrunnable
scheduler
new
yield,
notify, notifyAll,IO complete, sleep expired
running blocked
ytime slice
running blocked
terminateIO, sleep,wait, join
dead
24
Thread Scheduling
SchedulerDetermines which runnable threads to runCan be based on thread priorityCan be based on thread priorityPart of OS or Java Virtual Machine (JVM)
Preempted byScheduling policy
Non-preemptive (cooperative) scheduling
Preempted by scheduler
Non preemptive (cooperative) schedulingPreemptive scheduling
25
Let’s do a bit analysisTwo threads A and B
They both do the following workprint 1, print 2, print 3
The main thread first starts A, then starts B
main (): thread A, thread B
thread A: println 1, println 2, println 3
thread B: println 1, println 2, println 3
26
Let’s do a bit analysisNon-preemptive Preemptive scheduler scheduler
Single-core ???1, 2, 3, 1, 2, 3, , , , ,Multi-core ??? ???
main (): thread A, thread B
thread A: println 1, println 2, println 3
thread B: println 1, println 2, println 3
27
Java Thread Examplepublic class ThreadExample extends Thread {
public void run() {for (int i = 1; i <= 3; i++)
System.out.println(i);y p ( );try {
sleep((int)(Math.random() * 1000));} catch (InterruptedException e) { }} catch (InterruptedException e) { }
}
public static void main(String[] args) {public static void main(String[] args) {Thread A = new ThreadExample();Thread B = new ThreadExample();A t t()A.start();B.start();
System.out.println("Done");}
} 28
Java Thread Example
Possible outputsPossible outputs1,1,Done,2,2,3,3 1 Done 1 2 2 3 31,Done,1,2,2,3,31,1,Done,2,3,2,3 Done 1 1 2 2 3 3Done,1,1, 2,2,3,3…
Multi-threads can interleave their executions!!!
29
Another examplepublic class Worker extends Thread {
static int x = 0;static int x 0;public void run() {
for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;
}}
First increase x by 1Then decrease x by 1
}public static void main (String[] args) {
x = 0;for (int i = 0; i < 1000; i++)
new Worker().start(); Start 1000 threads
… // wait for all threads exit
System.out.println(x);System.out.println(x);}
} 30
Data Racepublic class Worker extends Thread {
static int x = 0;static int x 0;public void run() {
for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;
}}
First increase x by 1Then decrease x by 1
}public static void main (String[] args) {
x = 0;for (int i = 0; i < 1000; i++)
new Worker().start(); Start 1000 threads
… // wait for all threads exit
System.out.println(x); X is not always 0 !!System.out.println(x);}
} 31
X is not always 0 !!
Quiz time
32
Answer: Yes!
33
How Can This Happen?
Compiler can reorder statementsOr keep values in registersp g
The Java memory model is designed to allow aggressive optimizationaggressive optimizationOn multi-processor, values not synchronized to global memoryGood for performancep
34
Observations
Multithread execution is non-deterministicDepends on scheduler & single or multi-core
Thread scheduling may cause data racesThread scheduling may cause data racesModifying same data from multiple threadsResult depends on thread execution orderResult depends on thread execution order
Complier can reorder statementsThe memory model is designed to allow aggressive optimization
35
How to deal with it?
We need protocol to make sure the shared pdata is manipulated safelyWe need to force the compiler to ensure thatWe need to force the compiler to ensure that the modified shared data visible to other th dthreadsSynchronization
LockSemaphore Java synchronizationpMonitor
y
36
Java synchronization
Java uses the concept of monitorsJava uses the concept of monitorsJava uses the concept that every object is
i t d ith l kassociated with a lockuse synchronized keywordTwo ways
Synchronized methodsSynchronized methodsSynchronized blocks
37
Synchronized methodCalling a synchronized method attempts to possess the lock
If no one owns the lock, then this thread has/owns the lock and proceeds.Otherwise, it has to wait until the lock is released by some other thread
LockFor static method, it is the lock associated with the Class object to which the static method belongs toFor instance method, it is the lock associated with th bj t hi h th th d ll i b i dthe object on which the method call is being made
38
Synchronized blocks
Very similar to synchronized method but it isVery similar to synchronized method, but it is Blocks of code, rather than entire methods
public void someMethod() {{
// non-critical sectionsynchronized(someobject) synchronized(this){
// critical section}}// non-critical section
}}
39
Data Racepublic class Worker extends Thread {
static int x = 0;static int x 0;public void run() {
for (int i = 0; i < 1000; i++) {x = x + 1; First increase x by 1x = x + 1;x = x – 1;
}}
First increase x by 1Then decrease x by 1
}public static void main (String[] args) {
x = 0;for (int i = 0; i < 1000; i++)
new Worker().start(); Start 1000 threads
… // wait for all threads exit
System.out.println(x); X is not always 0 !!System.out.println(x);}
} 40
X is not always 0 !!
Using synchronization
public class Worker extends Thread {public class Worker extends Thread {static int x = 0;static Object lock = new Object();j jpublic void run() {
synchronized(lock) {f (i t i 0 i < 1000 i++) {for (int i = 0; i < 1000; i++) {
x = x + 1;x = x – 1;x x 1;
}}
}}
41
QuestionsWhat would happen if the lock field were not static?Why don’t we just make the run methodWhy don’t we just make the run method synchronized?Why don’t we just synchronize on x?
42
But, it is hard in practice
Holding locks is a global propertyHolding locks is a global propertyaffects entire program, cannot be hidden behind an abstract interfacean abstract interface
Results in lack of modularitycallers cannot ignore what locks their calleesacquire or what locations they access
f id b t l f l b lnecessary for race avoidance, but also for global ordering to avoid deadlock part of a method’s protocol which lock needs topart of a method’s protocol which lock needs to be held when called, which locks it acquires
43
Bank Account Example
l A t {
Atomicity violationclass Account {private int balance = 0;
public void deposit(int n) {int r read();public read() {
int r;synchronized(this) {
int r = read();
synchronized(this) { b l
other threads can update balanceyr = balance;
}return r;
balance = r + n; }
}return r;}
}}
Race freedom is not sufficientRace-freedom is not sufficient44
Optimized Bank Account
l A t {class Account {private int balance = 0;
public void deposit(int n) {synchronized(this) { public read() {
return balance;}
synchronized(this) { int r = balance;balance = r + n;
}
}
}}
45
Another Account Exampleclass Account {
private int balance = 0;private int balance = 0;public synchronized int withdraw(int amt){…}public synchronized void deposit(int i) {…}
}}class Client1{
public synchronized void move(Account a1, Account a2){
a2.deposit(a1.withdraw(1000000));}
}class Client2 … // same as Client1
Client1: move(a1,a2);Client move(a ,a );Client2: move(a2,a1); 46
Deadlockclient1 holds lock for account a1, waits for account a2client2 holds lock for account a2, waits for ,account a1Both of them can not proceedBoth of them can not proceed
Cli 1
a1 a2Client1 Client2
a1 a2
47
Impair performance
Frequent lock acquire and lock releaseFrequent lock acquire and lock release operationsWe are sacrificing performance for dataWe are sacrificing performance for data safetyBut sometimes there is no conflictOn some extreme situations, performance isOn some extreme situations, performance is greatly impaired
48
Is it hard?
49
Let’s make a conclusion
Multithreading is quite usefulg qBut you need to be carefulBugs are everywhere
50
What we have learned
Benefits of multithreadsPerformanceUser responsivenessp…
How to make multithreaded Java programsHow to make multithreaded Java programsThread class/Runnable interface
Diffic lties in M ltithreadingDifficulties in MultithreadingStatements reorder/Data races/Atomicity i l ti /D dl kviolation/Deadlock
Java synchronizationSyncrhonized methods/blocks
51
QuestionsWhat would happen if the lock field were not static?Why don’t we just make the run methodWhy don’t we just make the run method synchronized?Why don’t we just synchronize on x?
52
Using synchronization
public class Worker extends Thread {public class Worker extends Thread {static int x = 0;static Object lock = new Object();j jpublic void run() {
synchronized(lock) {f (i t i 0 i < 1000 i++) {for (int i = 0; i < 1000; i++) {
x = x + 1;x = x – 1;x x 1;
}}
}}
53
See you next time
Backupsp
Daemon Threads
Java threads typesUserDaemon
Provide general services Typically never terminateCall setDaemon() before start()
Program termination1. All user threads finish2. Daemon threads are terminated by JVM3. Main program finishesp g
56
Atomicity violation
57
Thread flow graph
58
Non-preemptive Scheduling
Threads continue execution until Thread terminatesExecutes instruction causing wait (e.g., IO)Thread volunteering to stop (invoking yield or sleep)
59
Preemptive Scheduling
Threads continue execution untilSame reasons as non-preemptive schedulingPreempted by scheduler
60
Transactional Memory
Instead of using synchronizations, the runtime system ensures that the critical section executes t ti lltransactionallyYou can just go ahead when entering a critical
ti h iti th iti l ti thsection; when you are exiting the critical section, the runtime system will check the data safety for you
if th i fli t j t dif there is no conflict, just proceedOtherwise, rollback
The transaction memory system guaranteesThe transaction memory system guaranteesother threads cannot see intermediate values of the transaction (all-or-nothing)( g)the transaction cannot see values of other transactions in the middle of its execution 61
Hardware & software TM
Hardware transactional memory(HTM)Software transactional memory (STM)Software transactional memory (STM)
62
Advantages
No deadlock!we never refer to locks explicitly
Composability & modularityp y yno need to know what callees do w.r.t. synchronization
Performance could better in some situations
63