1 17-214

PrinciplesofSoftwareConstruction: Objects,Design,andConcurrencyPart3:ConcurrencyIntroductiontoconcurrency,part3Concurrencyprimitives,libraries,anddesignpatternsCharlieGarrodChrisTimperley

2 17-214

Administrivia

•  Homework5bdue11:59p.m.Tuesday–  TurninbyWednesday9a.m.tobeconsideredasaBestFramework

•  Optionalreadingduetoday:–  JavaConcurrencyinPractice,Chapter10

3 17-214

KeyconceptsfromTuesday

4 17-214

Avoidingdeadlock

•  Thewaits-forgraphrepresentsdependenciesbetweenthreads–  Eachnodeinthegraphrepresentsathread–  AnedgeT1->T2representsthatthreadT1iswaitingforalockT2owns

•  Deadlockhasoccurrediffthewaits-forgraphcontainsacycle•  Onewaytoavoiddeadlock:lockingprotocolsthatavoidcycles

ab

c

d

f

e

h

g

i

5 17-214

Encapsulatingthesynchronizationimplementation

publicclassBankAccount{privatelongbalance;privatefinallongid=SerialNumber.generateSerialNumber();privatefinalObjectlock=newObject();publicBankAccount(longbalance){this.balance=balance;}

staticvoidtransferFrom(BankAccountsource,BankAccountdest,longamount){BankAccountfirst=source.id<dest.id?source:dest;BankAccountsecond=first==source?dest:source;synchronized(first.lock){synchronized(second.lock){source.balance-=amount;dest.balance+=amount;}}}…

6 17-214

Anaside:JavaConcurrencyinPracticeannotations

@ThreadSafepublicclassBankAccount{@GuardedBy("lock")privatelongbalance;privatefinallongid=SerialNumber.generateSerialNumber();privatefinalObjectlock=newObject();publicBankAccount(longbalance){this.balance=balance;}

staticvoidtransferFrom(BankAccountsource,BankAccountdest,longamount){BankAccountfirst=source.id<dest.id?source:dest;BankAccountsecond=first==source?dest:source;synchronized(first.lock){synchronized(second.lock){source.balance-=amount;dest.balance+=amount;}…

7 17-214

Anaside:JavaConcurrencyinPracticeannotations

•  @ThreadSafe•  @NotThreadSafe•  @GuardedBy•  @Immutable

8 17-214

Today

•  Strategiesforsafety•  Javalibrariesforconcurrency•  Buildingthread-safedatastructures

–  Javaprimitivesforconcurrentcoordination

•  Programstructureforconcurrency

9 17-214

Policiesforthreadsafety

•  Thread-confined•  Sharedread-only•  Sharedthread-safe

–  Objectsthatperforminternalsynchronization

•  Guarded–  Objectsthatmustbesynchronizedexternally

10 17-214

Stackconfinement

•  Primitivelocalvariablesareneversharedbetweenthreads

11 17-214

Threadconfinementwithjava.lang.ThreadLocal<T>

•  Sharablevariablethatconfinesstatetoeachthread–  LogicallysimilartoaMap<Thread,T>ThreadLocal<T>:Tget();//getsvalueforcurrentthreadvoidset(Tvalue);//setsvalueforcurrentthread

12 17-214

Sharedread-only

•  Immutabledataisalwayssafetoshare

13 17-214

Sharedthread-safe

•  "Thread-safe"objectsthatperforminternalsynchronization•  Buildyourown,orknowtheJavaconcurrencylibraries

14 17-214

java.util.concurrentisBIG(1)

•  Atomicvariables:java.util.concurrent.atomic–  Supportvariousatomicread-modify-writeops

•  Executorframework–  Tasks,futures,threadpools,completionservice,etc.

•  Locks:java.util.concurrent.locks–  Read-writelocks,conditions,etc.

•  Synchronizers–  Semaphores,cyclicbarriers,countdownlatches,etc.

15 17-214

java.util.concurrentisBIG(2)

•  Concurrentcollections–  Sharedmaps,sets,lists

•  Dataexchangecollections–  Blockingqueues,deques,etc.

•  Pre-packagedfunctionality:java.util.Arrays–  Parallelsort,parallelprefix

16 17-214

Thejava.util.concurrent.atomicpackage

•  Concreteclassessupportingatomicoperations,e.g.:–  AtomicLong

longget();voidset(longnewValue);longgetAndSet(longnewValue);longgetAndAdd(longdelta);longgetAndIncrement();booleancompareAndSet(longexpectedValue, longnewValue);longgetAndUpdate(LongUnaryOperatorupdateFunction);longupdateAndGet(LongUnaryOperatorupdateFunction);…

17 17-214

AtomicLongexample

publicclassSerialNumber{privatestaticAtomicLongnextSerialNumber=newAtomicLong();publicstaticlonggenerateSerialNumber(){returnnextSerialNumber.getAndIncrement();}}

18 17-214

Overviewofjava.util.concurrent.atomic

•  Atomic{Boolean,Integer,Long}–  Boxedprimitivesthatcanbeupdatedatomically

•  AtomicReference<T>–  Objectreferencethatcanbeupdatedatomically

•  Atomic{Integer,Long,Reference}Array–  Arraywhoseelementsmaybeupdatedatomically

•  Atomic{Integer,Long,Reference}FieldUpdater–  Reflection-basedutilityenablingatomicupdatestovolatilefields

•  LongAdder,DoubleAdder–  Highlyconcurrentsums

•  LongAccumulator,DoubleAccumulator–  Generalizationofaddertoarbitraryfunctions(max,min,etc.)

19 17-214

Concurrentcollections

•  Providehighperformanceandscalability

Unsynchronized ConcurrentHashMap ConcurrentHashMapHashSet ConcurrentHashSetTreeMap ConcurrentSkipListMapTreeSet ConcurrentSkipListSet

20 17-214

java.util.concurrent.ConcurrentHashMap

•  Implementsjava.util.Map<K,V>–  Highconcurrencylockstriping

•  Internallyusesmultiplelocks,eachdedicatedtoaregionofhashtable–  Externally,canuseConcurrentHashMaplikeanyothermap…

Locks

Hashtable

21 17-214

Atomicread-modify-writemethods

•  VputIfAbsent(Kkey,Vvalue);•  booleanremove(Objectkey,Objectvalue);•  Vreplace(Kkey,Vvalue);•  booleanreplace(Kkey,VoldValue,VnewValue);•  Vcompute(Kkey,BiFunction<...>remappingFn);•  VcomputeIfAbsent(Kkey,Function<...>mappingFn);•  VcomputeIfPresent(Kkey,BiFunction<...>remapFn);•  Vmerge(Kkey,Vvalue,BiFunction<...>remapFn);

22 17-214

java.util.concurrent.BlockingQueue

•  Implementsjava.util.Queue<E>•  java.util.concurrent.SynchronousQueue

–  Eachputdirectlywaitsforacorrespondingpoll•  java.util.concurrent.ArrayBlockingQueue

–  put blocksifthequeueisfull–  poll blocksifthequeueisempty

23 17-214

The CopyOnWriteArrayList

•  Implementsjava.util.List<E>•  Allwritestothelistcopythearraystoringthelistelements

24 17-214

Example:addingconcurrencytotheobserverpattern

privatefinalList<Observer<E>>observers=newArrayList<>();publicvoidaddObserver(Observer<E>observer){synchronized(observers){observers.add(observer);}}publicbooleanremoveObserver(Observer<E>observer){synchronized(observers){returnobservers.remove(observer);}}privatevoidnotifyOf(Eelement){synchronized(observers){for(Observer<E>observer:observers)observer.notify(this,element);}}

//Notthreadsafe.Containsasubtlebug.

25 17-214

Example:addingconcurrencytotheobserverpattern

privatefinalList<Observer<E>>observers=newArrayList<>();publicvoidaddObserver(Observer<E>observer){synchronized(observers){observers.add(observer);}}publicbooleanremoveObserver(Observer<E>observer){synchronized(observers){returnobservers.remove(observer);}}privatevoidnotifyOf(Eelement){synchronized(observers){for(Observer<E>observer:observers)observer.notify(this,element);//Riskslivenessand}//safetyfailures!}

26 17-214

Onesolution:snapshotiteration

privatevoidnotifyOf(Eelement){List<Observer<E>>snapshot=null;synchronized(observers){snapshot=newArrayList<>(observers);}for(Observer<E>observer:snapshot){observer.notify(this,element);//Safe}}

27 17-214

Abettersolution:CopyOnWriteArrayList

privatefinalList<Observer<E>>observers=newCopyOnWriteArrayList<>();publicvoidaddObserver(Observer<E>observer){observers.add(observer);}publicbooleanremoveObserver(Observer<E>observer){returnobservers.remove(observer);}privatevoidnotifyOf(Eelement){for(Observer<E>observer:observers)observer.notify(this,element);}

28 17-214

Definingyourownthread-safeobjects

•  Identifyvariablesthatrepresenttheobject'sstate•  Identifyinvariantsthatconstrainthestatevariables•  Establishapolicyformaintaininginvariantswithconcurrent

accesstostate

29 17-214

Policiesforthreadsafety(again)

•  Thread-confined•  Sharedread-only•  Sharedthread-safe

–  Objectsthatperforminternalsynchronization

•  Guarded–  Objectsthatmustbesynchronizedexternally

30 17-214

Atoyexample:Read-writelocks(a.k.a.sharedlocks)

privatefinalRwLocklock=newRwLock();lock.readLock();try{//Dostuffthatrequiresread(shared)lock}finally{lock.unlock();}lock.writeLock();try{//Dostuffthatrequireswrite(exclusive)lock}finally{lock.unlock();}

Sampleclientcode:

31 17-214

Anaside:MoreJavaprimitives,forcoordination

•  Goal:guardedsuspensionwithoutspin-waitingvolatilebooleanready=…;while(!ready);//loopuntilready…

•  Objectmethodsforcoordination:voidwait();voidwait(longtimeout);voidnotify();voidnotifyAll();

32 17-214

Atoyexample:Read-writelocks(implementation1/2)

@ThreadSafepublicclassRwLock{//StatefieldsareprotectedbyRwLock'sintrinsiclock/**Numthreadsholdinglockforread.*/@GuardedBy("this")privateintnumReaders=0;/**Whetherlockisheldforwrite.*/@GuardedBy("this")privatebooleanwriteLocked=false;publicsynchronizedvoidreadLock()throwsInterruptedException{while(writeLocked){wait();}numReaders++;}

33 17-214

Atoyexample:Read-writelocks(implementation2/2)

publicsynchronizedvoidwriteLock()throwsInterruptedException{while(numReaders!=0||writeLocked){wait();}writeLocked=true;}publicsynchronizedvoidunlock(){if(numReaders>0){numReaders--;}elseif(writeLocked){writeLocked=false;}else{thrownewIllegalStateException("Locknotheld");}notifyAll();//Wakeanywaiters}}

34 17-214

Adviceforbuildingthread-safeobjects

•  Doaslittleaspossibleinsynchronizedregion:getin,getout–  Obtainlock–  Examineshareddata–  Transformasnecessary–  Dropthelock

•  Ifyoumustdosomethingslow,moveitoutsidethesynchronizedregion

35 17-214

Documentation

•  Documentaclass'sthreadsafetyguaranteesforitsclients•  Documentaclass'ssynchronizationpolicyforitsmaintainers•  Use@ThreadSafe,@GuardedByannotations

36 17-214

SummaryofourRwLockexample

•  Generally,avoidwait/notify•  Neverinvokewaitoutsidealoop

–  Mustcheckcoordinationconditionafterwaking

•  GenerallyusenotifyAll,notnotify•  DonotuseourRwLock–it'sjustatoy

–  Instead,knowthestandardlibraries…•  Discuss:sun.misc.Unsafe

37 17-214

Today

•  Strategiesforsafety•  Javalibrariesforconcurrency•  Buildingthread-safedatastructures

–  Javaprimitivesforconcurrentcoordination

•  Programstructureforconcurrency

38 17-214

Producer-consumerdesignpattern

•  Goal:Decoupletheproducerandtheconsumerofsomedata•  Consequences:

–  Removescodedependencybetweenproducersandconsumers–  Producersandconsumerscanproduceandconsumeatdifferentrates

39 17-214

java.util.concurrent.BlockingQueue

•  Implementsjava.util.Queue<E>•  java.util.concurrent.SynchronousQueue

–  Eachputdirectlywaitsforacorrespondingpoll•  java.util.concurrent.ArrayBlockingQueue

–  put blocksifthequeueisfull–  poll blocksifthequeueisempty

40 17-214

Thefork-joinpattern

if(myportionoftheworkissmall)dotheworkdirectlyelsesplitmyworkintopiecesinvokethepiecesandwaitfortheresults

Image from: Wikipedia

41 17-214

Themembranepattern

•  Multipleroundsoffork-join,eachroundwaitingforthepreviousroundtocomplete

Image from: Wikipedia

42 17-214

Executionoftasks

•  Naturalboundariesofcomputationdefinetasks,e.g.:publicclassSingleThreadWebServer{publicstaticvoidmain(String[]args)throwsIOException{ServerSocketsocket=newServerSocket(80);while(true){Socketconnection=socket.accept();handleRequest(connection);}}privatestaticvoidhandleRequest(Socketconnection){…//request-handlinglogichere}}

43 17-214

Apoordesignchoice:Athreadpertask

publicclassThreadPerRequestWebServer{publicstaticvoidmain(String[]args)throwsIOException{ServerSocketsocket=newServerSocket(80);while(true){Socketconnection=socket.accept();newThread(()->handleRequest(connection)).start();}}privatestaticvoidhandleRequest(Socketconnection){…//request-handlinglogichere}}

44 17-214

Thej.u.cexecutorframework

•  Keyabstractions–  Runnable,Callable<T>:kindsoftasks

•  Executor:thingthatexecutestasks•  Future<T>:apromisetogiveyouaT•  Executorservice:AnExecutorthat

–  Letsyoumanagetermination–  CanproduceFutureinstances

45 17-214

Aframeworkforasynchronouscomputation

•  Thejava.util.concurrent.Future<V>interfaceVget();Vget(longtimeout,TimeUnitunit);booleanisDone();booleancancel(booleanmayInterruptIfRunning);booleanisCancelled();

46 17-214

Aframeworkforasynchronouscomputation

•  Thejava.util.concurrent.Future<V>interfaceVget();Vget(longtimeout,TimeUnitunit);booleanisDone();booleancancel(booleanmayInterruptIfRunning);booleanisCancelled();

•  Thejava.util.concurrent.ExecutorServiceinterfacevoidexecute(Runnabletask);Futuresubmit(Runnabletask);Future<V>submit(Callable<V>task);List<Future<V>>invokeAll(Collection<Callable<V>>tasks);Future<V>invokeAny(Collection<Callable<V>>tasks);

47 17-214

Executorsforcommoncomputationalpatterns

•  Fromthejava.util.concurrent.ExecutorsclassstaticExecutorServicenewSingleThreadExecutor();staticExecutorServicenewFixedThreadPool(intn);staticExecutorServicenewCachedThreadPool();staticExecutorServicenewScheduledThreadPool(intn);

48 17-214

Exampleuseofexecutorservice

publicclassThreadPoolWebServer{privatestaticfinalExecutorexec=Executors.newFixedThreadPool(100);//100threadspublicstaticvoidmain(String[]args)throwsIOException{ServerSocketsocket=newServerSocket(80);while(true){Socketconnection=socket.accept();exec.execute(()->handleRequest(connection));}}privatestaticvoidhandleRequest(Socketconnection){…//request-handlinglogichere}}

49 17-214

Summary

•  Reuse,don'tbuild:knowthej.u.c.libraries•  Usecommonpatternsforprogramstructure

–  Decomposeworkintoindependenttasks