concurrent queues companion slides for the art of multiprocessor programming by maurice herlihy...

Concurrent Queues

Companion slides forThe Art of Multiprocessor

Programmingby Maurice Herlihy & Nir Shavit

Art of Multiprocessor Programming© Herlihy-Shavit

2007

2

Another Fundamental Problem

• We told you about – Sets implemented using linked lists

• Next: queues• Next: stacks


2007

3

Queues

• Pool of items• Queue

– enq() & deq()– First-in-first-out (FIFO) order


2007

4

Bounded vs Unbounded

• Bounded– Fixed capacity– Good when resources an issue

• Unbounded– Holds any number of objects


2007

5

Blocking vs Non-Blocking

• Problem cases:– Removing from empty pool– Adding to full (bounded) pool

• Blocking– Caller waits until state changes

• Non-Blocking– Method throws exception


2007

6

This Lecture

• Bounded, Blocking, Lock-based Queue

• Unbounded, Non-Blocking, Lock-free Queue

• ABA problem• Unbounded Non-Blocking Lock-free

Stack• Elimination-Backoff Stack


2007

7

Queue: Concurrency

enq(x) y=deq()

enq() and deq() work at

different ends of the object

tail head


2007

8

Concurrency

enq(x)

Challenge: what if the queue is empty or full?

y=deq()ta

ilhead


2007

9

Bounded Queue

Sentinel

head

tail


2007

10

Bounded Queue

head

tail

First actual item


2007

11

Bounded Queue

head

tail

Lock out other deq() calls

deqLock


2007

12

Bounded Queue

head

tail

Lock out other enq() calls

deqLock

enqLock


2007

13

Not Done Yet

head

tail

deqLock

enqLock

Need to tell whether queue is

full or empty


2007

14

Not Done Yet

head

tail

deqLock

enqLock

Permission to enqueue 8 items

permits

8


2007

15

Not Done Yet

head

tail

deqLock

enqLock

Incremented by deq()Decremented by enq()

permits

8


2007

16

Enqueuer

head

tail

deqLock

enqLock

permits

8

Lock enqLock


2007

17

Enqueuer

head

tail

deqLock

enqLock

permits

8

Read permits

OK


2007

18

Enqueuer

head

tail

deqLock

enqLock

permits

8

No need to lock

tail


2007

19

Enqueuer

head

tail

deqLock

enqLock

permits

8

Enqueue Node


2007

20

Enqueuer

head

tail

deqLock

enqLock

permits

87

getAndDecrement()


2007

21

Enqueuer

head

tail

deqLock

enqLock

permits

8 Release lock7


2007

22

Enqueuer

head

tail

deqLock

enqLock

permits

7

If queue was empty, notify/signal waiting

dequeuers


2007

23

Unsuccesful Enqueuer

head

tail

deqLock

enqLock

permits

0

Uh-oh

Read permits


2007

24

Dequeuer

head

tail

deqLock

enqLock

permits

8

Lock deqLock


2007

25

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read sentinel’s next field

OK


2007

26

Dequeuer

head

tail

deqLock

enqLock

permits

7

Read value


2007

27

Dequeuer

head

tail

deqLock

enqLock

permits

7

Make first Node new sentinel


2007

28

Dequeuer

head

tail

deqLock

enqLock

permits

8

Increment permits


2007

29

Dequeuer

head

tail

deqLock

enqLock

permits

7Release deqLock


2007

30

Unsuccesful Dequeuer

head

tail

deqLock

enqLock

permits

8

Read sentinel’s next field

uh-oh


2007

31

Bounded Queue

public class BoundedQueue<T> { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition();}


2007

32

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Create condition to wait on


2007

33

Lock Conditions

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }


2007

34


Lock Conditions

Release lock and wait on condition


2007

35


Lock Conditions

Wake up one waiting thread


2007

36


Lock Conditions

Wake up all waiting threads


2007

37

Await

• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns

q.await()


2007

38

Signal

• Awakens one waiting thread– Which will reacquire lock

q.signal();


2007

39

Signal All

• Awakens all waiting threads– Which will each reacquire lock

q.signalAll();


2007

40

A Monitor Lock

Cri

tical S

ecti

on

waiting roomLock(

)

unLock()


2007

41

Unsuccessful Deq

Cri

tical S

ecti

on

waiting roomLock

()

await()

Deq()

Oh no, Empty!


2007

42

Another One

Cri

tical S

ecti

on

waiting roomLock

()

await()

Deq()

Oh no, Empty!


2007

43

Enqueur to the Rescue

Cri

tical S

ecti

on

waiting roomLock(

)

signalAll()

Enq( )

unLock()

Yawn!Yawn!


2007

44

Yawn!

Monitor Signalling

Cri

tical S

ecti

on

waiting room

Yawn!

Awakend thread might still lose lock to outside contender…


2007

45

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Found it

Yawn!


2007

46

Yawn!

Dequeurs Signalled

Cri

tical S

ecti

on

waiting room

Still empty!


2007

47

Dollar Short + Day Late

Cri

tical S

ecti

on

waiting room


2007 48

Lost Wake-Up

Cri

tical S

ecti

on

waiting roomLock(

)

signal ()Enq( )

unLock()

Yawn!


2007

49

Lost Wake-Up

Cri

tical S

ecti

on

waiting roomLock(

)

Enq( )

unLock()

Yawn!


2007

50

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Yawn!


2007

51

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Found it


2007

52

What’s Wrong Here?

Cri

tical S

ecti

on

waiting room

zzzz….!

Solution to Lost Wakeup

• Always use signalAll and notifyAll • Not signal and notify


2007

53


2007

54

The Bounded Queue



2007

55

Bounded Queue Fields


Enq & deq locks


2007

56



Enq lock’s associated condition


2007

57



Num permits: 0 to capacity


2007

58



Head and Tail


2007

59

Enq Method Part Onepublic void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}


2007

60

public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } …}

Enq Method Part One

Lock and unlock enq

lock


2007

61


Enq Method Part One

If queue is full, patiently await further

instructions …


2007

62


Enq Method Part One

Add new node


2007

63


Enq Method Part One

If queue was empty, wake frustrated dequeuers


2007

64

Enq Method Part Deux

public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }


2007

65


public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } } }Are there dequeuers to be signaled?


2007

66



Lock and unlock deq

lock


2007

67



Signal dequeuers that queue no longer

empty


2007

68

The Enq() & Deq() Methods

• Share no locks– That’s good

• But do share an atomic counter– Accessed on every method call– That’s not so good

• Can we alleviate this bottleneck?


2007

69

A Lock-Free Queue

Sentinel

head

tail


2007

70

Compare and Set

CAS


2007

71

Enqueue

head

tail

Enq( )


2007

72

Enqueue

head

tail


2007

73

Logical Enqueue

head

tail

CAS


2007

74

Physical Enqueue

head

tail

Enqueue Node

CAS


2007

75

Enqueue

• These two steps are not atomic• The tail field refers to either

– Actual last Node (good)– Penultimate Node (not so good)

• Be prepared!


200776

Enqueue

• What do you do if you find– A trailing tail?

• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next

• As in the universal construction


2007

77

When CASs Fail

• During logical enqueue– Abandon hope, restart– Still lock-free (why?)

• During physical enqueue– Ignore it (why?)


2007

78

Enq Methodpublic void enq (T value) {

Node node = new Node(value);while (true) {

Node last = tail.get();Node next = last.next.get();if (last == tail.get()) {

if (next == null) {if (last.next.compareAndSet(next,

node)) {tail.compareAndSet(last, node);return;

}} else {

tail.compareAndSet(last, next);}

}}

}


2007

79

Dequeuer

head

tail

Read value


2007

80

Dequeuer

head

tail

Make first Node new sentinel

CAS


2007

81

Deq Methodpublic T deq() throws EmptyException {

while (true) {Node first = head.get();Node last = tail.get();Node next = first.next.get();if (first == head.get()) {

if (first == last) {if (next == null) {

throw new EmptyException(); }tail.compareAndSet(last, next);

} else {T value = next.value;if (head.compareAndSet(first, next))return value;

}}}}


2007

82

Memory Reuse?

• What do we do with nodes after we dequeue them?

• Java: let garbage collector deal?• Suppose there is no GC, or we

prefer not to use it?


2007

83

Dequeuer

head

tail

CAS

Can recycle


2007

84

Simple Solution

• Each thread has a free list of unused queue nodes

• Allocate node: pop from list• Free node: push onto list• Deal with underflow somehow …


2007

85

Why Recycling is Hard

Free pool

head tail

Want to redirect

tailfrom

grey to red

zzz…


2007

86

Both Nodes Reclaimed

Free pool

zzz

head tail


2007

87

One Node Recycled

Free pool

Yawn!

head tail


2007

88

Why Recycling is Hard

Free pool

CAShead tail

OK, here I go!


2007

89

Final State

Free pool

zOMG what went wrong?

head tail

Bad news


2007

90

The Dreaded ABA Problem

Head pointer has value AThread reads value A

head tail


2007

91

Dreaded ABA continued

zzz Head pointer has value BNode A freed

head tail


2007

92


Yawn! Head pointer has value A againNode A recycled & reinitialized

head tail


2007

93


CAS succeeds because pointer matcheseven though pointer’s meaning has changed

CAShead tail


2007

94

The Dreaded ABA Problem

• Is a result of CAS() semantics– I blame Sun, Intel, AMD, …

• Not with Load-Locked/Store-Conditional– Good for IBM?


2007

95

Dreaded ABA – A Solution

• Tag each pointer with a counter• Unique over lifetime of node• Pointer size vs word size issues• Overflow?

– Don’t worry be happy?– Bounded tags?

• AtomicStampedReference class


2007

96

Atomic Stamped Reference

• AtomicStampedReference class– Java.util.concurrent.atomic package

address S

Stamp

Reference

Can get reference and stamp atomically


2007

97

This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.

• You are free:– to Share — to copy, distribute and transmit the work – to Remix — to adapt the work

• Under the following conditions:– Attribution. You must attribute the work to “The Art of

Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).

– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to– http://creativecommons.org/licenses/by-sa/3.0/.

• Any of the above conditions can be waived if you get permission from the copyright holder.

• Nothing in this license impairs or restricts the author's moral rights.

http://creativecommons.org/licenses/by-sa/2.5/



concurrent queues companion slides for the art of multiprocessor programming by maurice herlihy...

Documents