lecture 7 concurrent queues and stacks › wies › teaching › ppc-14 › material ›...

Programming Paradigms for Concurrency Lecture 7 – Concurrent Queues and Stacks

Based on companion slides for The Art of Multiprocessor Programming

by Maurice Herlihy & Nir Shavit

Modified by Thomas Wies

New York University

TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA

2 2

The Five-Fold Path

• Coarse-grained locking

• Fine-grained locking

• Optimistic synchronization

• Lazy synchronization

• Lock-free synchronization

3 3

Another Fundamental Problem

• I told you about

– Sets implemented by linked lists

• Next: queues

• Next: stacks

4 4

Queues & Stacks

• pool of items

5 5

Queues

deq()/ enq( )

Total order

First in

First out

6 6

Stacks

pop()/

push( )

Total order

Last in

First out

7 7

Bounded

• Fixed capacity

• Good when resources an issue

8 8

Unbounded

• Unlimited capacity

• Often more convenient

9

Blocking

zzz …

Block on attempt to remove from empty stack or queue

10

Blocking

zzz …

Block on attempt to add to full bounded stack or queue

11

Non-Blocking

Ouch!

Throw exception on attempt to remove from empty stack or queue

12 12

This Lecture

• Queue

– Bounded, blocking, lock-based

– Unbounded, non-blocking, lock-free

• Stack

– Unbounded, non-blocking lock-free

– Elimination-backoff algorithm

13 13

Queue: Concurrency

enq(x) y=deq()

enq() and deq()

work at different

ends of the object

tail head

14 14

Concurrency

enq(x)

Challenge: what if

the queue is empty

or full?

y=deq()

15 15

Bounded Queue

Sentinel

head

tail

16 16

Bounded Queue

head

tail

First actual item

17 17

Bounded Queue

head

tail

Lock out other

deq() calls

deqLock

18 18

Bounded Queue

head

tail

Lock out other

enq() calls

deqLock

enqLock

19 19

Not Done Yet

Need to tell whether

queue is full or

empty

head

tail

deqLock

enqLock

20 20

Not Done Yet

head

tail

deqLock

enqLock

Max size is 8 items

size

1

21 21

Not Done Yet

head

tail

deqLock

enqLock

Incremented by enq()

Decremented by deq()

size

1

22 22

Enqueuer

head

tail

deqLock

enqLock

size

1

Lock enqLock

23 23

Enqueuer

head

tail

deqLock

enqLock

size

1

Read size

OK

24 24

Enqueuer

head

tail

deqLock

enqLock

size

1

No need to

lock tail

25 25

Enqueuer

head

tail

deqLock

enqLock

size

1

Enqueue Node

26 26

Enqueuer

head

tail

deqLock

enqLock

size

1 2

getAndincrement()

27 27

Enqueuer

head

tail

deqLock

enqLock

size

8 Release lock

2

28 28

Enqueuer

head

tail

deqLock

enqLock

size

2

If queue was empty,

notify waiting dequeuers

29 29

Unsuccesful Enqueuer

head

tail

deqLock

enqLock

size

8

Uh-oh

Read size

…

30 30

Dequeuer

head

tail

deqLock

enqLock

size

2

Lock deqLock

31 31

Dequeuer

head

tail

deqLock

enqLock

size

2

Read sentinel’s

next field

OK

32 32

Dequeuer

head

tail

deqLock

enqLock

size

2

Read value

33 33

Dequeuer

head

tail

deqLock

enqLock

size

2

Make first Node

new sentinel

34 34

Dequeuer

head

tail

deqLock

enqLock

size

1

Decrement

size

35 35

Dequeuer

head

tail

deqLock

enqLock

size

1 Release

deqLock

36 36

Unsuccesful Dequeuer

head

tail

deqLock

enqLock

size

0

Read sentinel’s

next field

uh-oh

37 37

Bounded Queue

public class BoundedQueue<T> {

ReentrantLock enqLock, deqLock;

Condition notEmptyCondition, notFullCondition;

AtomicInteger size;

Node head;

Node tail;

int capacity;

enqLock = new ReentrantLock();

notFullCondition = enqLock.newCondition();

deqLock = new ReentrantLock();

notEmptyCondition = deqLock.newCondition();

}

38 38

Bounded Queue




AtomicInteger size;

Node head;

Node tail;

int capacity;





}

enq & deq locks

39 39

Bounded Queue Fields




AtomicInteger size;

Node head;

Node tail;

int capacity;





}

Enq lock’s associated

condition

40 40





AtomicInteger size;

Node head;

Node tail;

int capacity;





}

size: 0 to capacity

41 41





AtomicInteger size;

Node head;

Node tail;

int capacity;





}

Head and Tail

42 42

Enq Method Part One public void enq(T x) {

boolean mustWakeDequeuers = false;

enqLock.lock();

try {

while (size.get() == Capacity)

notFullCondition.await();

Node e = new Node(x);

tail.next = e;

tail = tail.next;

if (size.getAndIncrement() == 0)

mustWakeDequeuers = true;

} finally {

enqLock.unlock();

}

…

}

43 43

public void enq(T x) {


enqLock.lock();

try {

while (size.get() == capacity)



tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Enq Method Part One

Lock and unlock

enq lock

44 44



enqLock.lock();

try {




tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Enq Method Part One

Wait while queue is full …

45 45



enqLock.lock();

try {




tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Enq Method Part One

when await() returns, you

might still fail the test !

46 46



enqLock.lock();

try {




tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Be Afraid

After the loop: how do we know the

queue won’t become full again?

47 47



enqLock.lock();

try {




tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Enq Method Part One

Add new node

48 48



enqLock.lock();

try {




tail.next = e;

tail = tail.next;



} finally {

enqLock.unlock();

}

…

}

Enq Method Part One

If queue was empty, wake

frustrated dequeuers

49 49

Beware Lost Wake-Ups

Cri

tical S

ecti

on

waiting room

lock()

Queue empty

so signal ()

enq( )

unlock()

Yawn!

50 50

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

lock()

enq( )

unlock()

Yawn!

Queue not

empty so no

need to signal

51 51

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Yawn!

52 52

Lost Wake-Up

Cri

tical S

ecti

on

waiting room

Found it

53 53

What’s Wrong Here?

Cri

tical S

ecti

on

waiting room

Still waiting ….!

54 54

Enq Method Part Deux


…

if (mustWakeDequeuers) {

deqLock.lock();

try {

notEmptyCondition.signalAll();

} finally {

deqLock.unlock();

}

}

}

55 55



…


deqLock.lock();

try {


} finally {

deqLock.unlock();

}

}

} Are there dequeuers to be signaled?

56 56


…


deqLock.lock();

try {


} finally {

deqLock.unlock();

}

}

}


Lock and

unlock deq lock

57 57


…


deqLock.lock();

try {


} finally {

deqLock.unlock();

}

}

}


Signal dequeuers that

queue is no longer empty

58 58

The Enq() & Deq() Methods

• Share no locks

– That’s good

• But do share an atomic counter

– Accessed on every method call

– That’s not so good

• Can we alleviate this bottleneck?

59 59

Split the Counter

• The enq() method

– Increments only

– Cares only if value is capacity

• The deq() method

– Decrements only

– Cares only if value is zero

60 60

Split Counter

• Enqueuer increments enqSize

• Dequeuer decrements deqSize

• When enqueuer runs out

– Locks deqLock

– computes size = enqSize - DeqSize

• Intermittent synchronization

– Not with each method call

– Need both locks! (careful …)

61 61

A Lock-Free Queue

Sentinel

head

tail

62 62

Compare and Set

CAS

63 63

Enqueue

head

tail

enq( )

64 64

Enqueue

head

tail

65 65

Logical Enqueue

head

tail

CAS

66 66

Physical Enqueue

head

tail CAS

67 67

Enqueue

• These two steps are not atomic

• The tail field refers to either

– Actual last Node (good)

– Penultimate Node (not so good)

• Be prepared!

68 68

Enqueue

• What do you do if you find

– A trailing tail?

• Stop and help fix it

– If tail node has non-null next field

– CAS the queue’s tail field to tail.next

69 69

When CASs Fail

• During logical enqueue

– Abandon hope, restart

– Still lock-free (why?)

• During physical enqueue

– Ignore it (why?)

70 70

Dequeuer

head

tail

Read value

71 71

Dequeuer

head

tail

Make first Node

new sentinel

CAS

72 72

Memory Reuse?

• What do we do with nodes after we

dequeue them?

• Java: let garbage collector deal?

• Suppose there is no GC, or we prefer

not to use it?

73 73

Dequeuer

head

tail

CAS

Can recycle

74 74

Simple Solution

• Each thread has a free list of unused

queue nodes

• Allocate node: pop from list

• Free node: push onto list

• Deal with underflow somehow …

75 75

Why Recycling is Hard

Free pool

head tail

Want to

redirect

head from

gray to red

zzz…

76 76

Both Nodes Reclaimed

Free pool

zzz

head tail

77 77

One Node Recycled

Free pool

Yawn!

head tail

78 78

Why Recycling is Hard

Free pool

CAS

head tail

OK, here

I go!

79 79

Recycle FAIL

Free pool

OMG what went wrong?

head tail

80 80

The Dreaded ABA Problem head tail

Head reference has value A

Thread reads value A

81 81

Dreaded ABA continued

zzz

head tail

Head reference has value B

Node A freed

82 82


Yawn!

head tail

Head reference has value A again

Node A recycled and reinitialized

83 83


CAS

head tail

CAS succeeds because references match,

even though reference’s meaning has changed

84 84

The Dreaded ABA FAIL

• Is a result of CAS() semantics

– blame Sun, Intel, AMD, …

• Not with Load-Locked/Store-Conditional

– Good for IBM, ARM?

85 85

Dreaded ABA – A Solution

• Tag each pointer with a counter

• Unique over lifetime of node

• Pointer size vs word size issues

• Overflow?

– Don’t worry be happy?

– Bounded tags?

• AtomicStampedReference class

86 86

Atomic Stamped Reference

• AtomicStampedReference class

– Java.util.concurrent.atomic package

address S

Stamp

Reference

Can get reference & stamp atomically

87 87

Concurrent Stack

• Methods

– push(x)

– pop()

• Last-in, First-out (LIFO) order

• Lock-Free!

88 88

Empty Stack

Top

89 89

Push

Top

90 90

Push

Top CAS

91 91

Push

Top

92 92

Push

Top

93 93

Push

Top

94 94

Push

Top CAS

95 95

Push

Top

96 96

Pop

Top

97 97

Pop

Top CAS

98 98

Pop

Top CAS

mine!

99 99

Pop

Top CAS

100 100

Pop

Top

101 101

public class LockFreeStack {

private AtomicReference top =

new AtomicReference(null);

public boolean tryPush(Node node){

Node oldTop = top.get();

node.next = oldTop;

return(top.compareAndSet(oldTop, node))

}

public void push(T value) {

Node node = new Node(value);

while (true) {

if (tryPush(node)) {

return;

} else backoff.backoff();

}}

Lock-free Stack

102 102


private AtomicReference top = new

AtomicReference(null);

public Boolean tryPush(Node node){


node.next = oldTop;


}



while (true) {


return;

} else backoff.backoff()

}}

Lock-free Stack

tryPush attempts to push a node

103 103






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

Read top value

104 104






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

current top will be new node’s successor

105 105






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

Try to swing top, return success or failure

106 106






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

Push calls tryPush

107 107






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

Create new node

108 108






node.next = oldTop;


}



while (true) {


return;


}}

Lock-free Stack

If tryPush() fails,

back off before retrying

109 109

Lock-free Stack

• Good

– No locking

• Bad

– Without GC, fear ABA

– Without backoff, huge contention at top

– In any case, no parallelism

110 110

Big Question

• Are stacks inherently sequential?

• Reasons why

– Every pop() call fights for top item

• Reasons why not

– Stay tuned …

111 111

Elimination-Backoff Stack

• How to

– “turn contention into parallelism”

• Replace familiar

– exponential backoff

• With alternative

– elimination-backoff

112 112

Observation

Push( )

Pop()

linearizable stack

After an equal number

of pushes and pops,

stack stays the same Yes!

113 113

Idea: Elimination Array

Push( )

Pop()

stack

Pick at

random

Pick at

random Elimination

Array

114 114

Push Collides With Pop

Push( )

Pop()

stack

continue

continue

No need to

access stack Yes!

115 115

No Collision

Push( )

Pop()

stack

If no collision,

access stack

If pushes collide or

pops collide

access stack

116 116


• Lock-free stack + elimination array

• Access Lock-free stack,

– If uncontended, apply operation

– if contended, back off to elimination array

and attempt elimination

117 117


Push( )

Pop() Top CAS

If CAS fails, back off

118 118

Dynamic Range and Delay

Push( )

Pick random range and

max waiting time based

on level of contention

encountered

119 119

Linearizability

• Un-eliminated calls

– linearized as before

• Eliminated calls:

– linearize pop() immediately after matching

push()

• Combination is a linearizable stack

120 120

Un-Eliminated Linearizability

push(v1)

time time

push(v1)

pop(v1) pop(v1)

121 121

Eliminated Linearizability

pop(v2) push(v1)

push(v2)

time time

push(v2)

pop(v2) push(v1)

pop(v1)

Collision

Point

Red calls are

eliminated

pop(v1)

122 122

Backoff Has Dual Effect

• Elimination introduces parallelism

• Backoff to array cuts contention on lock-

free stack

• Elimination in array cuts down number

of threads accessing lock-free stack

123 123

public class EliminationArray {

private static final int duration = ...;

private static final int timeUnit = ...;

Exchanger<T>[] exchanger;

public EliminationArray(int capacity) {

exchanger = new Exchanger[capacity];

for (int i = 0; i < capacity; i++)

exchanger[i] = new Exchanger<T>();

…

}

…

}

Elimination Array

124 124


private static final int duration = ...;

private static final int timeUnit = ...;

Exchanger<T>[] exchanger;

public EliminationArray(int capacity) {

exchanger = new Exchanger[capacity];

for (int i = 0; i < capacity; i++)

exchanger[i] = new Exchanger<T>();

…

}

…

}

Elimination Array

An array of Exchangers

125 125

public class Exchanger<T> {

AtomicStampedReference<T> slot

= new AtomicStampedReference<T>(null, 0);

Digression: A Lock-Free

Exchanger

126 126

public class Exchanger<T> {

AtomicStampedReference<T> slot

= new AtomicStampedReference<T>(null, 0);

A Lock-Free Exchanger

Atomically modifiable

reference + status

127 127

public T Exchange(T myItem, long nanos)

throws TimeoutException {

long timeBound = System.nanoTime() + nanos;

int[] stampHolder = {EMPTY};

while (true) {

if (System.nanoTime() > timeBound)

throw new TimeoutException();

T herItem = slot.get(stampHolder);

int stamp = stampHolder[0];

switch(stamp) {

case EMPTY: … // slot is free

case WAITING: … // someone waiting for me

case BUSY: … // others exchanging

}

}

The Exchange

Art of Multiprocessor Programming 128 128





while (true) {





switch(stamp) {




}

}

The Exchange

Item and timeout

Art of Multiprocessor Programming 129 129





while (true) {





switch(stamp) {




}

}

The Exchange

Array holds status

130 130





while (true) {





switch(stamp) {




}

}

The Exchange

Loop until timeout

131 131





while (true) {





switch(stamp) {




}

}

The Exchange

Get other’s item and status

132 132





while (true) {





switch(stamp) {




}

}

The Exchange

An Exchanger has three possible states

133 133

Lock-free Exchanger

EMPTY

134 134

EMPTY

Lock-free Exchanger

CAS

135 135

WAITING

Lock-free Exchanger

136 136

Lock-free Exchanger

In search of

partner …

WAITING

137 137

WAITING

Lock-free Exchanger

Slot

Still waiting …

Try to exchange

item and set

status to BUSY

CAS

138 138

BUSY

Lock-free Exchanger

Slot

Partner showed

up, take item and

reset to EMPTY

item status

139 139

EMPTY BUSY

Lock-free Exchanger

Slot

item status

Partner showed

up, take item and

reset to EMPTY

140 140

case EMPTY: // slot is free

if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {

while (System.nanoTime() < timeBound){

herItem = slot.get(stampHolder);

if (stampHolder[0] == BUSY) {

slot.set(null, EMPTY);

return herItem;

}}

if (slot.CAS(myItem, null, WAITING, EMPTY)){


} else {



return herItem;

}

} break;

Exchanger State EMPTY

141 141







return herItem;

}}



} else {



return herItem;

}

} break;


Try to insert myItem and

change state to WAITING

142 142







return herItem;

}}



} else {



return herItem;

}

} break;


Spin until either

myItem is taken or timeout

143 143







return herItem;

}}



} else {



return herItem;

}

} break;


myItem was taken,

so return herItem

that was put in its place

144 144







return herItem;

}}



} else {



return herItem;

}

} break;


Otherwise we ran out of time,

try to reset status to EMPTY

and time out

145 145


if (slot.CAS(herItem, myItem, WAITING, BUSY)) {





return herItem;

}}



} else {

herItem = slot.get(stampHolder)


return herItem;

}

} break;


If reset failed,

someone showed up after all,

so take that item

146 146







return herItem;

}}



} else {



return herItem;

}

} break;


Clear slot and take that item

147 147







return herItem;

}}



} else {



return herItem;

}

} break;


If initial CAS failed,

then someone else changed status

from EMPTY to WAITING,

so retry from start

148 148

case WAITING: // someone waiting for me

if (slot.CAS(herItem, myItem, WAITING, BUSY))

return herItem;

break;

case BUSY: // others in middle of exchanging

break;

default: // impossible

break;

}

}

}

}

States WAITING and BUSY

149 149



return herItem;

break;


break;


break;

}

}

}

}


someone is waiting to exchange,

so try to CAS my item in

and change state to BUSY

150 150



return herItem;

break;


break;


break;

}

}

}

}


If successful, return other’s item,

otherwise someone else took it,

so try again from start

151 151



return herItem;

break;


break;


break;

}

}

}

}


If BUSY,

other threads exchanging,

so start again

152 152

The Exchanger Slot

• Exchanger is lock-free

• Because the only way an exchange can

fail is if others repeatedly succeeded or

no-one showed up

• The slot we need does not require

symmetric exchange

153 153


…

public T visit(T value, int range)


int slot = random.nextInt(range);

int nanodur = convertToNanos(duration, timeUnit));

return (exchanger[slot].exchange(value, nanodur)

}}

Back to the Stack: the

Elimination Array

154 154


…






}}

Elimination Array

visit the elimination array

with fixed value and range

155 155


…






}}

Elimination Array

Pick a random array entry

156 156


…






}}

Elimination Array

Exchange value or time out

157 157


...

while (true) {


return;

} else try {

T otherValue =

eliminationArray.visit(value,policy.range);

if (otherValue == null) {

return;

}

}

Elimination Stack Push

158 158


...

while (true) {


return;

} else try {

T otherValue =



return;

}

}


First, try to push

159 159


...

while (true) {


return;

} else try {

T otherValue =



return;

}

}


If I failed, backoff & try to eliminate

160 160


...

while (true) {


return;

} else try {

T otherValue =



return;

}

}


Value pushed and range to try

161 161


...

while (true) {


return;

} else try {

T otherValue =



return;

}

}


Only pop() leaves null,

so elimination was successful

162 162


...

while (true) {


return;

} else try {

T otherValue =



return;

}

}


Otherwise, retry push() on lock-free stack

163 163

public T pop() {

...

while (true) {

if (tryPop()) {

return returnNode.value;

} else

try {

T otherValue =

eliminationArray.visit(null,policy.range;

if (otherValue != null) {

return otherValue;

}

}

}}

Elimination Stack Pop

164 164

public T pop() {

...

while (true) {

if (tryPop()) {

return returnNode.value;

} else

try {

T otherValue =

eliminationArray.visit(null,policy.range;

if ( otherValue != null) {

return otherValue;

}

}

}}

Elimination Stack Pop

If value not null, other thread is a push(),

so elimination succeeded

165 165

Summary

• We saw both lock-based and lock-free implementations of

• queues and stacks

• Don’t be quick to declare a data structure inherently sequential

– Linearizable stack is not inherently sequential (though it is in worst case)

• ABA is a real problem, pay attention

166 166

This work is licensed under a Creative Commons Attribution-

ShareAlike 2.5 License.

• You are free:

– to Share — to copy, distribute and transmit the work

– to Remix — to adapt the work

• Under the following conditions:

– Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).

– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to

– http://creativecommons.org/licenses/by-sa/3.0/.

• Any of the above conditions can be waived if you get permission from the copyright holder.

• Nothing in this license impairs or restricts the author's moral rights.

http://creativecommons.org/licenses/by-sa/2.5/







lecture 7 concurrent queues and stacks › wies › teaching › ppc-14 › material ›...

Documents