Skiplist-based Concurrent Priority Queues

Itay LotanStanford University

Nir ShavitSun Microsystems Laboratories

2

Priority Queues For Large-Scale MultiProcessor Machines

• Priority Queue – A data structure that allows n asynchronous processes to insert an item, or delete the item with highest priority

• Large Scale Machine – Hundreds of Processors– Usually different architecture than small scale

machines

3

Fixed Set Priority Queue

• A priority queue with a fixed and predetermined set of priorities

• Used by operating systems to distribute CPU time.

• There is a scalable solution - Funnel Tree Funnel Tree [Shavit, Zemach][Shavit, Zemach]

4

General Priority Queue

• Supports an unlimited range of priorities

• Between any two priorities may be another

• Useful for – Heuristic searches– Graph searches

• Best solution – HeapHeap algorithm of Hunt et Hunt et alal

• Works better than binary search treesbinary search trees

5

Heaps [Hunt et alHunt et al]

6

Scalability Problem of Heaps

7

New Approach - SkipQueues

• Based on Skip ListSkip List of PughPugh

• All locking is distributed and local

• Balancing of structure is probabilistic

• Deletions evenly distributed, minimal contention

• No need to pre-allocate memory

• All operations in expected logarithmic time

8

Coming up Next …

1. The inside story on concurrent Skip ListsSkip Lists [PughPugh]

2. How to make a SkipQueueSkipQueue

3. Experimental Results

9

Skip List Structure [Pugh]

10

Inserting an Element

• InsertInsert (key_t k):– Randomly choose a level l – Find at each level of the list up to l the largest

key smaller than k– For each level from 1 to l

• Acquire a lock on the item found before

• Insert the new key after it

• Release lock

11

Skip List - InsertionNew Item

12

Deleting an Element

• DeleteDelete (key_t k)– Find k at all levels at which it appears.– For each level from l to 1

• Acquire locks on k and the item before it.

• Remove k

• Release locks

13

Skip List - Deletion

14

SkipQueues

• Problem: How to allow Delete_MinDelete_Min operations without creating a bottle neck around the first element?

15

Our Solution

• Observation - The lowest level of the Skip Skip ListList contains all the elements in ascending order

• Processes can advance down this list and “logically” delete the first available key

• Each process can then delete the key that it previously deleted “logically”

16

Delete_Min Operation

• Traverse the bottom level of the Skip ListSkip List structure

• For each traversed Item– Try to SWAP its Deleted flag

– If it was not already deleted, return the key of the item

– Else, go on to next item

• If at the end of the list, return EMPTY• No locking at all

17

From Skip List to SkipQueue

18

Performance Benchmarks

• We compared the performance of 3 structures:

1. The HeapHeap structure of Hunt et alHunt et al

2.2. FunnelListFunnelList – a simple linked list, access to which is governed by a combining-funnelcombining-funnel [Shavit, ZemachShavit, Zemach] structure

3.3. SkipQueueSkipQueue structure as described before

19

Benchmark Methodology

• We used the ProteusProteus multiprocessor simulator by Brewer et alBrewer et al

• We simulated a 256 processors machine similar to the MIT Alewife

• Processors alternate between performing small amount of work and accessing the queue.

20

Benchmark Methodology Cont’d

• Processors randomly choose whether to insert or delete

• Priorities of inserted items are chosen uniformly at random

• We measured the average latency of InsertInsert and Delete_MinDelete_Min operations

21

Small Structure - Deletions

22

Small Structure - Insertions

23

Large Structure - Deletions

24

Large Structure - Insertions

25

Large Structure Benchmark With 70% Deletions - Deletions

26

Large Structure Benchmark With 70% Deletions - Insertions

27

Conclusions

• SkipQueuesSkipQueues scale significantly better than HeapsHeaps

• SkipQueuesSkipQueues are highly distributed – no hot spots or bottle necks

• Deletes are 3 times faster and Inserts are 10 times faster when concurrency reaches 256 processors

• Future Directions– Implementation without locks– Experimenting with other data structures

28

Delete_Min Cont’d

• In order to assure a stronger ordering property we added the following:– Each item is time–stamped when its insertion is

completed– Each Delete_MinDelete_Min operation notes the time at

which it begins– The Delete_MinDelete_Min operation will only try to

“logically” delete items that were inserted before it started.

29

Serialization Order

• Each successful Delete_MinDelete_Min is ordered at the time its “logical” deletion was completed

• Each unsuccessful Delete_MinDelete_Min that returned EMPTYEMPTY is ordered at the time of its return instruction

• Each uncompleted Delete_MinDelete_Min is ordered at the time of its invocation

30

SkipQueue Specification

• For every Delete_MinDelete_Min operation– Let I be the set of keys inserted by Insert

operations serialized before it– Let D be the set of keys removed by

Delete_MinDelete_Min operations serialized before it

• The returned value is the smallest in the set I – D, or EMPTYEMPTY if I – D is empty.