1 heaps & priority queues array and tree implementations
TRANSCRIPT
2
Priority queuePriority queue
A stack is first in, last out
A queue is first in, first out
A priority queue is least-first-out
The “smallest” element is the first one removed
3
The definition of “smallest” is up to the programmer (for example,
you might define it by implementing Comparator or Comparable)
4
Priority queuePriority queue
If there are several “smallest” elements,
the implementer must decide which to remove first
Remove any “smallest” element (don’t care which)
Remove the first one added
5
A priority queue is an:
ADT where items are removed based on their priority
(highest-to-lowest), regardless of the order they ARE inserted in the structure
6
A priority queue ADTA priority queue ADT
PriorityQueue(): Methods
void add(Comparable o): void add(Comparable o): inserts o into the priority queue
Comparable removeLeast(): Comparable removeLeast(): removes and returns the least element
Comparable getLeast(): Comparable getLeast(): returns (but does not remove) the least element
boolean isEmpty():isEmpty(): returns true iff empty
int size(): int size(): returns the number of elements
void clear():void clear(): discards all elements
7
Evaluating implementationsEvaluating implementations
When we choose a data structure,
it is important to look at usage patternsusage patterns
If we load an array once and do thousands of searches on it,
we want to make searching fast—
so we would probably sort the array
8
When we choose a data structure,
it is important to look at usage patterns
If we load a huge array and expect to do only a few searchesfew searches,
we probably don’t want to spend time sorting the array
9
Evaluating implementationsEvaluating implementations
For almost all uses of a queue (including a uses of a queue (including a priority queue),priority queue),
we eventually remove everything that we remove everything that we addadd
10
TIMINGTIMINGHence,
when we analyze a priority queue, analyze a priority queue,
neither “add” nor “remove” is more important—
we need to look at the timing for timing for (“add + remove”)
11
Array implementationsArray implementations
A priority queue could be implemented as an unsorted array (with a count of elements)
Adding an element would take O(1) time (why?)
Removing an element would take O(n) time (why?)
Hence, (adding and removing) an element takes O(n) time
This is an inefficient representation inefficient representation – a BigO(N)
12
Array implementationsArray implementations
A priority queue could be implemented as a sorted array (again, with a counter of elements)
Adding an element would take O(n) time (why?)
Removing an element would take O(1) time (why?)
So adding and removing) an element takes
O(n) time
Again, this is very inefficient
13
Linked list implementationsLinked list implementations
A priority queue could be implemented as an
unsorted linked list
Adding an element would take O(1)O(1) time (why?)
Removing an element would take (N) time (why?)
Again, inefficient!Again, inefficient!
14
SORTED LINKED LISTSORTED LINKED LIST
A priority queue could be implemented as a
sorted linked list
Adding an element Adding an element would take O(n) time (why?)
Removing an element Removing an element would take O(1) time (why?)
Again, an inefficient algorithminefficient algorithm
15
Binary tree implementationsBinary tree implementations
A priority queue could be represented as a A priority queue could be represented as a
balancedbalanced binary search tree binary search tree
Insertion and removal could destroy the balanceInsertion and removal could destroy the balance
We would need an algorithm to rebalance the binary tree
Good rebalancing algorithms Good rebalancing algorithms require O(log n)O(log n) time time, but are
complicatedcomplicated
16
Heap ImplementatonHeap Implementaton
The concepts of heaps is used to refer to memory in a computer,
E.g. the part of the memory that provides dynamic memory allocation ( get it as you need it)
“Heap” is also used as a name for a special kind of binary tree
17
Heap is a complete binary tree Heap is a complete binary tree
A heap is a complete binary tree
with values stored in its nodes such that
no child has a value bigger than the value of its parent –
(though it can be equal)
18
Therefore we would like to find a better algorithm. We want to:
Insert and Remove with a Big O of oneInsert and Remove with a Big O of one
19
HEAPS & Priority QueuesHEAPS & Priority Queues
A binary tree represented as a heap facilitates the operation
to find the maximum element:
“it is always in the root of the tree “
Because finding the node with the maximum value in a heap is a BigO(I)
A HEAP is the most efficient structure for implementing priority queues
20
HeapsHeapsHeapsHeaps
A A heapheap is a certain is a certain kind of complete kind of complete binary tree.binary tree.
When a completebinary tree is built,
its first node must bethe root.
When a completebinary tree is built,
its first node must bethe root.
Root
21
HeapsHeapsHeapsHeaps
Complete binary tree.
Right childRight child
The Second and thirdNodes are always the left and right childleft and right childof the root.of the root.
The Second and thirdNodes are always the left and right childleft and right childof the root.of the root.
Left Left childchild
22
HeapsHeapsHeapsHeaps
Complete binary tree.
The next nodesalways fill the next
level from left-to-right.left-to-right.
The next nodesalways fill the next
level from left-to-right.left-to-right.
24
HeapsHeapsHeapsHeaps
A heap is a A heap is a certaincertain kind of kind of complete binary complete binary tree.tree.
Each node in a heapEach node in a heapcontains a key thatcontains a key thatcan be compared tocan be compared toother nodes' keys.other nodes' keys.
Each node in a heapEach node in a heapcontains a key thatcontains a key thatcan be compared tocan be compared toother nodes' keys.other nodes' keys.
19
4222127
23
45
35
25
To implement a heap as a priority queueTo implement a heap as a priority queue
A node has the A node has the heap property heap property if it is if it is Equal to or greater Equal to or greater than than as its children as its children
ORORif it is smaller than or equal to its children ( (since smaller since smaller numbers represent higher prioritiesnumbers represent higher priorities))
12
8 3
Heap: Yellow node has the heap property
3
8 12
Priority queue: Yellow node has the Priority queue: Yellow node has the heap property - less than its childrenheap property - less than its children
26
HeapsHeaps
A heap is a certain kind of complete binary tree.
This example is a max heap.
The "heap property"requires that each
node's key is >= to thekeys of its children
The "heap property"requires that each
node's key is >= to thekeys of its children
19
4222127
23
45
35
27
Priority QueuesPriority Queues
A heap can easily implement a priority queue
but what happens with we remove the element remove the element of highest priority? of highest priority?
How do we re-arrange the tree?
28
Removing from a heap Removing from a heap
There are two things we need to consider when rearranging:
We want to end up with a heap which means that
The tree has to be complete –
ALL HEAPS ARE COMPLETE TREES!ALL HEAPS ARE COMPLETE TREES!
29
To remove an element at top of treeTo remove an element at top of tree::
Remove the element at location 0
Move the element in the lastIndex to location to location 00, , decrement decrement lastIndex
(lastIndex is at the end of the heap)(lastIndex is at the end of the heap)
Reheap the new root node Reheap the new root node (the one now at location 0)
This is called down-heap bubbling or down-heap bubbling or percolating down percolating down
Down-heap bubblingDown-heap bubbling requires O(log n) time
to remove an element
30
Removing the Top of a HeapRemoving the Top of a Heap(max heap)(max heap)
Removing the Top of a HeapRemoving the Top of a Heap(max heap)(max heap)
Move the last node onto the root.
19
4222135
23
45
42
27
31
Removing the Top of a HeapRemoving the Top of a Heap(max heap)(max heap)
Removing the Top of a HeapRemoving the Top of a Heap(max heap)(max heap)
Move the value in the last node into the root.
19
4222135
23
27
42
32
Removing the Top of a HeapRemoving the Top of a HeapRemoving the Top of a HeapRemoving the Top of a Heap
Move the last node onto the root.
Push the out-of-Push the out-of-place node place node downwarddownward, ,
swapping with its swapping with its larger child larger child
until the new node until the new node reaches an reaches an acceptable location.acceptable location.
19
4222135
23
27
42
33
Removing the Top of a HeapRemoving the Top of a HeapRemoving the Top of a HeapRemoving the Top of a Heap
Move the last node onto the
root.
Push the out-of-Push the out-of-place node place node downward, downward, swapping with its larger child until the new node reaches an acceptable location.acceptable location.
19
4222135
23
42
27
34
Removing the Top of a HeapRemoving the Top of a HeapRemoving the Top of a HeapRemoving the Top of a Heap
Move the last node onto the root.
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location.
Note that we discard half of the tree with each move downward
19
4222127
23
42
35
35
reheapificationreheapification downwarddownward..reheapificationreheapification downwarddownward..
We stop when:
The children all have keys <= to the out-of-keys <= to the out-of-place node, orplace node, or
The node reaches the The node reaches the leaf.leaf.
The process of pushing the node downward is called: reheapification downward.
19
4222127
23
42
35
36
Priority QueuesPriority Queues
Another Example :
How to delete a node from a Priority Queue
Since it is queue - first in first out,
we remove the first one
The Root
37
An Example of a MAX HeapAn Example of a MAX Heap
The root has largest valueThe root has largest value
23
1612
7 9 13 5
21 3
38
Deleting a node from the heapDeleting a node from the heapWe use a heap a heap to implement a priority queue priority queue
the "next" element ( the root next" element ( the root ) in the queue was removed (pop operation).
We end up with a root with no value
1612
7 9 13 5
21 3
Root has no valueRoot has no value
39
Deleting a node from the heapDeleting a node from the heap
1612
7 9 13 5
21 3
The obvious questionis: which node can we
use to replace this one?
The obvious questionis: which node can we
use to replace this one?
40
Deleting a node from the heapDeleting a node from the heap
1612
7 9 13 5
21 3
If we want this tree to stay complete
the rightmost element in thebottommost level is the
obvious choice
If we want this tree to stay complete
the rightmost element in thebottommost level is the
obvious choice
41
Deleting a node from the heapDeleting a node from the heap
2
1612
7 9 13 5
1 3
We now have aproblem: We have a complete tree butthis is not a heap!
WHY???
This can be solved by applying systematic swap of nodes.
We now have aproblem: We have a complete tree butthis is not a heap!
WHY???
This can be solved by applying systematic swap of nodes.
42
Deleting a node from the heapDeleting a node from the heap
2
1612
7 9 13 5
1 3We systematically swapthe root with the largerof the children nodes
until no more swaps canbe done
We systematically swapthe root with the largerof the children nodes
until no more swaps canbe done
45
Deleting a node from the heapDeleting a node from the heap
16
1312
7 9 2 5
1 3
The tree has restored itsheap property!
The tree has restored itsheap property!
46
Insert operationInsert operation
Another example - the heap below
Recall that while delete causes pairwise swaps from the root to the Recall that while delete causes pairwise swaps from the root to the bottom, bottom,
insert causes pairwise swaps from the bottom to the top. insert causes pairwise swaps from the bottom to the top.
Suppose that we want to insert an element with value 20insert an element with value 2022
1712
11 3 13 5
16 4
51
Implementing a HeapImplementing a HeapImplementing a HeapImplementing a Heap
We will store the data from the nodes in a partially-filled array.
An array of dataAn array of data
2127
23
42
35
52
Implementing a HeapImplementing a HeapImplementing a HeapImplementing a Heap
Data from the root goes in the first location of the array.
An array of dataAn array of data
2127
23
42
35
42
53
Implementing a HeapImplementing a HeapImplementing a HeapImplementing a Heap
Data from the next row goes in the next two array locations.
An array of dataAn array of data
2127
23
42
35
42 35 23
54
Implementing a HeapImplementing a HeapImplementing a HeapImplementing a Heap
Data from the next row goes in the next two array locations.
An array of dataAn array of data
2127
23
42
35
42 35 23 27 21
55
Implementing a HeapImplementing a HeapImplementing a HeapImplementing a Heap
Data from the next row goes in the next two array locations.
An array of dataAn array of data
2127
23
42
35
42 35 23 27 21
56
Points about the ImplementationPoints about the ImplementationPoints about the ImplementationPoints about the Implementation
The links between the tree's nodes are not actually stored as pointers, or in any other way.
The only way we "know" that "the array is a tree" is from the way we manipulate the data.manipulate the data.
An array of dataAn array of data
2127
23
42
35
42 35 23 27 21
57
Array representation of a heapArray representation of a heap
Left child of node i is 2*i + 1, right child is 2*i + 2
Unless the value is larger than lastIndex lastIndex -therefore no such child
Parent of node i is (i – 1)/2 unless i == 0
12
1418
6
8
3
3 12 6 18 14 8
0 1 2 3 4 5 6 7 8 9 10 11 12
lastIndex = location 5
58
algorithm for the array implementationalgorithm for the array implementation
Increase lastIndex and put the new value where lastIndex now new value where lastIndex now references -- references --
that is add to the end of the arraythat is add to the end of the array
ReheapReheap the newly added node
This is called up-heap bubbling or percolating up-heap bubbling or percolating up
Up-heap bubbling requires O(log n) time
Adding to heap Adding to heap - - - Implementation- Implementation
59
Points about the Points about the ImplementationImplementation
Points about the Points about the ImplementationImplementation
If you know the index of a node, then it is easy to figure out the indexes of that node's parent and children.
[0][0] [1] [2] [3] [4]
2127
23
42
35
42 35 23 27 21
60
Points about the ImplementationPoints about the ImplementationPoints about the ImplementationPoints about the Implementation
If we add 45 to the tree and to the array, 45’s parent is at what index? i = ? (i-1)/2 = 2
[0][0] [1] [2] [3] [4] [5]
2127
23
42
35
42 35 23 27 21
45
45
61
Points about the ImplementationPoints about the ImplementationPoints about the ImplementationPoints about the Implementation
If we add 45 to the tree and the array, 45’s parent is at what index?
(i-1)/2 which is 2.
We do a We do a comparison of 45 with its comparison of 45 with its parent and parent and
swap if it is greater than the swap if it is greater than the parent - it is greater than itparent - it is greater than it
[0][0] [1] [2] [3] [4] [5]
2127
23
42
35
42 35 23 27 21
45
45
62
Points about the ImplementationPoints about the ImplementationPoints about the ImplementationPoints about the Implementation
If we add 45 to the tree and to the tree and the array,
45’s parent is at what index? (i-1)/2 which is 2.
We do a comparison and swap.
It is still out of place.It is still out of place.
[0][0] [1] [2] [3] [4] [5]
2127
45
42
35
42 35 45 27 21
23
23
63
Points about the ImplementationPoints about the ImplementationPoints about the ImplementationPoints about the Implementation
45 is still out of place,
so we do another comparison
and swap with 45’s parent which is 42. So what is the terminating condition)(s)?
[0][0] [1] [2] [3] [4] [5]
2127
42
45
35
45 35 42 27 21
23
23
64
A heap is a complete binary tree, where
the entry at each node is greater than or equal to the entries in its children.
OR
the entry at each node is less than or equal to the entries in its children.
Summary Summary
65
Summary Adding/Deleting
To add an entry to a heap, To add an entry to a heap,
place the new entry at the next available spot, place the new entry at the next available spot,
and perform a reheapification upward.and perform a reheapification upward.
To remove the biggest entryTo remove the biggest entry, ,
move the value in the last node onto the root, and move the value in the last node onto the root, and
perform a reheapification downward.perform a reheapification downward.
66
Reheapifying a TreeReheapifying a Tree
Fine,
we know how to delete one element and we know how to delete one element and
restore the heap,restore the heap,
but there are several other operations but there are several other operations involving heapsinvolving heaps
67
If we find a systematic way of heapifying a tree,
we could perform any operation
because we can execute the procedure heapify
68
Reheapifying a TreeReheapifying a Tree
we can heapify a tree using a quite simple heapify a tree using a quite simple idea.idea.
Given a binary tree T (not necessarily a heap) with root R Given a binary tree T (not necessarily a heap) with root R and left subtree Tand left subtree T
leftleft and T and Trightright,,
if these two subtrees are heaps, if these two subtrees are heaps,
we can make T a heap using the process described in the we can make T a heap using the process described in the previous slide (of applying systematic swaps)previous slide (of applying systematic swaps)
We start the process of heapifying a tree We start the process of heapifying a tree from the lower levelsfrom the lower levels
69
Algorithm for Reheapifying TreeAlgorithm for Reheapifying Tree
The algorithm aims at heapifying a tree by The algorithm aims at heapifying a tree by always comparing always comparing three nodes at a time.three nodes at a time.
To To start from the lower levels start from the lower levels
we choose to look at the internal nodes in we choose to look at the internal nodes in
reverse level orderreverse level order
External nodExternal nodes have no children so they es have no children so they are (individually) are (individually) heapsheaps
70
HEAPIFYHEAPIFY
void heapify (Heap h) {void heapify (Heap h) {
NodeType n;NodeType n;
for (n = for (n = the internal nodes of h in reverse level-the internal nodes of h in reverse level-orderorder) {) {
Reheapify the heap h Reheapify the heap h starting at node nstarting at node n
}}
}}
void heapify (Heap h) {void heapify (Heap h) {
NodeType n;NodeType n;
for (n = for (n = the internal nodes of h in reverse level-the internal nodes of h in reverse level-orderorder) {) {
Reheapify the heap h Reheapify the heap h starting at node nstarting at node n
}}
}}
72
An exampleAn example6
174
12 1 13 5
311 22
Internal Nodes in Reverse Level-Order are: 1, 12, 17, 4, 6 ,
start with the leftmost leaf : compare 3 To 1
73
6
174
12 3 13 5
111 22
3 and 1 have swapped3 and 1 have swappedInternal Nodes in Reverse Level-Order: 3, 12, 17, 4, 6
74
An exampleAn example
6
174
12 3 13 5
111 22
Compare 12 with its children and swap 12 and 22 :Compare 12 with its children and swap 12 and 22 :
Internal Nodes in Reverse Level-Order: 3, 22, 17, 4, 6
75
An exampleAn example
6
174
22 3 13 5
111 12
Compare 17 with its children and no changes necessary: 3, 22, 17, 4, 6 --- NEXT COMPARE 4 AND 22
76
6
1722
4 3 13 5
111 12
Compare Compare 22 with 4 22 with 4 and swap : and swap : 4 IS NOW OUT OF PLACE4 IS NOW OUT OF PLACE
77
An exampleAn example
6
1722
12 3 13 5
111 4
Compare 12 and 4 and swap, NEXT compare 22 to 6 :
3, 12, 17, 22, 6
82
heapsortheapsort
Heapsort is another algorithm for sorting lists
The idea behind heap sort is that
to sort a list
we can insert the elements of a list into a heap using heapify
Then remove all the elements using remove.
83
The idea behind heap sort is that
to sort a list we can insert the elements of a list into a heap using heapify
Then remove all the elements using remove.
It is guaranteed that the largest element is removed first, largest element is removed first,
So by systematically removing the elements
we get the list ordered ( in reverse order)we get the list ordered ( in reverse order)