analysis of algorithms cs 477/677 instructor: monica nicolescu lecture 9
TRANSCRIPT
![Page 1: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/1.jpg)
Analysis of AlgorithmsCS 477/677
Instructor: Monica Nicolescu
Lecture 9
![Page 2: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/2.jpg)
CS 477/677 - Lecture 9
A Job Scheduling Application
• Job scheduling
– The key is the priority of the jobs in the queue
– The job with the highest priority needs to be executed
next
• Operations
– Insert, remove maximum
• Data structures
– Priority queues
– Ordered array/list, unordered array/list
![Page 3: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/3.jpg)
CS 477/677 - Lecture 9
PQ Implementations & Cost
Worst-case asymptotic costs for a PQ with N items
Insert Remove max
ordered array
ordered list
unordered array
unordered list
N
N
N
N
1
1
1
1
Can we implement both operations efficiently?
![Page 4: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/4.jpg)
CS 477/677 - Lecture 9
Background on Trees
• Def: Binary tree = structure composed of a finite set of nodes that either:– Contains no nodes, or– Is composed of three disjoint sets of nodes: a root
node, a left subtree and a right subtree
2
14 8
1
16
4
3
9 10
root
Right subtreeLeft subtree
![Page 5: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/5.jpg)
CS 477/677 - Lecture 9
Special Types of Trees
• Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2.
• Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2.
Full binary tree
2
14 8
1
16
7
4
3
9 10
12
Complete binary tree
2
1
16
4
3
9 10
![Page 6: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/6.jpg)
CS 477/677 - Lecture 9
The Heap Data Structure
• Def: A heap is a nearly complete binary tree with the following two properties:– Structural property: all levels are full, except
possibly the last one, which is filled from left to right– Order (heap) property: for any node x
Parent(x) ≥ x
Heap
5
7
8
4It doesn’t matter that 4 in
level 1 is smaller than 5 in
level 22
![Page 7: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/7.jpg)
CS 477/677 - Lecture 9
Definitions
• Height of a node = the number of edges on a longest simple path from the node down to a leaf
• Depth of a node = the length of a path from the root to the node
• Height of tree = height of root node
= lgn, for a heap of n elements
2
14 8
1
16
4
3
9 10
Height of root = 3
Height of (2)= 1 Depth of (10)= 2
![Page 8: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/8.jpg)
CS 477/677 - Lecture 9
Array Representation of Heaps
• A heap can be stored as an
array A.– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves
• The root is the maximum
element of the heap
![Page 9: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/9.jpg)
CS 477/677 - Lecture 9
Heap Types
• Max-heaps (largest element at root), have the
max-heap property: – for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
• Min-heaps (smallest element at root), have the
min-heap property:– for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]
![Page 10: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/10.jpg)
CS 477/677 - Lecture 9
Operations on Heaps
• Maintain the max-heap property– MAX-HEAPIFY
• Create a max-heap from an unordered array– BUILD-MAX-HEAP
• Sort an array in place– HEAPSORT
• Priority queue operations
![Page 11: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/11.jpg)
CS 477/677 - Lecture 9
Operations on Priority Queues
• Max-priority queues support the following
operations:
– INSERT(S, x): inserts element x into set S
– EXTRACT-MAX(S): removes and returns element of
S with largest key
– MAXIMUM(S): returns element of S with largest key
– INCREASE-KEY(S, x, k): increases value of element
x’s key to k (assume k ≥ current key value at x)
![Page 12: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/12.jpg)
CS 477/677 - Lecture 9
Building a Heap
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i, n)
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are
leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
4 1 3 2 16 9 10 14 8 7A:
![Page 13: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/13.jpg)
CS 477/677 - Lecture 9
Example: A 4 1 3 2 16 9 10 14 8 7
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
1
16
7
4
10
9 3
1
2 3
4 5 6 7
8 9 10
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 1014
2 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
14
2 8
16
7
1
4
10
9 3
1
2 3
4 5 6 7
8 9 108
2 4
14
7
1
16
10
9 3
1
2 3
4 5 6 7
8 9 10
i = 5 i = 4 i = 3
i = 2 i = 1
![Page 14: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/14.jpg)
CS 477/677 - Lecture 9
Correctness of BUILD-MAX-HEAP• Loop invariant:
– At the start of each iteration of the for loop, each node i + 1, i + 2,…, n is the root of a max-heap
• Initialization:– i = n/2: Nodes n/2+1, n/2+2, …, n are leaves
they are the root of trivial max-heaps
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
![Page 15: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/15.jpg)
CS 477/677 - Lecture 9
Correctness of BUILD-MAX-HEAP• Maintenance:
– MAX-HEAPIFY makes node i a max-heap root and preserves the property that nodes i + 1, i + 2, …, n are roots of max-heaps
– Decrementing i in the for loop reestablishes the loop invariant
• Termination:– i = 0 each node 1, 2, …, n is the root of
a max-heap (by the loop invariant)
2
14 8
1
16
7
4
3
9 10
1
2 3
4 5 6 7
8 9 10
![Page 16: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/16.jpg)
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
It would seem that running time is O(nlgn)
• This is not an asymptotically tight upper bound
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
3. do MAX-HEAPIFY(A, i, n) O(lgn)O(n)
![Page 17: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/17.jpg)
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
• HEAPIFY takes O(h) the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree
Height Level
h0 = 3 (lgn)
h1 = 2
h2 = 1
h3 = 0
i = 0
i = 1
i = 2
i = 3 (lgn)
No. of nodes
20
21
22
23
hi = h – i height of the heap rooted at level ini = 2i number of nodes at level i
i
h
iihnnT
0
)( ihh
i
i 0
2 )(nO
![Page 18: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/18.jpg)
CS 477/677 - Lecture 9
Running Time of BUILD MAX HEAP
i
h
iihnnT
0
)( Cost of HEAPIFY at level i number of nodes at that level
ihh
i
i 0
2 Replace the values of ni and hi computed before
hh
iih
ih2
20
Multiply by 2h both at the nominator and denominator and
write 2i as i2
1
h
kk
h k
0 22 Change variables: k = h - i
0 2k
k
kn The sum above is smaller than the sum of all elements to
and h = lgn
)(nO The sum above is smaller than 2
Running time of BUILD-MAX-HEAP: T(n) = O(n)
![Page 19: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/19.jpg)
CS 477/677 - Lecture 9
Binary Search Trees• Tree representation:
– A linked data structure in which each node is an object
• Node representation:– Key field– Satellite data– Left: pointer to left child– Right: pointer to right child– p: pointer to parent (p [root [T]] =
NIL)
• Satisfies the binary search tree property
Left child Right child
L Rparent
key data
![Page 20: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/20.jpg)
CS 477/677 - Lecture 9
Binary Search Tree Example
• Binary search tree property:
– If y is in left subtree of x,
then key [y] ≤ key [x]
– If y is in right subtree of x,
then key [y] ≥ key [x]
2
3
5
5
7
9
![Page 21: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/21.jpg)
CS 477/677 - Lecture 9
Binary Search Trees
• Support many dynamic set operations
– SEARCH, MINIMUM, MAXIMUM, PREDECESSOR,
SUCCESSOR, INSERT, DELETE
• Running time of basic operations on binary
search trees
– On average: (lgn)• The expected height of the tree is lgn
– In the worst case: (n)• The tree is a linear chain of n nodes
![Page 22: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/22.jpg)
CS 477/677 - Lecture 9
Red-Black Trees
• “Balanced” binary trees guarantee an O(lgn) running time on the basic dynamic-set operations
• Red-black tree– Binary tree with an additional attribute for its nodes:
color which can be red or black– Constrains the way nodes can be colored on any path
from the root to a leaf• Ensures that no path is more than twice as long as another
the tree is balanced
– The nodes inherit all the other attributes from the binary-search trees: key, left, right, p
![Page 23: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/23.jpg)
CS 477/677 - Lecture 9
Red-Black Trees Properties
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black• No two red nodes in a row on a simple path from the
root to a leaf
5. For each node, all paths from the node to
descendant leaves contain the same number of
black nodes
![Page 24: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/24.jpg)
CS 477/677 - Lecture 9
Example: RED-BLACK TREE
• For convenience we use a sentinel NIL[T] to represent all the NIL nodes at the leafs– NIL[T] has the same fields as an ordinary node– Color[NIL[T]] = BLACK– The other fields may be set to arbitrary values
26
17 41
30 47
38 50
NIL NIL
NIL
NIL NIL NIL NIL
NIL
![Page 25: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/25.jpg)
CS 477/677 - Lecture 9
Black-Height of a Node
• Height of a node: the number of edges in a longest path to a leaf
• Black-height of a node x: bh(x) is the number of black nodes (including NIL) on a path from x to leaf, not counting x
26
17 41
30 47
38 50
NIL NIL
NIL
NIL NIL NIL NIL
NIL
h = 4bh = 2
h = 3bh = 2
h = 2bh = 1
h = 1bh = 1
h = 1bh = 1
h = 2bh = 1 h = 1
bh = 1
![Page 26: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/26.jpg)
CS 477/677 - Lecture 9
Properties of Red-Black Trees• Claim
– Any node with height h has black-height ≥ h/2• Proof
– By property 4, there are at most h/2 red nodes on the path from the node to a leaf
– Hence at least h/2 are black26
17 41
30 47
38 50
Property 4: if a node is red then both its children are black
![Page 27: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/27.jpg)
CS 477/677 - Lecture 9
Properties of Red-Black Trees
Claim
The subtree rooted at any node x contains at least
2bh(x) - 1 internal nodes
Proof: By induction on height of x
Basis: height[x] = 0
x is a leaf (NIL[T])
bh(x) = 0
# of internal nodes: 20 - 1 = 0
NIL
x
![Page 28: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/28.jpg)
CS 477/677 - Lecture 9
Properties of Red-Black TreesInductive step: • Let height(x) = h and bh(x) = b• Any child y of x has:
– bh (y) =– bh (y) =
26
17 41
30 47
38 50
b (if the child is red), or
b - 1 (if the child is black)
![Page 29: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/29.jpg)
CS 477/677 - Lecture 9
Properties of Red-Black Trees• Want to prove:
– The subtree rooted at any node x contains at
least 2bh(x) - 1 internal nodes
• Assume true for children of x: – Their subtrees contain at least 2bh(x) - 1 – 1
internal nodes
• The subtree rooted at x contains at least:
(2bh(x) - 1 – 1) + (2bh(x) - 1 – 1) + 1 =
2 · (2bh(x) - 1 - 1) + 1 =
2bh(x) - 1 internal nodes
x
l r
![Page 30: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/30.jpg)
CS 477/677 - Lecture 9
Properties of Red-Black Trees
Lemma: A red-black tree with n internal nodes has height at most 2lg(n + 1).
Proof:
n
• Add 1 to all sides and then take logs:
n + 1 ≥ 2b ≥ 2h/2
lg(n + 1) ≥ h/2 h ≤ 2 lg(n + 1)
root
l r
height(root) = hbh(root) = b
number n of internal nodes
≥ 2b - 1 ≥ 2h/2 - 1
since b h/2
![Page 31: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/31.jpg)
CS 477/677 - Lecture 9
Operations on Red-Black Trees
• The non-modifying binary-search tree operations
MINIMUM, MAXIMUM, SUCCESSOR,
PREDECESSOR, and SEARCH run in O(h) time
– They take O(lgn) time on red-black trees
• What about TREE-INSERT and TREE-DELETE?
– They will still run in O(lgn)
– We have to guarantee that the modified tree will still be
a red-black tree
![Page 32: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/32.jpg)
CS 477/677 - Lecture 9
INSERT
INSERT: what color to make the new node?• Red? Let’s insert 35!
– Property 4: if a node is red, then both its children are black
• Black? Let’s insert 14!– Property 5: all paths from a node to its leaves contain
the same number of black nodes
26
17 41
30 47
38 50
![Page 33: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/33.jpg)
CS 477/677 - Lecture 9
DELETE
DELETE: what color was the
node that was removed? Red?
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
5. For each node, all paths from the node to descendant
leaves contain the same number of black nodes
OK!
OK!
OK!
OK! Does not createtwo red nodes in a row
OK! Does not changeany black heights
26
17 41
30 47
38 50
![Page 34: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/34.jpg)
CS 477/677 - Lecture 9
DELETE
DELETE: what color was the
node that was removed? Black?
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
5. For each node, all paths from the node to descendant
leaves contain the same number of black nodes
OK!
OK!
Not OK! Could createtwo red nodes in a row
Not OK! Could change theblack heights of some nodes
26
17 41
30 47
38 50
Not OK! If removing the root and the child that replaces it is red
![Page 35: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/35.jpg)
CS 477/677 - Lecture 9
Rotations
• Operations for restructuring the tree after insert
and delete operations on red-black trees
• Rotations take a red-black tree and a node within
the tree and:– Together with some node re-coloring they help restore
the red-black tree property
– Change some of the pointer structure
– Do not change the binary-search tree property
• Two types of rotations:– Left & right rotations
![Page 36: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/36.jpg)
CS 477/677 - Lecture 9
Left Rotations• Assumption for a left rotation on a node x:
– The right child of x (y) is not NIL
• Idea:– Pivots around the link from x to y– Makes y the new root of the subtree– x becomes y’s left child– y’s left child becomes x’s right child
![Page 37: Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f205503460f94c39037/html5/thumbnails/37.jpg)
CS 477/677 - Lecture 9
Readings
• Chapter 8, Chapter 6