9 trees iv
TRANSCRIPT
TREES
Table of Contents:
•Heapsort
• B Tree
• Huffman’s Algorithm
Heap
• Suppose H is a complete binary tree with n elements. Then H is called a heap or a maxheap if each node N of H has the property that value of N is greater than or equal to value at each of the children of N.
• Analogously, a minheap is a heap such that value at N is less than or equal to the value of each of its children
97
88 95
66 55 87 48
25 3877623524
26
48
40 39
30
18 17
Example of Max Heap
Inserting an Element in a Heap Suppose H is a heap with N elements, and suppose an ITEM of
information is given. We insert ITEM into the heap H as follows:
• First adjoin the ITEM at the end of H so that H is still a complete tree but not necessarily a heap.
• Then let the ITEM rise to its appropriate place in H so that H is finally a heap.
[Heap is more efficiently implemented through array rather than linked list. In a heap, the location of parent of a node PTR is given by PTR/2 ]
Build a Maxheap
Following are the elements:
44,30,50, 22,60,55,77,55
Algorithm: INSHEAP( TREE, N, ITEM)
A heap H with N elements is stored in the array TREE and an ITEM of information is given. This procedure inserts the ITEM as the new element of H. PTR gives the location of ITEM as it rises in the tree and PAR denotes the parent of ITEM
1. [Add new node to H and Initialize PTR]
Set N:= N +1 and PTR:=N
2. [Find Location to Insert ITEM
Repeat steps 3 to 6 while PTR > 1
3. Set PAR:= └PTR/2 ┘ [Location of Parent node]
4. If ITEM ≤ TREE[PAR], then:
Set TREE[PTR]:=ITEM and Return
[End of If Structure]
5. Set TREE[PTR]:=TREE[PAR] [Moves node down]
6. Set PTR:=PAR [updates PTR]
[End of step 2 Loop]
7. Set TREE[1]:=ITEM
8. Return
Deleting the Root node in a heap
Suppose H is a heap with N elements and suppose we want to delete the root R of H. This is accomplished as follows:
• Assign the root R to some variable ITEM
• Replace the deleted node R by last node L of H so that H is still a complete tree but not necessarily a heap.
• Let L sink to its appropriate place in H so that H is finally a heap.
95
85 70
55 33 30 65
15 20 15 22
DELETE 95
Algorithm: DELHEAP( TREE, N , ITEM )
A heap H with N elements is stored in the array TREE. This algorithm assigns the root TREE[1] of H to the variable ITEM and then reheaps the remaining elements. The variable LAST stores the value of the original last node of H. The pointers PTR, LEFT and RIGHT give the Location of LAST and its left and right children as LAST sinks into the tree.
1: Set ITEM:=TREE[1] [removes root of H]
2: Set LAST:=TREE[N] and N:=N-1 [removes last node of H]
3: Set PTR:=1, LEFT:=2 and RIGHT:=3
4: Repeat step 5 to 7 while RIGHT ≤ N:
5: If LAST ≥ TREE[LEFT] and LAST ≥ TREE [RIGHT] , then:
Set TREE[PTR]:=LAST and Return
6: If TREE[RIGHT]≤ TREE[LEFT], then:
Set TREE[PTR]:=TREE[LEFT]
Set PTR:=LEFT
Else:
Set TREE[PTR]:=TREE[RIGHT] and PTR:=RIGHT
[End of If structure]
Set LEFT:= 2* PTR and RIGHT:=LEFT + 1
[End of Loop]
7: If LEFT=N and If LAST < TREE[LEFT], then:
Set TREE[PTR]:=TREE[LEFT] and Set PTR:=LEFT
8: Set TREE[PTR]:=LAST
9: Return
Application of Heap
HeapSort- One of the important applications of heap is sorting of an array using heapsort method. Suppose an array A with N elements is to be sorted. The heapsort algorithm sorts the array in two phases:
• Phase A: Build a heap H out of the elements of A
• Phase B: Repeatedly delete the root element of H
Since the root element of heap contains the largest element of the heap, phase B deletes the elements in decreasing order. Similarly, using heapsort in minheap sorts the elements in increasing order as then the root represents the smallest element of the heap.
Algorithm: HEAPSORT(A,N)
An array A with N elements is given. This algorithm sorts the elements of the array
• Step 1: [Build a heap H, call the procedure ]
Repeat for J=1 to N-1:
Call INSHEAP(A, J, A[J+1])
[End of Loop]• Step 2: [Sort A repeatedly deleting the root of H]
Repeat while N > 1:
(a) Call DELHEAP( A, N, ITEM)
(b) Set A[N + 1] := ITEM [Store the elements deleted from
the heap]
[End of loop]• Step 3: Exit
Complexity of HeapSort
• Phase1 (Build a heap H out of the ‘n’ elements of A):
g(n) ≤ nlog2n
• Phase 2 (Repeatedly delete the root element of H):
h(n) ≤ nlog2n
Therefore, f(n) = O(nlog2n) (In worst case)
• Better than Bubblesort (O(n2 )) and Quicksort (Avg- O(nlog2n), Worst O(n2 ))
• Problem: Create a Heap out of the following data:
jan feb mar apr may jun jul aug sept oct nov dec
B-Trees
Motivation for B-Trees
• Index structures for large datasets cannot be stored in main memory
• Storing it on disk requires different approach to efficiency
• Assuming that a disk spins at 3600 RPM, one revolution occurs in 1/60 of a second, or 16.7ms
• Crudely speaking, one disk access takes about the same time as 200,000 instructions
Motivation (cont.)
• Assume that we use an AVL tree to store about 20 million records
• We end up with a very deep binary tree with lots of different disk accesses; log2 20,000,000 is about 24, so this takes about 0.2 seconds
• We know we can’t improve on the log n lower bound on search for a binary tree
• But, the solution is to use more branches and thus reduce the height of the tree!
– As branching increases, depth decreases
Definition of a B-tree• A B-tree of order m is an m-way tree (i.e., a tree
where each node may have up to m children) in which:
1.the number of keys in each non-leaf node is one less than the number of its children and these keys partition the keys in the children in the fashion of a search tree
2.all leaves are on the same level
3.all non-leaf nodes except the root have at least m / 2 children
4.the root is either a leaf node, or it has from two to m children
5.a leaf node contains no more than m – 1 keys
• The number m should always be odd
An example B-Tree
6251426 12
26
55 60 7064 9045
1 2 4 7 8 1513 18 25
27 29 46 48 53
A B-tree of order 5 containing 26 items
Note that all the leaves are at the same levelNote that all the leaves are at the same level
• Suppose we start with an empty B-tree and keys arrive in the following order:1 12 8 2 25 5 14 28 17 7 52 16 48 68 3 26 29 53 55 45
• We want to construct a B-tree of order 5
• The first four items go into the root:
• To put the fifth item in the root would violate condition 5
• Therefore, when 25 arrives, pick the middle key to make a new root
Constructing a B-tree
1 2 8 12
Constructing a B-tree (contd.)
1 2
8
12 25
6, 14, 28 get added to the leaf nodes:
1 2
8
12 146 25 28
Constructing a B-tree (contd.)
Adding 17 to the right leaf node would over-fill it, so we take the middle key, promote it (to the root) and split the leaf
8 17
12 14 25 281 2 6
7, 52, 16, 48 get added to the leaf nodes
8 17
12 14 25 281 2 6 16 48 527
Constructing a B-tree (contd.)Adding 68 causes us to split the right most leaf, promoting 48 to the root, and adding 3 causes us to split the left most leaf, promoting 3 to the root; 26, 29, 53, 55 then go into the leaves
3 8 17 48
52 53 55 6825 26 28 291 2 6 7 12 14 16
Adding 45 causes a split of 25 26 28 29
and promoting 28 to the root then causes the root to split
Constructing a B-tree (contd.)
17
3 8 28 48
1 2 6 7 12 14 16 52 53 55 6825 26 29 45
Inserting into a B-Tree
• Attempt to insert the new key into a leaf
• If this would result in that leaf becoming too big, split the leaf into two, promoting the middle key to the leaf’s parent
• If this would result in the parent becoming too big, split the parent into two, promoting the middle key
• This strategy might have to be repeated all the way to the top
• If necessary, the root is split in two and the middle key is promoted to a new root, making the tree one level higher
Exercise in Inserting a B-Tree
• Insert the following keys to a 5-way B-tree:
3, 7, 9, 23, 45, 1, 5, 14, 25, 24, 13, 11, 8, 19, 4, 31, 35, 56
Removal from a B-tree
• During insertion, the key always goes into a leaf. For deletion we wish to remove from a leaf. There are three possible ways we can do this:
CASE: 1 - If the key is already in a leaf node, and removing it doesn’t cause that leaf node to have too few keys, then simply remove the key to be deleted.
CASE: 2 - If the key is not in a leaf then it is guaranteed (by the nature of a B-tree) that its predecessor or successor will be in a leaf -- in this case we can delete the key and promote the predecessor or successor key to the non-leaf deleted key’s position.
Removal from a B-tree (2)• If (1) or (2) lead to a leaf node containing less than
the minimum number of keys then we have to look at the siblings immediately adjacent to the leaf in question:
CASE: 3- If one of them has more than the min. number of keys then we can promote one of its keys to the parent and take the parent key into our lacking leaf
CASE:4 - If neither of them has more than the min. number of keys then the lacking leaf and one of its neighbours can be combined with their shared parent (the opposite of promoting a key) and the new leaf will have the correct number of keys; if this step leave the parent with too few keys then we repeat the process up to the root itself, if required
Type #1: Simple leaf deletion
1212 2929 5252
22 77 99 1515 2222 5656 6969 72723131 4343
Delete 2: Since there are enoughkeys in the node, just delete it
Assuming a 5-wayB-Tree, as before...
Type #2: Simple non-leaf deletion
1212 2929 5252
77 99 1515 2222 5656 6969 72723131 4343
Delete 52
Borrow the predecessoror (in this case) successor
5656
Delete 52Delete 72
Type #4: Too few keys in node and its siblings
1212 2929 5656
77 99 1515 2222 6969 72723131 4343
Delete 72Too few keys!
Join back together
Type #4: Too few keys in node and its siblings
1212 2929
77 99 1515 2222 696956563131 4343
Delete 22
Type #3: Enough siblings
1212 2929
77 99 1515 2222 696956563131 4343
Delete 22
Demote root key andpromote leaf key
Type #4: Too few keys in node and its siblings
1212 2929 5656
77 99 1515 2222 6969 72723131 4343
Delete 72Too few keys!
Join back together
Type #3: Enough siblings
1212
292977 99 1515
3131
696956564343
Deletion Exercise
Delete 95,226
Given a B tree of Order 5
Result after deletion of 95,226
Delete 221
Result after deletion of 221
Delete 70
B Tree after Deletion of 70
Exercise to do
• Given 5-way B-tree created by these data (last exercise):
3, 7, 9, 23, 45, 1, 5, 14, 25, 24, 13, 11, 8, 19, 4, 31, 35, 56
• Further Add the following keys:
– 2, 6,12
• Delete the following keys:
– 4, 5, 7, 3, 14
Comparing Trees• Binary trees
– Can become unbalanced and lose their good time complexity (big O)
– AVL trees are strict binary trees that overcome the balance problem
– Heaps remain balanced but only prioritise (not order) the keys
• Multi-way trees
– B-Trees can be m-way, they can have any (odd) number of children
– One B-Tree, the 2-3 (or 3-way) B-Tree, approximates a permanently balanced binary tree, exchanging the AVL tree’s balancing operations for insertion and (more complex) deletion operations
Huffman Coding: An Application of Binary Trees and
Priority Queues
Encoding and Compression of Data
• ASCII
• Variations on ASCII
– min number of bits needed
– cost of savings
– patterns
– modifications
Purpose of Huffman Coding
• Proposed by Dr. David A. Huffman in 1952
– “A Method for the Construction of Minimum Redundancy Codes”
• Applicable to many forms of data transmission
– Our example: text files
The Basic Algorithm
• Huffman coding is a form of statistical coding
• Not all characters occur with the same frequency!
• Yet all characters are allocated the same amount of space
– 1 char = 1 byte, be it e or x
The Basic Algorithm
• Any savings in tailoring codes to frequency of character?
• Code word lengths are no longer fixed like ASCII.
• Code word lengths vary and will be shorter for the more frequently used characters.
The (Real) Basic Algorithm
1. Scan text to be compressed and tally occurrence of all characters.
2. Sort or prioritize characters based on number of occurrences in text.
3. Build Huffman code tree based on prioritized list.
4. Perform a traversal of tree to determine all code words.
5. Scan text again and create new file using the Huffman codes.
Building a TreeScan the original text
• Consider the following short text:
Eerie eyes seen near lake.• Count up the occurrences of all characters in the text
Building a TreeScan the original text
Eerie eyes seen near lake.
• What characters are present?
E e r i space y s n a r l k .
Building a TreeScan the original text
Eerie eyes seen near lake.• What is the frequency of each character in the text?
Building a TreePrioritize characters
• Create binary tree nodes with character and frequency of each character
• Place nodes in a priority queue
– The lower the occurrence, the higher the priority in the queue
Building a Tree
• The queue after inserting all nodes
• Null Pointers are not shown
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
Building a Tree
• While priority queue contains two or more nodes
– Create new node
– Dequeue node and make it left subtree
– Dequeue next node and make it right subtree
– Frequency of new node equals sum of frequency of left and right children
– Enqueue new node back into queue
Building a Tree
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
Building a Tree
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
Building a Tree
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
Building a Tree
E
1
i
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
y
1
l
1
2
Building a Tree
E
1
i
1
k
1
.
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
y
1
l
1
2
Building a Tree
E
1
i
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
Building a Tree
E
1
i
1
r
2
s
2
n
2
a
2
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
Building a Tree
E
1
i
1
n
2
a
2sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
Building a Tree
E
1
i
1
n
2
a
2
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
Building a Tree
E
1
i
1
sp
4
e
82
y
1
l
1
2k
1
.
1
2
r
2
s
2
4
n
2
a
2
4 4
Building a Tree
E
1
i
1
sp
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4 4
6
Building a Tree
E
1
i
1
sp
4
e
82
y
1
l
1
2
k
1
.
1
2r
2
s
2
4
n
2
a
2
4 4 6
What is happening to the characters with a low number of occurrences?
Building a Tree
E
1
i
1
sp
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4 6
8
Building a Tree
E
1
i
1
sp
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4 6 8
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46
8
10
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2r
2
s
2
4
n
2
a
2
4 46
8 10
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46
8
10
16
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46
8
10 16
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46
8
1016
26
Building a Tree
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46
8
1016
26
After enqueueing this node there is only one node left in priority queue.
Building a Tree
Dequeue the single node left in the queue.
This tree contains the new code words for each character.
Frequency of root node should equal number of characters in text.
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46 8
1016
26
Eerie eyes seen near lake. � 26 characters
Encoding the FileTraverse Tree for Codes
• Perform a traversal of the tree to obtain new code words
• Going left is a 0 going right is a 1
• code word is only completed when a leaf node is reached
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46 8
1016
26
Encoding the FileTraverse Tree for Codes
Char CodeE 0000i 0001y 0010l 0011k 0100. 0101space011e 10r 1100s 1101n 1110a 1111
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46 8
1016
26
Encoding the File
• Rescan text and encode file using new code words
Eerie eyes seen near lake.
Char CodeE 0000i 0001y 0010l 0011k 0100. 0101space011e 10r 1100s 1101n 1110a 1111
0000101100000110011100010101101101001111101011111100011001111110100100101
• Why is there no need for a separator character?
.
Encoding the FileResults
• Have we made things any better?
• 73 bits to encode the text
• ASCII would take 8 * 26 = 208 bits
0000101100000110011100010101101101001111101011111100011001111110100100101
If modified code used 4 bits per character are needed. Total bits 4 * 26 = 104. Savings not as great.
Decoding the File
• How does receiver know what the codes are?
• Tree constructed for each text file.
– Considers frequency for each file
– Big hit on compression, especially for smaller files
• Tree predetermined
– based on statistical analysis of text files or file types
• Data transmission is bit based versus byte based
Decoding the File
• Once receiver has tree it scans incoming bit stream
• 0 ⇒ go left
• 1 ⇒ go right
E
1
i
1
sp
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
46 8
1016
26
10100011011110111101111110000110101
Summary
• Huffman coding is a technique used to compress files for transmission
• Uses statistical coding
– more frequently used symbols have shorter code words
• Works well for text and fax transmissions
• An application that uses several data structures
HUFFMAN’S Algorithm
• Data ITEM : A B C D E F G H
• Weight 22 5 11 19 2 11 25 5
A
BCF
DE
G
G
CBA
E DF
H
Z
Y
W S
TPR
X Q