csc401 -- analysis of algorithms 3-1 csc401 – analysis of algorithms chapter 3 search trees and...

CSC401 -- Analysis of AlgorithmsCSC401 -- Analysis of Algorithms 3-3-11

CSC401 – Analysis of AlgorithmsCSC401 – Analysis of Algorithms Chapter 3Chapter 3

Search Trees and Skip ListsSearch Trees and Skip ListsObjectives:Objectives:

Review binary search trees and present Review binary search trees and present operations on binary search treesoperations on binary search treesAnalyze the performance of binary search tree Analyze the performance of binary search tree operationsoperationsReview balanced binary search trees: AVL tree Review balanced binary search trees: AVL tree and Red-Black tree, and multiway search treesand Red-Black tree, and multiway search treesIntroduce splay trees and skip listsIntroduce splay trees and skip listsAnalyze the performance of splay trees and Analyze the performance of splay trees and skip listsskip lists


Binary Search TreeBinary Search Tree

A binary search tree is A binary search tree is a binary tree storing a binary tree storing keys (or key-element keys (or key-element pairs) at its internal pairs) at its internal nodes and satisfying nodes and satisfying the following property:the following property:– Let Let uu, , vv, and , and ww be three be three

nodes such that nodes such that uu is in is in the left subtree of the left subtree of vv and and ww is in the right subtree is in the right subtree of of vv. We have . We have keykey((uu)) keykey((vv) ) keykey((ww))

External nodes do not External nodes do not store itemsstore items

An inorder traversal of An inorder traversal of a binary search trees a binary search trees visits the keys in visits the keys in increasing orderincreasing order

6

92

41 8


SearchSearchTo search for a key To search for a key kk, , we trace a downward we trace a downward path starting at the rootpath starting at the root

The next node visited The next node visited depends on the depends on the outcome of the outcome of the comparison of comparison of kk with with the key of the current the key of the current nodenode

If we reach a leaf, the If we reach a leaf, the key is not found and we key is not found and we return NO_SUCH_KEYreturn NO_SUCH_KEY

Example: Example: findElementfindElement(4)(4)

Algorithm findElement(k, v)if T.isExternal (v)

return NO_SUCH_KEYif k key(v)

return findElement(k, T.leftChild(v))else if k key(v)

return element(v)else { k key(v) }

return findElement(k, T.rightChild(v))

6

92

41 8


InsertionInsertionTo perform operation To perform operation insertIteminsertItem(k, o), we (k, o), we search for key ksearch for key k

Assume k is not Assume k is not already in the tree, already in the tree, and let let w be the and let let w be the leaf reached by the leaf reached by the searchsearch

We insert k at node w We insert k at node w and expand w into an and expand w into an internal nodeinternal node

Example: insert 5Example: insert 5

6

92

41 8

w

6

92

41 8

5w


DeletionDeletionTo perform operation To perform operation removeElementremoveElement((kk), we ), we search for key search for key kk

Assume key Assume key kk is in the is in the tree, and let let tree, and let let vv be the be the node storing node storing kk

If node If node vv has a leaf child has a leaf child ww, we remove , we remove vv and and ww from the tree with from the tree with operation operation removeAboveExternalremoveAboveExternal((ww))

Example: remove 4Example: remove 4

6

92

51 8

6

92

41 8

5

vw


Deletion (cont.)Deletion (cont.)We consider the case We consider the case where the key where the key kk to be to be removed is stored at a removed is stored at a node node vv whose children whose children are both internalare both internal– we find the internal node we find the internal node

w w that follows that follows vv in an in an inorder traversalinorder traversal

– we copy we copy keykey((ww)) into node into node vv– we remove node we remove node w w and its and its

left child left child z z (which must be (which must be a leaf) by means of a leaf) by means of operation operation removeAboveExternalremoveAboveExternal((zz))

Example: remove 3Example: remove 3

3

1

8

6 9

5

v

w

z

2

5

1

8

6 9

v

2


PerformancePerformanceConsider a dictionary Consider a dictionary with with nn items items implemented by means implemented by means of a binary search tree of a binary search tree of height of height hh– the space used is the space used is OO((nn))– methods methods findElement findElement , ,

insertIteminsertItem and and removeElementremoveElement take take OO((hh) ) timetime

The height The height hh is is OO((nn) ) in in the worst case and the worst case and OO(log (log nn)) in the best case in the best case


AVL Tree DefinitionAVL Tree Definition

AVL trees are AVL trees are balanced.balanced.

An AVL Tree is a An AVL Tree is a binary search treebinary search tree such that for every such that for every internal node v of T, internal node v of T, the the heights of the heights of the children of v can differ children of v can differ by at most 1.by at most 1.

88

44

17 78

32 50

48 62

2

4

1

1

2

3

1

1

An example of an AVL tree where the heights are shown next to the nodes


Height of an AVL TreeHeight of an AVL TreeFactFact: The : The heightheight of an AVL tree of an AVL tree

storing n keys is O(log n).storing n keys is O(log n).ProofProof: Let us bound : Let us bound n(h):n(h): the minimum number of the minimum number of internal nodes of an AVL tree of height h.internal nodes of an AVL tree of height h.We easily see that n(1) = 1 and n(2) = 2We easily see that n(1) = 1 and n(2) = 2For n > 2, an AVL tree of height h contains the root For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of node, one AVL subtree of height n-1 and another of height n-2.height n-2.That is, n(h) = 1 + n(h-1) + n(h-2)That is, n(h) = 1 + n(h-1) + n(h-2)Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). SoKnowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). Son(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by

induction), n(h) > 2induction), n(h) > 2iin(h-2i)n(h-2i)

Solving the base case we get: n(h) > 2 Solving the base case we get: n(h) > 2 h/2-1h/2-1

Taking logarithms: h < 2log n(h) +2Taking logarithms: h < 2log n(h) +2Thus the height of an AVL tree is O(log n)Thus the height of an AVL tree is O(log n)

3

4 n(1)

n(2)


Insertion in an AVL TreeInsertion in an AVL TreeInsertion is as in a binary search treeInsertion is as in a binary search treeAlways done by expanding an external node.Always done by expanding an external node.Example:Example: 44

17 78

32 50 88

48 62

54

w

b=x

a=y

c=z

44

17 78

32 50 88

48 62

before insertion after insertion


Trinode RestructuringTrinode Restructuringlet (let (aa,,bb,,cc) be an inorder listing of ) be an inorder listing of xx, , yy, , zzperform the rotations needed to make perform the rotations needed to make bb the the topmost node of the threetopmost node of the three

b=y

a=z

c=x

T0

T1

T2 T3

b=y

a=z c=x

T0 T1 T2 T3

c=y

b=x

a=z

T0

T1 T2

T3 b=x

c=ya=z

T0 T1 T2 T3

case 1: single rotation(a left rotation about a)

case 2: double rotation(a right rotation about c, then a left rotation about a)

(other two cases are symmetrical)


Insertion Example, Insertion Example, continuedcontinued

88

44

17 78

32 50

48 62

2

5

1

1

3

4

2

1

54

1

T0T2

T3

x

y

z

2

3

4

5

67

1

88

44

17

7832 50

48

622

4

1

1

2 2

3

154

1

T0 T1

T2

T3

x

y z

unbalanced...

...balanced

1

2

3

4

5

6

7

T1


Restructuring (Single Rotations)Restructuring (Single Rotations)Single Rotations:Single Rotations:

T0T1

T2

T3

c = xb = y

a = z

T0 T1 T2

T3

c = xb = y

a = zsingle rotation

T3T2

T1

T0

a = xb = y

c = z

T0T1T2

T3

a = xb = y

c = zsingle rotation


Restructuring (Double Rotations)Restructuring (Double Rotations)double rotations:double rotations:

double rotationa = z

b = xc = y

T0T2

T1

T3 T0

T2T3T1

a = zb = x

c = y

double rotationc = z

b = xa = y

T0T2

T1

T3 T0

T2T3 T1

c = zb = x

a = y


Removal in an AVL TreeRemoval in an AVL TreeRemoval begins as in a binary search tree, Removal begins as in a binary search tree, which means the node removed will become an which means the node removed will become an empty external node. Its parent, w, may cause empty external node. Its parent, w, may cause an imbalance.an imbalance.Example: Example:

44

17

7832 50

8848

62

54

44

17

7850

8848

62

54

before deletion of 32 after deletion


Rebalancing after a Rebalancing after a RemovalRemoval

Let Let zz be the be the first unbalancedfirst unbalanced node encountered node encountered while travelling up the tree from w. Also, let y be while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of z with the larger height, and let x be the child of y with the larger height.the child of y with the larger height.We perform We perform restructurerestructure(x) to restore balance at z.(x) to restore balance at z.As this restructuring may upset the balance of As this restructuring may upset the balance of another node higher in the tree, we must continue another node higher in the tree, we must continue checking for balance until the root of T is reachedchecking for balance until the root of T is reached

44

17

7850

8848

62

54

w

c=x

b=y

a=z

44

17

78

50 88

48

62

54


Running Times for AVL Running Times for AVL TreesTrees

a single restructure is O(1)a single restructure is O(1)– using a linked-structure binary treeusing a linked-structure binary tree

find is O(log n)find is O(log n)– height of tree is O(log n), no restructures neededheight of tree is O(log n), no restructures needed

insert is O(log n)insert is O(log n)– initial find is O(log n)initial find is O(log n)– Restructuring up the tree, maintaining heights is O(log n)Restructuring up the tree, maintaining heights is O(log n)

remove is O(log n)remove is O(log n)– initial find is O(log n)initial find is O(log n)– Restructuring up the tree, maintaining heights is O(log n)Restructuring up the tree, maintaining heights is O(log n)


Red-Black TreeRed-Black TreeA red-black tree can also be defined as a binary A red-black tree can also be defined as a binary search tree that satisfies the following properties:search tree that satisfies the following properties:– Root PropertyRoot Property: the root is black: the root is black– External PropertyExternal Property: every leaf is black: every leaf is black– Internal PropertyInternal Property: the children of a red node are black: the children of a red node are black– Depth PropertyDepth Property: all the leaves have the same black depth: all the leaves have the same black depth

9

154

62 12

7

21

Theorem:Theorem: A red-black A red-black tree storing tree storing nn items items has height has height OO(log (log nn))– The height of a red-The height of a red-

black tree is at most black tree is at most twice the height of its twice the height of its associated (2,4) tree, associated (2,4) tree, which is which is OO(log (log nn))

The search algorithm for a binary search tree is the The search algorithm for a binary search tree is the same as that for a binary search treesame as that for a binary search treeBy the above theorem, searching in a red-black tree By the above theorem, searching in a red-black tree takes takes OO(log (log nn)) time time


InsertionInsertionTo perform operation To perform operation insertIteminsertItem((kk, , oo)), we execute the , we execute the insertion algorithm for binary search trees and color insertion algorithm for binary search trees and color redred the the newly inserted node newly inserted node z z unless it is the rootunless it is the root– We preserve the root, external, and depth propertiesWe preserve the root, external, and depth properties– If the parent If the parent vv of of zz is black, we also preserve the internal is black, we also preserve the internal

property and we are done property and we are done – Else (Else (vv is red ) we have a is red ) we have a double reddouble red (i.e., a violation of (i.e., a violation of

the internal property), which requires a reorganization of the internal property), which requires a reorganization of the treethe tree

Example where the insertion of 4 causes a double red:Example where the insertion of 4 causes a double red:

6

3 8

6

3 8

4z

v v

z


Remedying a Double RedRemedying a Double RedConsider a double red with child Consider a double red with child z z and parent and parent vv, and let , and let ww be the sibling of be the sibling of vv

4

6

7z

vw2

4 6 7

.. 2 ..

Case 1Case 1: : ww is black is black– The double red is an The double red is an

incorrect replacement incorrect replacement of a 4-nodeof a 4-node

– RestructuringRestructuring: we : we change the 4-node change the 4-node replacementreplacement

Case 2Case 2: : ww is red is red– The double red The double red

corresponds to an corresponds to an overflowoverflow

– RecoloringRecoloring: we : we perform the perform the equivalent of a equivalent of a splitsplit

4

6

7z

v

2 4 6 7

2w


RestructuringRestructuringA restructuring remedies a child-parent double red when A restructuring remedies a child-parent double red when the parent red node has a black siblingthe parent red node has a black sibling

It is equivalent to restoring the correct replacement of a 4-It is equivalent to restoring the correct replacement of a 4-nodenode

The internal property is restored and the other properties The internal property is restored and the other properties are preservedare preserved

4

6

7z

vw2

4 6 7

.. 2 ..

4

6

7

z

v

w2

4 6 7

.. 2 ..


Restructuring (cont.)Restructuring (cont.)There are four restructuring configurations There are four restructuring configurations depending on whether the double red nodes depending on whether the double red nodes are left or right childrenare left or right children

2

4

6

6

2

4

6

4

2

2

6

4

2 6

4


RecoloringRecoloringA recoloring remedies a child-parent double red when the A recoloring remedies a child-parent double red when the parent red node has a red siblingparent red node has a red siblingThe parent The parent vv and its sibling and its sibling ww become black and the become black and the grandparent grandparent uu becomes red, unless it is the root becomes red, unless it is the rootIt is equivalent to performing a split on a 5-nodeIt is equivalent to performing a split on a 5-nodeThe double red violation may propagate to the The double red violation may propagate to the grandparent grandparent uu

4

6

7z

v

2 4 6 7

2w

4

6

7z

v

6 7

2w

… 4 …

2


Analysis of InsertionAnalysis of InsertionRecall that a red-black Recall that a red-black tree has tree has OO(log (log nn)) height height

Step 1 takes Step 1 takes OO(log (log nn)) time time because we visit because we visit OO(log (log nn)) nodesnodes

Step 2 takes Step 2 takes OO(1)(1) time time

Step 3 takes Step 3 takes OO(log (log nn)) time time because we performbecause we perform– OO(log (log nn) ) recolorings, each recolorings, each

taking taking OO(1)(1) time, and time, and– at most one restructuring at most one restructuring

taking taking OO(1)(1) time time

Thus, an insertion in a Thus, an insertion in a red-black tree takes red-black tree takes OO(log (log nn)) time time

AlgorithmAlgorithm insertIteminsertItem((kk, , oo))

1.1. We search for key We search for key kk to locate to locate the insertion node the insertion node zz

2.2. We add the new item (We add the new item (kk, , oo) at ) at node node z z and color and color zz red red

3. 3. whilewhile doubleReddoubleRed((zz))if if isBlackisBlack((siblingsibling((parentparent((zz))))))

z z restructure restructure((zz))returnreturn

else else {{ siblingsibling((parentparent((zz) is red }) is red } z z recolor recolor((zz))


DeletionDeletionTo perform operation To perform operation removeremove((kk)), we first execute the , we first execute the deletion algorithm for binary search treesdeletion algorithm for binary search trees

Let Let vv be the internal node removed, be the internal node removed, ww the external node the external node removed, and removed, and rr the sibling of the sibling of ww– If either If either vv of of rr was red, we color was red, we color rr black and we are done black and we are done– Else (Else (vv and and rr were both black) we color were both black) we color rr double blackdouble black, which , which

is a violation of the internal property requiring a reorganization is a violation of the internal property requiring a reorganization of the treeof the tree

Example where the deletion of 8 causes a double black:Example where the deletion of 8 causes a double black:

6

3 8

4

v

r w

6

3

4

r


Remedying a Double BlackRemedying a Double BlackThe algorithm for remedying a double black node The algorithm for remedying a double black node ww with with sibling sibling yy considers three cases considers three casesCase 1Case 1: : yy is black and has a red child is black and has a red child– We perform a We perform a restructuringrestructuring, equivalent to a , equivalent to a transfer transfer , and , and

we are donewe are done

Case 2Case 2: : yy is black and its children are both black is black and its children are both black– We perform a We perform a recoloringrecoloring, equivalent to a , equivalent to a fusionfusion, which may , which may

propagate up the double black violationpropagate up the double black violation

Case 3Case 3: : yy is red is red– We perform an We perform an adjustmentadjustment, equivalent to choosing a , equivalent to choosing a

different representation of a 3-node, after which either different representation of a 3-node, after which either Case 1 or Case 2 appliesCase 1 or Case 2 applies

Deletion in a red-black tree takes Deletion in a red-black tree takes OO(log (log nn)) time time


Red-Black Tree ReorganizationRed-Black Tree Reorganizationremedy double redremedy double redInsertionInsertion

double red removed double red removed or propagated upor propagated upsplitsplitrecoloringrecoloring

double red removeddouble red removedchange of 4-node change of 4-node representationrepresentationrestructuringrestructuring

resultresult(2,4) tree action(2,4) tree actionRed-black tree actionRed-black tree action

remedy double blackremedy double blackDeletionDeletion

restructuring or restructuring or recoloring followsrecoloring follows

change of 3-node change of 3-node representationrepresentationadjustmentadjustment

double black removed double black removed or propagated upor propagated upfusionfusionrecoloringrecoloring

double black removeddouble black removedtransfertransferrestructuringrestructuring

resultresult(2,4) tree action(2,4) tree actionRed-black tree actionRed-black tree action


Multi-Way Search TreeMulti-Way Search TreeA multi-way search tree is an ordered tree such that A multi-way search tree is an ordered tree such that – Each internal node has at least two children and stores Each internal node has at least two children and stores dd1 1

key-element items key-element items ((kkii, , ooii)), where , where d d is the number of children is the number of children – For a node with children For a node with children vv1 1 vv22 … … vvdd storing keys storing keys kk1 1 kk22 … … kkdd11

keys in the subtree of keys in the subtree of vv1 1 are less than are less than kk11

keys in the subtree of keys in the subtree of vvii are between are between kkii1 1 and and kkii ((ii = 2, …, = 2, …, dd1)1)

keys in the subtree of keys in the subtree of vvdd are greater than are greater than kkdd11

– The leaves store no items and serve as placeholdersThe leaves store no items and serve as placeholders11 24

2 6 8 15

30

27 32


Multi-Way Inorder TraversalMulti-Way Inorder TraversalWe can extend the notion of inorder traversal from We can extend the notion of inorder traversal from binary trees to multi-way search treesbinary trees to multi-way search trees

Namely, we visit item Namely, we visit item ((kkii, , ooii)) of node of node vv between the between the recursive traversals of the subtrees of recursive traversals of the subtrees of vv rooted at rooted at children children vvii and and vvii11

An inorder traversal of a multi-way search tree visits the An inorder traversal of a multi-way search tree visits the keys in increasing orderkeys in increasing order

11 24

2 6 8 15

30

27 32

1 3 5 7 9 11 13 19

15 17

2 4 6 14 18

8 12

10

16


Multi-Way SearchingMulti-Way SearchingSimilar to search in a binary search treeSimilar to search in a binary search tree

A each internal node with children A each internal node with children vv1 1 vv22 … … vvdd and keys and keys kk1 1 kk22 … … kkdd11

– kk kkii ( (ii = 1, …, = 1, …, dd1)1): the search terminates successfully: the search terminates successfully– kk kk11: we continue the search in child : we continue the search in child vv11

– kkii1 1 kk kkii ( (ii = 2, …, = 2, …, dd1)1): we continue the search in child : we continue the search in child vvii

– kk kkdd11: we continue the search in child : we continue the search in child vvdd

Reaching an external node terminates the search Reaching an external node terminates the search unsuccessfullyunsuccessfullyExample: search for 30Example: search for 30 11 24

2 6 8 15

30

27 32


(2,4) Tree(2,4) TreeA (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi-A (2,4) tree (also called 2-4 tree or 2-3-4 tree) is a multi-way search with the following propertiesway search with the following properties– Node-Size PropertyNode-Size Property: every internal node has at most : every internal node has at most

four childrenfour children– Depth PropertyDepth Property: all the external nodes have the same : all the external nodes have the same

depthdepth

Depending on the number of children, an internal node of Depending on the number of children, an internal node of a (2,4) tree is called a 2-node, 3-node or 4-nodea (2,4) tree is called a 2-node, 3-node or 4-node

10 15 24

2 8 12 27 3218


Height of a (2,4) TreeHeight of a (2,4) TreeTheorem:Theorem: A (2,4) tree storing A (2,4) tree storing nn items has height items has height OO(log (log nn))Proof:Proof:– Let Let hh be the height of a (2,4) tree with be the height of a (2,4) tree with n n itemsitems– Since there are at least Since there are at least 22ii items at depth items at depth ii 0, … , 0, … , h h 1 1 and and

no items at depth no items at depth hh, we have, we have nn 1 1 2 2 4 4 … … 2 2hh1 1 22h h 11

– Thus, Thus, hh log (log (n n 1)1)

Searching in a (2,4) tree with Searching in a (2,4) tree with nn items takes items takes OO(log (log nn)) timetime

1

2

2h1

0

items

0

1

h1

h

depth


InsertionInsertionWe insert a new item We insert a new item ((kk, , oo)) at the parent at the parent vv of the of the leaf reached by searching for leaf reached by searching for kk– We preserve the depth property but We preserve the depth property but – We may cause an We may cause an overflowoverflow (i.e., node (i.e., node vv may become a may become a

5-node)5-node)

Example: inserting key 30 causes an overflowExample: inserting key 30 causes an overflow

27 32 35

10 15 24

2 8 12 18

v

10 15 24

2 8 12 27 30 32 3518v


Overflow and SplitOverflow and SplitWe handle an We handle an overflowoverflow at a 5-node at a 5-node vv with a with a split operationsplit operation::– let let vv11 … … vv55 be the children of be the children of vv and and kk11 … … kk44 be the keys of be the keys of vv

– node node vv is replaced nodes is replaced nodes vv' ' and and vv""vv'' is a 3-node with keys is a 3-node with keys kk11 kk22 and children and children vv11 vv22 vv33

vv"" is a 2-node with key is a 2-node with key kk44 and children and children vv44 vv55

– key key kk3 3 is inserted into the parent is inserted into the parent uu of of v v (a new root may be created)(a new root may be created)

The overflow may propagate to the parent node The overflow may propagate to the parent node uu

15 24

12 27 30 32 3518v

u

v1 v2 v3 v4 v5

15 24 32

12 27 3018v'

u

v1 v2 v3 v4 v5

35v"


Analysis of InsertionAnalysis of Insertion

AlgorithmAlgorithm insertIteminsertItem((kk, , oo))

1.1. We search for key We search for key kk to locate to locate the insertion node the insertion node vv

2.2. We add the new item (We add the new item (kk, , oo) at ) at node node vv

3. 3. whilewhile overflowoverflow((vv))

if if isRootisRoot((vv))

create a new empty root create a new empty root above above vv

v v split split((vv))

Let Let TT be a (2,4) tree be a (2,4) tree with with nn items items– Tree Tree TT has has O O(log (log nn) ) heightheight – Step 1 takes Step 1 takes OO(log (log nn)) time time

because we visit because we visit OO(log (log nn)) nodesnodes

– Step 2 takes Step 2 takes OO(1)(1) time time– Step 3 takes Step 3 takes OO(log (log nn)) time time

because each split takes because each split takes OO(1)(1) time and we perform time and we perform OO(log (log nn) ) splitssplits

Thus, an insertion in a Thus, an insertion in a (2,4) tree takes (2,4) tree takes OO(log (log nn)) timetime


DeletionDeletionWe reduce deletion of an item to the case where the item We reduce deletion of an item to the case where the item is at the node with leaf childrenis at the node with leaf children

Otherwise, we replace the item with its inorder successor Otherwise, we replace the item with its inorder successor (or, equivalently, with its inorder predecessor) and delete (or, equivalently, with its inorder predecessor) and delete the latter itemthe latter item

Example: to delete key 24, we replace it with 27 (inorder Example: to delete key 24, we replace it with 27 (inorder successor)successor)

27 32 35

10 15 24

2 8 12 18

32 35

10 15 27

2 8 12 18


Underflow and FusionUnderflow and FusionDeleting an item from a node Deleting an item from a node vv may cause an may cause an underflowunderflow, where node , where node vv becomes a 1-node with one becomes a 1-node with one child and no keyschild and no keysTo handle an underflow at node To handle an underflow at node v v with parent with parent uu, we , we consider two casesconsider two casesCase 1:Case 1: the adjacent siblings of the adjacent siblings of vv are 2-nodes are 2-nodes– Fusion operation:Fusion operation: we merge we merge vv with an adjacent sibling with an adjacent sibling ww and and

move an item from move an item from uu to the merged node to the merged node vv''– After a fusion, the underflow may propagate to the parent After a fusion, the underflow may propagate to the parent uu

9 14

2 5 7 10

u

v

9

10 14

u

v'w2 5 7


Underflow and TransferUnderflow and TransferTo handle an underflow at node To handle an underflow at node v v with parent with parent uu, we , we consider two casesconsider two casesCase 2:Case 2: an adjacent sibling an adjacent sibling ww of of vv is a 3-node or a 4- is a 3-node or a 4-nodenode– Transfer operation:Transfer operation:

1. we move a child of 1. we move a child of ww to to vv 2. we move an item from 2. we move an item from uu to to vv3. we move an item from 3. we move an item from ww to to uu

– After a transfer, no underflow occursAfter a transfer, no underflow occurs

4 9

6 82

u

vw

4 8

62 9

u

vw


Analysis of DeletionAnalysis of DeletionLet Let TT be a (2,4) tree with be a (2,4) tree with nn items items– Tree Tree TT has has O O(log (log nn) ) heightheight

In a deletion operationIn a deletion operation– We visit We visit OO(log (log nn)) nodes to locate the node from nodes to locate the node from

which to delete the itemwhich to delete the item– We handle an underflow with a series of We handle an underflow with a series of OO(log (log

nn)) fusions, followed by at most one transfer fusions, followed by at most one transfer– Each fusion and transfer takes Each fusion and transfer takes OO(1)(1) time time

Thus, deleting an item from a (2,4) tree Thus, deleting an item from a (2,4) tree takes takes OO(log (log nn)) time time


From (2,4) to Red-Black TreesFrom (2,4) to Red-Black TreesA red-black tree is a representation of a (2,4) tree by A red-black tree is a representation of a (2,4) tree by means of a binary tree whose nodes are colored means of a binary tree whose nodes are colored redred or or blackblack

In comparison with its associated (2,4) tree, a red-In comparison with its associated (2,4) tree, a red-black tree hasblack tree has– same logarithmic time performancesame logarithmic time performance– simpler implementation with a single node typesimpler implementation with a single node type

2 6 73 54

4 6

2 7

5

3

3

5OR


Splay Tree DefinitionSplay Tree Definitiona a splay treesplay tree is a binary search tree where is a binary search tree where a node is splayed after it is accessed (for a a node is splayed after it is accessed (for a search or update)search or update)– deepest internal node accessed is splayeddeepest internal node accessed is splayed– splaying costs O(h), where h is height of the splaying costs O(h), where h is height of the

tree – which is still O(n) worst-casetree – which is still O(n) worst-caseO(h) rotations, each of which is O(1)O(h) rotations, each of which is O(1)

Binary Search Tree Rules:Binary Search Tree Rules:– items stored only at internal nodesitems stored only at internal nodes– keys stored at nodes in the left subtree of keys stored at nodes in the left subtree of vv are less are less

than or equal to the key stored at than or equal to the key stored at vv– keys stored at nodes in the right subtree of keys stored at nodes in the right subtree of vv are greater are greater

than or equal to the key stored at than or equal to the key stored at vv

An inorder traversal will return the keys in orderAn inorder traversal will return the keys in orderSearch proceeds down the tree to the found item Search proceeds down the tree to the found item or an external node.or an external node.


Splay Trees do Rotations after Splay Trees do Rotations after Every Operation (Even Search)Every Operation (Even Search)new operation: new operation: splaysplay– splaying moves a node to the root using rotationssplaying moves a node to the root using rotations

right rotation makes the left child x of a node y into

y’s parent; y becomes the right child of x

y

x

T1 T2

T3

y

x

T1

T2T3

left rotation makes the right child y of a node x

into x’s parent; x becomes the left child of y

y

x

T1 T2

T3

y

x

T1

T2T3

(structure of tree above y is not modified)

(structure of tree above x is not modified)

a right rotation about y a left rotation about x


Splaying:Splaying:

is x the root?

stop

is x a child of the root?

right-rotate about the root

left-rotate about the root

is x the left child of the

root?

is x a left-left grandchild?

is x a left-right grandchild?

is x a right-right grandchild?

is x a right-left grandchild?

right-rotate about g, right-rotate about p

left-rotate about g, left-rotate about p

left-rotate about p, right-rotate about g

right-rotate about p, left-rotate about g

start with node x

“x is a left-left grandchild” means x is a left child of its parent, which is itself a left child of its parent

p is x’s parent; g is p’s parent

no

yes

yes

yes

yes

yes

yes

no

no

yes zig-zig

zig-zag

zig-zag

zig-zig

zigzig


Visualizing the Splaying CasesVisualizing the Splaying Caseszig-zag

y

x

T2 T3

T4

z

T1

y

x

T2 T3 T4

z

T1

y

x

T1 T2

T3

z

T4

zig-zig

y

z

T4T3

T2

x

T1

zig

x

w

T1 T2

T3

y

T4

y

x

T2 T3 T4

w

T1


Splaying ExampleSplaying Examplelet let xx = (8,N) = (8,N)– xx is the right child of its is the right child of its

parent, which is the left parent, which is the left child of the grandparentchild of the grandparent

– left-rotate around left-rotate around pp, then , then right-rotate around right-rotate around gg

(20,Z)

(37,P)(21,O)(14,J)

(7,T)

(35,R)(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

x

g

p

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)(8,N)

(7,P)

(10,U)

x

g

p

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

g

p

1.(before rotating)

2.(after first rotation)

3. (after second rotation)

x is not yet the root, so we splay again


Splaying Example, ContinuedSplaying Example, Continuednownow x x is the left child of the root is the left child of the root– right-rotate around rootright-rotate around root

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

(10,A)

(20,Z)

(37,P)(21,O)

(35,R)

(36,L) (40,X)

(7,T)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(14,J)

(8,N)

(7,P)

(10,U)

x

1.(before applying rotation)

2.(after rotation)

x is the root, so stop


Example ResultExample Resulttree might not be more balancedtree might not be more balancede.g. splay (40,X)e.g. splay (40,X)– before, the depth of the before, the depth of the

shallowest leaf is 3 and the shallowest leaf is 3 and the deepest is 7deepest is 7

– after, the depth of shallowest after, the depth of shallowest leaf is 1 and deepest is 8leaf is 1 and deepest is 8

(20,Z)

(37,P)(21,O)(14,J)

(7,T)

(35,R)(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

(20,Z)

(37,P)

(21,O)

(14,J)(7,T)

(35,R)

(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P) (36,L)(10,U)

(40,X)

(20,Z)

(37,P)

(21,O)

(14,J)(7,T)

(35,R)

(10,A)

(1,C)

(1,Q)

(5,G)(2,R)

(5,H)

(6,Y)(5,I)

(8,N)

(7,P)

(36,L)

(10,U)

(40,X)

before

after first splayafter second splay


Splay Trees & Ordered DictionariesSplay Trees & Ordered Dictionaries

which nodes are splayed after each which nodes are splayed after each operation?operation?

use the parent of the internal node that was use the parent of the internal node that was actually removed from the tree (the parent actually removed from the tree (the parent of the node that the removed item was of the node that the removed item was swapped with)swapped with)

removeElementremoveElement

use the new node containing the item inserteduse the new node containing the item insertedinsertElementinsertElement

if key found, use that nodeif key found, use that nodeif key not found, use parent of ending external if key not found, use parent of ending external nodenode

findElementfindElement

splay nodesplay nodemethodmethod


Amortized Analysis of Splay TreesAmortized Analysis of Splay TreesRunning time of each operation is proportional to time for Running time of each operation is proportional to time for splaying.splaying.

Define rank(v) as the logn(v), where n(v) is the number of Define rank(v) as the logn(v), where n(v) is the number of nodes in subtree rooted at v.nodes in subtree rooted at v.

Costs: zig = $1, zig-zig = $2, zig-zag = $2.Costs: zig = $1, zig-zig = $2, zig-zag = $2.

Thus, cost for playing a node at depth d = $d.Thus, cost for playing a node at depth d = $d.

Imagine that we store rank(v) cyber-dollars at each node v of Imagine that we store rank(v) cyber-dollars at each node v of the splay tree (just for the sake of analysis).the splay tree (just for the sake of analysis).

Cost per Zig -- Cost per Zig -- Doing a Doing a zig at x costs at most zig at x costs at most rank’(x) - rank(x):rank’(x) - rank(x):– cost = rank’(x) + cost = rank’(x) +

rank’(y) - rank(y)rank’(y) - rank(y)

-rank(x)-rank(x)

<< rank’(x) - rank(x). rank’(x) - rank(x).

zig

x

w

T1 T2

T3

y

T4

y

x

T2 T3 T4

w

T1


Cost per zig-zig and zig-zagCost per zig-zig and zig-zag

Doing a zig-zig or zig-zag at x costs at most Doing a zig-zig or zig-zag at x costs at most 3(rank’(x) - rank(x)) - 2.3(rank’(x) - rank(x)) - 2.

– Proof: See Theorem 3.9, Page 192.Proof: See Theorem 3.9, Page 192.

y

x

T1 T2

T3

z

T4

zig-zig y

z

T4T3

T2

x

T1

zig-zagy

x

T2 T3

T4

z

T1

y

x

T2 T3 T4

z

T1


Performance of Splay TreesPerformance of Splay TreesCost of splaying a node Cost of splaying a node x at depth d of a tree x at depth d of a tree rooted at r is at most rooted at r is at most 3(rank(r)-rank(x))-d+23(rank(r)-rank(x))-d+2– Proof: Splaying x takes Proof: Splaying x takes

d/2 splaying substeps:d/2 splaying substeps: .2))(rank)(rank(3

2)/(2))(rank)(rank(3

2)2))(rank)(rank(3(

cost cost

0

1

2/

1

2/

1

dxr

ddxr

xx i

d

ii

i

d

i

Recall: rank of a node is logarithm of its size, thus, Recall: rank of a node is logarithm of its size, thus, amortized cost of any splay operation is amortized cost of any splay operation is O(log n).O(log n).

In fact, the analysis goes through for any reasonable In fact, the analysis goes through for any reasonable definition of rank(x).definition of rank(x).

This implies that splay trees can actually adapt to This implies that splay trees can actually adapt to perform searches on frequently-requested items perform searches on frequently-requested items much faster than O(log n) in some cases. (See much faster than O(log n) in some cases. (See Theorems 3.10 and 3.11.)Theorems 3.10 and 3.11.)


What is a Skip ListWhat is a Skip ListA A skip listskip list for a set for a set SS of distinct (key, element) of distinct (key, element) items is a series of lists items is a series of lists SS00, , SS1 1 , … , , … , SShh such that such that– Each list Each list SSii contains the special keys contains the special keys and and – List List SS00 contains the keys of contains the keys of S S in nondecreasing order in nondecreasing order – Each list is a subsequence of the previous one, i.e.,Each list is a subsequence of the previous one, i.e.,

SS0 0 SS1 1 … … SShh

– List List SSh h contains only the two special keyscontains only the two special keys

We show how to use a skip list to implement the We show how to use a skip list to implement the dictionary ADTdictionary ADT

56 64 78 31 34 44 12 23 26

31

64 31 34 23

S0

S1

S2

S3


SearchSearchWe search for a key We search for a key xx in a a skip list as follows: in a a skip list as follows:– We start at the first position of the top list We start at the first position of the top list – At the current position At the current position pp, we compare , we compare xx with with y y

keykey((afterafter((pp))))x x y y: we return : we return elementelement((afterafter((pp))))x x y y: we “scan forward” : we “scan forward” x x y y: we “drop down”: we “drop down”

– If we try to drop down past the bottom list, we return If we try to drop down past the bottom list, we return NO_SUCH_KEYNO_SUCH_KEY

Example: search for 78Example: search for 78

S0

S1

S2

S3

31

64 31 34 23

56 64 78 31 34 44 12 23 26


Randomized AlgorithmsRandomized AlgorithmsA A randomized randomized algorithmalgorithm performs performs coin tosses (i.e., uses coin tosses (i.e., uses random bits) to random bits) to control its executioncontrol its executionIt contains statements It contains statements of the typeof the type

bb randomrandom()()if if bb 0 0

do A …do A …elseelse { { bb 1} 1}

do B … do B …

Its running time Its running time depends on the depends on the outcomes of the coin outcomes of the coin tossestosses

We analyze the expected We analyze the expected running time of a running time of a randomized algorithm randomized algorithm under the following under the following assumptionsassumptions– the coins are unbiased, and the coins are unbiased, and – the coin tosses are the coin tosses are

independentindependent

The worst-case running The worst-case running time of a randomized time of a randomized algorithm is often large but algorithm is often large but has very low probability has very low probability (e.g., it occurs when all the (e.g., it occurs when all the coin tosses give “heads”)coin tosses give “heads”)We use a randomized We use a randomized algorithm to insert items algorithm to insert items into a skip listinto a skip list


To insert an item To insert an item ((xx, , oo)) into a skip list, we use a randomized into a skip list, we use a randomized algorithm:algorithm:– We repeatedly toss a coin until we get tails, and we denote We repeatedly toss a coin until we get tails, and we denote

with with i i the number of times the coin came up headsthe number of times the coin came up heads– If If i i h h, we add to the skip list new lists , we add to the skip list new lists SShh11, … , , … , SSi i 11, each , each

containing only the two special keyscontaining only the two special keys– We search for We search for x x in the skip list and find the positions in the skip list and find the positions pp00, , pp1 1 , …, , …,

ppi i of the items with largest key less than of the items with largest key less than xx in each list in each list SS00, , SS11, … , … , , SSii

– For For jj 0, …, 0, …, ii, we insert item , we insert item ((xx, , oo)) into list into list SSjj after position after position ppjj

Example: insert key Example: insert key 1515, with , with ii 2 2

InsertionInsertion

S0

S1

S2

S3

10 362315

15

2315

10 36

23

23

S0

S1

S2

p0

p1

p2


DeletionDeletionTo remove an item with key To remove an item with key xx from a skip list, we from a skip list, we proceed as follows:proceed as follows:– We search for We search for x x in the skip list and find the positions in the skip list and find the positions pp00, , pp1 1 , ,

…, …, ppi i of the items with key of the items with key xx, where position , where position ppjj is in list is in list SSjj

– We remove positions We remove positions pp00, , pp1 1 , …, , …, ppii from the lists from the lists SS00, , SS11, … , , … , SSii

– We remove all but one list containing only the two special We remove all but one list containing only the two special keyskeys

Example: remove key Example: remove key 3434

4512

23

23

S0

S1

S2

S0

S1

S2

S3

4512 23 34

34

23 34p0

p1

p2


ImplementationImplementationWe can implement a skip We can implement a skip list with quad-nodeslist with quad-nodes

A quad-node stores:A quad-node stores:– itemitem– link to the node beforelink to the node before– link to the node afterlink to the node after– link to the node belowlink to the node below– link to the node afterlink to the node after

Also, we define special Also, we define special keys PLUS_INF and keys PLUS_INF and MINUS_INF, and we modify MINUS_INF, and we modify the key comparator to the key comparator to handle them handle them

xquad-node


Space UsageSpace UsageThe space used by a skip The space used by a skip list depends on the list depends on the random bits used by random bits used by each invocation of the each invocation of the insertion algorithminsertion algorithm

We use the following two We use the following two basic probabilistic facts:basic probabilistic facts:Fact 1:Fact 1: The probability of The probability of

getting getting ii consecutive heads consecutive heads when flipping a coin is when flipping a coin is 1122ii

Fact 2:Fact 2: If each of If each of nn items is items is present in a set with present in a set with probability probability pp, the expected , the expected size of the set is size of the set is npnp

Consider a skip list with Consider a skip list with nn items items– By Fact 1, we insert an By Fact 1, we insert an

item in list item in list SSii with with probability probability 1122ii

– By Fact 2, the expected By Fact 2, the expected size of list size of list SSii is is nn22ii

The expected number The expected number of nodes used by the of nodes used by the skip list isskip list is

nnn h

ii

h

ii

22

1

2 00

Thus, the expected Thus, the expected space usage of a skip space usage of a skip list with list with nn items is items is OO((nn))


HeightHeightThe running time of the The running time of the search an insertion search an insertion algorithms is affected by algorithms is affected by the height the height hh of the skip list of the skip listWe show that with high We show that with high probability, a skip list with probability, a skip list with nn items has height items has height OO(log (log nn))We use the following We use the following additional probabilistic additional probabilistic fact:fact:Fact 3:Fact 3: If each of If each of nn events has events has

probability probability pp, the probability , the probability that at least one event that at least one event occurs is at most occurs is at most npnp

Consider a skip list with Consider a skip list with nn items items– By Fact 1, we insert an By Fact 1, we insert an

item in list item in list SSii with with probability probability 1122ii

– By Fact 3, the probability By Fact 3, the probability that list that list SSii has at least one has at least one item is at most item is at most nn22ii

By picking By picking ii 3log 3log nn, we , we have that the probability have that the probability that that SS3log 3log nn has at least one has at least one item isitem isat mostat most

nn223log 3log nn nnnn33 11nn22

Thus a skip list with Thus a skip list with nn items has height at most items has height at most 3log 3log nn with probability at with probability at least least 11 1 1nn22


Search and Update TimesSearch and Update TimesThe search time in a skip The search time in a skip list is proportional tolist is proportional to– the number of drop-down the number of drop-down

steps, plussteps, plus– the number of scan-the number of scan-

forward stepsforward steps

The drop-down steps are The drop-down steps are bounded by the height of bounded by the height of the skip list and thus are the skip list and thus are OO(log (log nn) ) with high with high probabilityprobabilityTo analyze the scan-To analyze the scan-forward steps, we use forward steps, we use yet another probabilistic yet another probabilistic fact:fact:Fact 4: Fact 4: The expected number The expected number

of coin tosses required in of coin tosses required in order to get tails is 2order to get tails is 2

When we scan forward in When we scan forward in a list, the destination key a list, the destination key does not belong to a does not belong to a higher listhigher list– A scan-forward step is A scan-forward step is

associated with a former associated with a former coin toss that gave tailscoin toss that gave tails

By Fact 4, in each list the By Fact 4, in each list the expected number of expected number of scan-forward steps is 2scan-forward steps is 2Thus, the expected Thus, the expected number of scan-forward number of scan-forward steps is steps is OO(log (log nn))We conclude that a We conclude that a search in a skip list takes search in a skip list takes OO(log (log nn) ) expected timeexpected timeThe analysis of insertion The analysis of insertion and deletion gives similar and deletion gives similar resultsresults


SummarySummaryA skip list is a data A skip list is a data structure for structure for dictionaries that uses dictionaries that uses a randomized a randomized insertion algorithminsertion algorithmIn a skip list with In a skip list with nn items items – The expected space The expected space

used is used is OO((nn))– The expected search, The expected search,

insertion and deletion insertion and deletion time is time is OO(log (log nn))

Using a more complex Using a more complex probabilistic analysis, probabilistic analysis, one can show that one can show that these performance these performance bounds also hold with bounds also hold with high probabilityhigh probability

Skip lists are fast and Skip lists are fast and simple to implement simple to implement in practicein practice

csc401 -- analysis of algorithms 3-1 csc401 – analysis of algorithms chapter 3 search trees and...

Documents

binary tree

avl tree of height h

binary search treesanalyze

internal node w

internal node v of t

bound nh

kif node v

avl subtree of height