data structures algorithmssubodh/courses/col106/lectures/... · 2019-09-20 · basic data...

Data Structures &

Algorithms

Subodh Kumar([email protected], Bharti 422)

Dept of Computer Sc. & Engg.

mailto:[email protected]

Basic Data Structures! Primitive types! List

! Global rank access! Access only at a single rank! Array, Vector

! Local relative access! Local update! Linked lists

!2

bitsmemory: sequence of bytes

base type

object

Linked List

ObjectArray

ReferenceArray




! Both! Sequence

!3Linked List

ReferenceArray




! Both! Sequence

! Queue/Deque! Stack

Restrict usage

True or False?! log (nk) = O(n)! (log n)k = O(n)! Average case complexity = O(worst case

complexity)! Average case complexity = o(worst case

complexity)

!5

[email protected]:t,t,t,f


Collection! Bag of Objects

! Need a way to identify objects ! Add a new object! Delete an identified object! Check if an identified object is present

!6

Dictionary

public interface Dictionary<K, V> { public V get(K key); public void put(K key, V value); public V remove(K key); public iterator<V> allvalues(); public iterator<K> allkeys();}

Array Implementation

!7

class Pair<A,B> { A one; B two;}public class ArrayMap<K,V> implements Dictionary<K, V> {

private Vector<Pair<K,V>> bag;public void put(K key, V value):

// Find an empty space and put Pair<K,V> therepublic V get(K key):

// Iterate of map looking for bag[i].one == key// Return bag[found].two

// Etc.}

! index = int i(key)

Hashing

!8

Array

(key,value)

Every key should have a different index.

Another key

key

Collision

! index = int i(key)! hashcode = int h(key)! index i = hashcode % N

Hashing

!9

Array

Every key should have a different hashcodeand then index also?

(key,value)

Every key should have a different index.

Another key

key

Collision

0

N-1

Collision Resolution! Separate chaining

!10

Array

(,) (,)

No. elementsLoad Factor ! =No. slots

Uniformly random keys ⇒Probability of collision ≤ min(!, 1)

= average chain length

Collision Resolution! Separate chaining

! Open Addressing! Linear probing

! Quadratic probing

! Double Hashing

! (Pseudo) Random probing!11

Array

k1, v1

k2, v2

Rehash

h1

h2

h3

until space locatedhi(k)= h(k) + ki

hi(k)= h(k) + ki2

hi(k)= h(k) + random(h(k), i)

hi(k)= h(k) + h’(k)i Index is “%N”Choose Prime N

size: N

Cuckoo Hashing

!12

Array

k1, v1k2, v2Rehash

h1

h2

h3

True or False?! Worst case query time for hash tables with

separate chaining is O(n2)! Worst case query time for hash tables with

linear probing is "(n)! Worst case insertion time for hash tables

with separate chaining is O(1)! Worst case deletion time for hash tables with

separate chaining is O(1)

!13

[email protected]:t,t,t,f


Hash of Integer! h(Integer i)

! [(a0 i + a1)% P ] %N ! P is a large prime > N ! (a0, a1) ∈ [0 .. P-1]

!14

Hash Functions! Middle r-bits! Add them up! Polynomial function

! Cyclic shift and add

! Universal hashing!15

object

Value Value Value Value Value

ha(k) = ∑ aiki %Ni=0

rWith prime a:

r+1 partsh = 0 h += Cyclic Shift(h) + ki

00001110011011101010

Universal Hash Function! Choose prime N! Divide key k into r+1 parts

k0, k1, .. kr s.t. max(k0) < N! Choose all possible hash functions where a0, a1, .. ar , are each in {0 .. N-1}! There are Nr+1 unique hash functions! At hash creation time, randomly choose a

! and let h = ha

! No need to explicitly enumerate the hash functions

!16

ha(k) = ∑ aiki % Ni=0

r

Probability (h(k1) = h(k2)) = 1/N

True or False?! Expected query time for hash tables with

load factor = 0.5 is O(1) for separate chaining.

! Expected query time for hash tables with load factor = 0.5 is O(1) for open addressing.

! Worst case deletion time for a hash table is O(1) if cuckoo hashing (open addressing) is used.

!17

[email protected]:t,t,f


Ordered Keys! Put it in a sorted list

! Insertion sort

!18

(,) (,) (,) (,)

(1,.) (3,.) (5,.) (9,.)8>key?

node p = sentinel;node n = sentinel.next;while ( p = n; n = n.next;}p.insertAfter(p, Pair(k, v))

< < <

Insert 8❌ ❌ ❌ ✔

(8,.)sentinel

n != null &&n.key < k) {

Skip-list

!19

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

(1,.) (5,.) (9,.) (14,.)

1495

(1,.) (9,.) ∞

∞

∞

Skip-list

!20

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

(1,.) (5,.) (9,.) (14,.)

(1,.) (9,.) ∞

∞

∞

Find 10

p n

-∞ nr

n

Skip-list

!21

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

(1,.) (5,.) (9,.) (14,.)

(1,.) (9,.) ∞

∞

∞

-∞sentinel

-∞

-∞

10?

10?

(10,.)Insert 10

Search in a Skip List

!22

search(n, k): node p = n; n = n.next; while (n != null && n.key < k) { p = n; n = n.next; } if( search(p.below, k);

if(n == null) return “Fail”;

search(sentinel, k);

n.key == k) return “answer”; n != null &&

insert(n, k, topinsert node p = n; n = n.next; while (n != null && n.key < k) { p = n; n = n.next; } // Handle duplicate if necessary if(level < h) insertafternode(p, k); insert(p.below, k, topinsert

Insert in a Skip List

!23

if(n == null) return;

topinsert = toss_coins_until_tails(maxtosses);insert(sentinel, k, topinsert

level

, height

, level):

, level-1);

);What if topinsert > height?

What about the below ref?

add below ref

Skip-list: Increase height

!24

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

(1,.) (5,.) (9,.) (14,.)

(1,.) (9,.) ∞

∞

∞

-∞sentinel

-∞

-∞

10?

Insert 10

10?

To Delete:Delete from all levels

Skip-list: Analysis

!25

Prob(level i exists) <= n/2i

Prob(level log n exists) <= n/2log n = n/n

Prob(level klog n exists) <= n/2klog n = n/nk = 1/n(k-1)

height = klog n

⇒

i successive heads in any of n experiments

Expected no. of nodes visited at any level = 2= Expected number of tosses before heads

Skip-list

!26

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

(1,.) (5,.) (9,.) (14,.)

(1,.) (9,.) ∞

∞

∞

-∞sentinel

-∞

-∞

Next to 10Just before 10?

Skip-list: Save Space

!27

(1,.) (3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)

sentinel

May store only references at the upper levels

Skip-list: Array

!28

sentinel esent

(3,.) (5,.) (8,.) (9,.) (11,.) (14,.) (19,.)(1,.)2

Could even store in arrays

1 2 1 1213 0(-∞)3

(+∞)

“Linked” Data structures

!29

class A { B b; C c; int i;}

2001b c

iA a = new A(..)

dsl;k;lk;lkd;lsk;lsk;lskdl;skl;dksl;dks

dklsjkdlsjkljslkdjs dsdkjslkdjs

dsjdkjs djskdjsljdsldkj

kjkjkjkjdkfjdkfjkdjf

dsl;k;lk;lkd;lskdkjkjkjlsk;lskdl;skl;dksl;dksdlkdjfkldjflkdjflkdjsdkjslkdjs

dsjdkjs fdkfjdkjfkdjkdjfkdjfkdjskdjsljdsldkj

} A a;

a

4400b c

i

a

90210b c

i

a

A a2;

“Linked” Data structures

!30

class A { B b; C c; int i; A a; A a2;}

…….b c

i

a a2A a = new A(..)

…….b c

i

a a2

…….b c

i

a a2…….

b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2…….

b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2 …….b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2

…….b c

i

a a2

Doubly Linked List

!31

…….b c

i

a a2…….

b c

i

a a2

At most two nodes refer to any given node, and are reciprocated.

…….b c

i

a a2

Binary Treeclass BinaryNode<T> { BinaryNode left; BinaryNode right; T value; methods etc.}

!32

class BinaryNode<T> { BinaryNode left;

BinaryNode right; T value; methods etc.

}

class BinaryNode<T> { BinaryNode left; BinaryNode right; T value; methods etc.}








Exactly one other node contains reference to any given node.One special node has no other node with reference to it.

root

Binary tree has two references per node

edge

pathnodenode

Quiz! Keeping skip-list towers in arrays saves

space. What else does it save? [1 word]! How? [maximum 10 words]! What deficiency does it suffer from?

[Maximum 3 words]

!33

[email protected]: time, No separate fetch of below node, fixed count


Binary Tree

!34

class BinaryNode<T> { BinaryNode<T> left; BinaryNode<T> right; T value; methods etc.}

class BinaryNode<T> { BinaryNode<T> left;

BinaryNode<T> right; T value; methods etc.

}









8

6 10

7 2092

1 4 15

Root

Ordered Tree

parent

child

leaf

Complete TreeProper Tree

Tree

!35

Not Tree?18

8

6 10

7 2092

1 4 15

Root8

6 10

7 2092

1 4 15

Tree xxRoot = null

Root

Tree

!36

8

6 10

7 2092

1 4 15

Root

Subtree

height

SubtreeLeft

RightLeft

= 3BST:left subtree has lowerright subtree has higher

!37

class BinaryNode<T> { BinaryNode<T> left; BinaryNode<T> right;

T value;}

class BinaryTree<T> { BinaryNode<T> root;

int height() { return height(root); } int height(BinaryNode<T> node) { if(node == null) return -1; return(1 + max( height(node.left), height(node.right)));} int count() { return count(root); } int count(BinaryNode<T> node) { if(node == null) return 0; return(1 + count(node.left) + count(node.right));}

}

Tree Operations

BST

!38

8

6 1

7 292

1 4 1

class BST<T> { BinaryNode<T> root;

BinaryNode<T> find(T v) { return find(root, v); } BinaryNode<T> find(BinaryNode<T> node, T v) { if(node == null) return null; if(node.value == v) return node; BinaryNode n = find(node.left, v); return (n == null)? find(node.right, v) : n;}

class BST<T extends Comparable<T>> { BinaryNode<T> root;

BinaryNode<T> find(T v) { return find(root, v); } BinaryNode<T> find(BinaryNode<T> node, T v) { if(node == null) return null;

int compared = v.compareTo(node.value); if(compared == 0) return node; if(compared < 0) return find(node.left, v); return find(node.right, v);}

BST Operations

!39

8

6 10

7 2092

1 4 15

Root

Insert 5

5

BST Operations

!40

8

6 10

7 2092

1 4 15

Root

Remove 10

5 17

1515

BST Operations

!41

8

6 10

7 92

1 4

Root

Remove 10

5

If Removed node has one child: Promote that child

Otherwise: Transplant up the successor;

Promote successor's child

Successor

!42

8

6 10

7 2092

1 4 15

5

Predecessor? Root

True or False?! The maximum height of a binary tree with n

nodes is n-1! The maximum height of a proper binary tree

with n nodes is 2logn + 1! The minimum height of a complete binary

tree with n nodes, is ceil(log (n+1))! The number of non-leaf nodes in an n-node

tree is always <= n/2

!43

Format: t,f,f,fMail: [email protected]


Traversal

!44

8

6 10

7 2092

1 4 15

Root

class BST<T extends Comparable<T>> { BinaryNode<T> root;

void iterate(Consumer<T> op) { iterate(root, op); } void iterate(BinaryNode<T> node, Consumer<T> op) { if(node == null) return; op.accept(node.value); iterate(node.left, op); iterate(node.right, op);}…

} Pre-order Traversal

Traversal

!45

8

6 10

7 2092

1 4 15

Rootinterface SortablePair<K extends Comparable<K>, V> { K key(); V value();}

interface Position2way<T> { Position2way<T> left(); Position2way<T> right(); T value();}

class BST<PT extends SortablePair> { Position2way<PT> root; int numPositions; void iterate(Consumer<PT> op) { iterate(root, op); } void iterate(Position2way<PT> node, Consumer<PT> op) { if(node == null) return; iterate(node.left(), op); op.accept(node.value()); iterate(node.right(), op); }…} In-order Traversal

tree.iterate(t -> {System.out.println(t.value());});

interface SortablePair<K extends Comparable<K>, V> { K key(); V value();}

Traversal

!46

8

6 10

7 2092

1 4 15

Root

!46

class BST<PT extends SortablePair> { Position2way<PT> root; int numPositions; void iterate(Consumer<PT> op) { iterate(root, op); } void iterate(Position2way<PT> node, Consumer<PT> op) { if(node == null) return; iterate(node.left(), op); op.accept(node.value()); iterate(node.right(), op); }…}

Traversal

!47

8

6 10

7 2092

1 4 15

Root

!47

opclass BST<PT extends SortablePair> { Position2way<PT> root; int numPositions; void iterate(Consumer<PT> op) { iterate(root, op); } void iterate(Position2way<PT> node, Consumer<PT> op) { if(node == null) return; iterate(node.left(), op); op.accept(node.value()); iterate(node.right(), op); }…}

Euler’s Tour

Depth First Traversal

if(node.left() != null) iterate(node.left(), op);op.accept(node.value());if(node.left() != null) iterate(node.right(), op);

if(root != null) iterate(root, op);

Traversal

!48

8

6 10

7 2092

1 4 15

Root

861 2 4 7 109 15 20!48

opclass BST<PT extends SortablePair> { Position2way<PT> root; int numPositions; void iterate(Consumer<PT> op) { iterate(root, op); } void iterate(Position2way<PT> node, Consumer<PT> op) { if(node == null) return; iterate(node.left(), op); op.accept(node.value()); iterate(node.right(), op); }…}

In-order Traversal

Breadth-first Traversal

!49

8

6 10

7 2092

1 4 15

Root

86 10 2 7 9 20 1 4 15

q.insert(root)while(q.hasNext()) { x = q.next(); q.insert(x.left); q.insert(x.right); op.accept(x);}

Queue

Depth-first Traversal

!50

8

6 10

7 2092

1 4 15

Root

8 610 27 14

stack.push(root)while(stack.hasany()) { x = stack.pop(); stack.push(x.right); stack.push(x.left); op.accept(x);}

QueueStack

True or False?! The number of internal nodes in a proper

binary tree is less than n/2.! The Euler’s tour of a binary tree enters each

node 3 times. ! The height of a complete binary tree with n

nodes, is ceil(log (n+1)) - 1.! In-order traversal of a binary search tree

operates on the keys of the tree in an increasing order.

!51

Format: f,f,f,fMail: [email protected]


Tree Balance

!52

159

20

10

5

1 4

2

8 Root3

10

9

20

15

41

5

2 8

15

9

20

10

4

1

5

2

8

Balanced Tree

!53

Height balanced: Height of left subtree is within ±1 of that of right subtree

Count balanced (Weight balanced): Weight of left subtree is at least # times the right subtree’s

# < .29

AVL Tree

Rotation

!54

5

2

10

10t

10

-1

AVL Tree Height

!55

159

20

10

5

1 4

2

8 Root3

2 2

0

00

1 0 1

nmin(h) = nmin(h-1) + nmin(h-2) + 1

nmin(0) = 1 nmin(-1) = 0

nmin(h) > 2 nmin(h-2)

h < ⌊1.44 log(n+2)⌋

n > 2(h+.328)*.694 – 2

Rotation

!56

5

2

10

8 15 20

17h h+2h+1

h/h-1 h/h-1

U

C

G

h+1n0

n1

n2n3

h-1 hh-1/h-2

h+2Imbalance on Insertion

Rotation

!57

5

10

15 20

17h h+2

h

U

C

G

h+1n0

n1

n2n3

h-1 h

h+2

20

17 h+2

10U

C

G

h+1

5h

n0

15h

n1n2

n3

hh-1

h+1

Imbalance on Insertion

Rebalancing AVL Tree

!58

10

20

17

10

17

20

20

10

17

20

17

10

GG

C C

2010

17

U.parent().setchild(m, parent.left==U? LEFT:RIGHT);m.left = lo; m.right = hi;lo.right = n1; hi.left = n2;

m

mm

m

hi

lo

lo

hi

lo

hi

lo

hi

m

lohi

n1 n2

n1

n2

n1 n2n1 n2

n1

n2

U UU

U

h

Rotation

!59

5

2

10

8 15 20

17hh-1 h+1

U

C

G

n0

n1

n2n3

h-1/h-2

h/h-1

h+2Imbalance on Deletion

h-1/h-2

/h-1

Rotation

!60

Imbalance on Deletion

h

5

10

15 20

17

h+2

h-1 h+1U

C

G

n0

n1

n2n3

h-1/h-2

h/h-1

h-1/h-2

20

17 h+2

10U

lo

C

Gh

5h-1

n0

15h/h-1

n1n2

n3

hi

mid

h/h+1

/h+1

h-1

h-1

Rotation

!61


h-1

5

10

15 20

17

h+2

h-1 h+1U

C

G

n0

n1

n2

h

h-1/h-2 h-1/h-2

n2n3

h-1/h-2 h-1/h-2

Case 2

Rotation

!62


h-1

5

10

15 20n3

17

h+2

h-1 h+1U

Cn0

n2

h

G

n1

h-1/h-2 h-1/h-2

17

15 h+2

10U

G

C

5h-1

n0n2

20n1n3

h-1h-1/h-2 h-1/h-2

h h

h+1

Case 2

True or False?! The height of an AVL tree with n nodes is

O(log n)! The difference in the heights of two children

of an AVL tree node is at most 1! The maximum number of nodes requiring re-

structuring when deleting a node in AVL tree with n nodes is log n

!63

Format: t,t,fMail: [email protected]


AVL Updates! On insertion, re-structure the deepest unbalanced

node! C and G lie on the side of the insertion! Reduces the height of the taller side and fixes the

imbalances of all ancestors! On deletion, re-structure the only unbalanced node

! C is on the other side of the deletion! G is the taller child of C

! If both children of C have the same height, choose the Left-left or Right-right configuration for U C G

! Restructure could reduce height causing an imbalance in the parent

! Traverse higher and repeat, until no imbalance remains!64

Common Leaf Level Tree

!65

15

10

20

11

4

1

5

2

7

155

207 1114

10

2

2-3 Tree

2 or 3 children/node

⌊log3(k)⌋ ≤ height ≤ ⌊log2(k)⌋

1 or 2 keys

⌊log3(2n)⌋ ≤ height ≤ ⌊log2(n)⌋For n nodes:

For k keys:

Insert in 2-3 Tree

!66

155

207 111

10

2,

2-3 Tree

4

13

, 14

Insert 12Insert 14

8

9

10

Insert in 2-3 Tree

!67

135

207 11,121

10

2,

2-3 Tree

4

Insert 25

14

25

21,

15,

• Always insert a child with a key‣ At the leaf, the child may be null

• If the insertion causes overflow⇒3 Keys, 4 references‣ Split into two nodes‣ One key, 2 refs each‣ Promote 1 key + 1 node

13 15

11 12

v

x y

• Always insert a child with a key‣ At the leaf, the child may be null

• If the insertion causes overflow⇒3 Keys, 4 references‣ Split into two nodes‣ 1 key, 2 refs each‣ And promote separator

1/2/3 keys per node

Insert in 2-4 Tree

!68

2-4 Tree

30

50

4 5

22 2623

25Works for a-b tree

a > 1

10 15

21 28 49

1 4 7 11 12 14 20

2 5 13

• Start @leaf‣ after replacing deleted key with successor

• Borrow from sibling if it has >1 key‣ Rotate: promote sibling key, demote parent key‣ Graft child

• Otherwise, merge with sibling‣ For parent:‣ demote separator‣ remove 1 reference‣ Fix any underflow

22 2511

Delete in 2-3 Tree

!69

15

1 4 12 14

2 5 13

Delete 10Delete 7

10

7

Delete 20

20

In a-b tree:Merged node must not exceed b childrena-1 children + a children <= b children

21

❌

❌

True/False for a-b tree?! All leaves are at the same depth! Each node has at least b and at most b

children! 2a ≤ b+1! The root is allowed to have fewer than a

children, but no fewer than 2! A node with x children has x-1 keys

!70

Format: t,f,t,t,t/fMail: [email protected]


Binar-izing a 2-4 Tree

!71

28

4921

10 15

21 28 49

10

15 10

15

10

Red Black Tree

Red Black Tree! No red colored node has a red colored child! The number of black colored node on the

path from the root to each null reference is the same

!72

=> height ≤ 2 log (n)

• Null references are assumed black• Root is always black

2-4 tree height: lg4n = .5 lg2n to lg2n [Up to 2 comparisons /node] 2-3 tre height: lg3n = .63 lg2n to lg2n [Up to 2 comparisons /node] AVL tree height between lg2n to 1.4lg2n [Up to 1 comparison /node]

Binar-ized 2-4 Tree

!73

10

15

2

513 21

28

49

1 4 7

11

1214 20 25 5030

10 15

21 28 49

30 50251 4 7 11 1214 20

2 5 13

49

Red-Black Tree Insert

!74

10

15

2

5

1 4 7

13

12

11

14

21

28

20 25 5030

24

22 23

Insert 24Insert 22Insert 23

Insert @null in node n

Red uncle

Color new node RedBlack parent ⇒ DoneRed parent ⇒ Black Uncle: Restructure

Red Uncle: Recolor + Recur up

R-B Tree Deletion

!75

10

15

5

2

1 4 7

13

12

11

14

21 49

28

20 25 5030

Delete 10

24

Expunge node n with @null in child.

Delete 20

Simple if n is red.Also simple if n ’s child is red.

S

P

d

d

S

n1

SP

n2

S

P

n1

n2

Complex Delete Cases

!76

S

P P

d

d-1

n1 n2d d

d

d

n2n1

d

d-1

d

d

S

P

d

n2n1

d-1

d-1

-1

ns

n1 n2 d d

-1-1

Count +1 if color == Red: "if singlechild().color == Red: singlechild.setcolor(black) "putblack();

putblack(): if root(): " if color == Red: setcolor(Black) " s = sibling(); if s.color == Red: Restructure s.child; s.swapcolor(); # if nephew() == !Red: // i.e., null or Black sibling().setcolor(Red); parent.putblack(); else: Restructure sibling.child() "

49

Red-Black Tree

!77

10

15

2

5

1 4 7

13

12

11

14

28

20 25 5030

Expunge node with @null in node n

24

Delete 20

26

21d = 3

Red Sibling ⇒ Restructure P,S,n

P←Red; S←Black

Black Sibling ⇒ No red nephew: Recolor S,P; Fix P. Red nephew: Restructure P,S,n; Done.

data structures algorithmssubodh/courses/col106/lectures/... · 2019-09-20 · basic data...

Documents