abstract data types stack, queue amortized analysis

1

Abstract Data TypesStack, Queue

Amortized analysis

Cormen: Ch 10, 17 (11, 18)

2

ADT is an interface

• It defines– the type of the data stored– operations, what each operation does

(not how)– parameters of each operation

3

ADT

Application

Implementation of theData structure

חוזה בין מתכנת האפליקציה

ומיישם מבנה הנתונים

ממשק

4

ADT : how to work with

• Advantage: – Application programming can be done

INDEPENDENTLY of implementation– If care about efficiency (complexity):

• BE CAREFUL!!• Using the ADT the “wrong way” might

become quite inefficient!!!

5

Example: Stacks

• Push(x,S) : Insert element x into S• Pop(S) : Delete the last (time) element

inserted into S• Empty?(S): Return yes if S is empty• Top(S): Return the last element inserted into

S• Size(S)• Make-stack()

6

The Stack Data Abstraction

push

push

push

7

The Stack Data Abstraction

push

push

push

pop

push

Last in,First out.

8

The Stack: Applications??1. REAL-LIFE:

1. Rifle

2. Some document handling?

Last in,First out.

1. Computers/ Communications:

1. In some applications – LIFO preferred on FIFO

2. Program control + algorithm == a KEY

9

• Evaluate an expression in postfix or Reverse Polish Notation

Infix Postfix

(2+ 3) * 5 2 3 + 5 *

( (5 * (7 / 3) ) – (2 * 7) ) 5 7 3 / * 2 7 *-

A stack application

Application

Implementation



ממשק

10

2 3 + 5 *

A stack application

23

Application

Implementation



ממשק

11

2 3 + 5 *

A stack application

55

12

2 3 + 5 *

A stack application

25

13

Pseudo = כאילו

Combination of: 1. Programming (like) commands2. “free style English”

Allows to conveniently express and algorithm.

See Cormen (one page) for his language constructs. No need for FORMAL syntax. (can deviate

Cormen)

A Word on Pseudo-code

14

S ← make-stack()while ( not eof ) do B ← read the next data;

if B is an operand then push(B,S) else X ← pop(S)

Y ← pop(S) Z ← Apply the operation B on X and Y push(Z,S)return(top(S))

Pseudo-code

15

Implementation

• We will be interested in algorithms to implement the ADT..

• And their efficiency..

Application

Implementation



ממשק

16

Using an array

12 1 3A

A[0] A[1]

t

The stack is represented by the array A and variable t

A[2] A[N-1]

1213

17

Using an array

12 1 3A

A[0] A[1]

t

make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1

The stack is represented by the array A and variable t

A[2] A[N-1]

18

Operations

12 1 3A

t

size(S): return (t+1)

empty?(S): return (t < 0)

top(S): if empty?(S) then error else return A[t]

A[N-1]A[0] A[1]

A[2]

19

Pop

12 1 3A

t

pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)

A[N-1]A[0] A[1]

A[2]

pop(S)

20

Pop

12 1 3A

t

A[N-1]A[0] A[1]

A[2]

pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)

pop(S)

21

Push

12 1 3A

t

push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x

A[N-1]A[0] A[1]

A[2]

push(5,S)

22

Push

12 1 5A

t

push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x

A[N-1]A[0] A[1]

A[2]

push(5,S)

23

Implementation with lists

12 1 5

top

size=3

x.element

x.nextx

24


12 1 5

top

make-stack(): top ← null size ← 0

size=3

25

Operations

12 1 5

top

size(S): return (size)

empty?(S): return (top = null)

top(S): if empty?(S) then error else return top.element

size=3

26

Pop

12 1 5

top

pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)

pop(S)

size=3

27

Pop

12 1 5

top


pop(S)

size=2

28

Garbage collection

1 5

top


pop(S)

size=2

29

Push

1 5

topsize=2

push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)

30

Push

1 5

topsize=2


5

31

Push

1 5

topsize=2


5

32

Push

1 5

topsize=3


5

33

Analysis

• Bound the running time of an operation on the worst-case

• As a function of the “size”, n, of the data structure

• Example: T(n) < 4n+7• Too detailed, most cases we are just

interested in the order of growth

34

Why order of growth

• Precise cost can change from computer to computer

• From Programmer to programmer

• Anyhow MOORE may go down by factor of 1.5 -2 next year

35

Big-O

) ו- כך ש:c - קיים ) ( ( ))T n O f n0n

0)()( nnnfcnT

דוגמא:2

( ) ( 1)T n n

)(4)1(222nOnn

36

Big-O

))(()( ngOnf

cg(n)

f(n)

n0

38

The running time of our stack and queue operations

• Each operation takes O(1) time

39

Stacks via extendable arrays

• Array implementation difficulties:• Pick large N: wasted space • Pick small N: stack is limited • Solution:

– Pick moderate N, and:

– When the array is full we will double its size (N) (DOUBLING)

40

Push

push(x,S):

/* N is size of array*/ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x

12 1A

t

A[N-1]A[0] A[1]

41

Push

push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x

12 1 3 3A 4 5 7 3 2 8 1

A[N-1]A[0] A[1]

t

push(5,S)

42

Push


12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

push(5,S)

43

Push


12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

12 1 3 3 4 5 7 3 2 8 1

t

push(5,S)

44

Push


t

A[2N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

push(5,S)

45

Push

push(x,S):

/* N is array size */ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x

5

A[2N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

push(5,S)

46

Analysis

• An operation may take O(n) worst case time !

• But that cannot happen often..

47

Amortized Analysis

• How long it takes to do m operations in the worst case ?

• Well, O(nm)

• Yes, but can it really take that long ?

50

x x

51

x x

x x x

52

x x

x x x x

53

x x

x x x x

x x x x x

54

x x

x x x x

x x x x x x

55

x x

x x x x

x x x x x x x

56

x x

x x x x

x x x x x x x x

57

x x

x x x x

x x x x x x x x

x x x x x x x x x

58

x x

x x x x

x x x x x x x x

x x x x x x x x x x

59

x x

x x x x

x x x x x x x x

x x x x x x x x x x x

60

x x x x x x x x

x x x x x x x x x

After N-1 pushes that cost 1 we will have a push that costs 2N+1

Let N be the size of the array we just copied (half the size of the new array)

Total time 3N for N pushes, 3 per push

N

x x x x

62

Theorem: A sequence of m operations on a stack takes O(m) time

We proved:

63

Amortized Analysis (The bank’s view)

• You come to do an operation with a bunch of tokens (amortized cost of the operation)

Cs

upto here 20/10/2009

64

Operation

65

Operation

You have to pay for each unit of work by a token (from your POCKET)

67

Operation

If you have more tokens than work you put the remaining tokens in the bank

68

Operation

69

Operation

If we have more work than tokens we pay with token from the bank

70

Operation

If we have more work than tokens we pay with tokens from the bank

71

Amortized Analysis (The bank’s view)

• Suppose we prove that the bank is never in a deficit

• The total work is no larger than the total number of tokens

• 3*m in our example if m is the number of operations

73

“Easy” push

• Each push comes with 3 tokens (pocket)

• If it’s an easy push then we pay with one token for the single unit of work, and put two in the bank.

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

74

5

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

“Easy” push• Imagine that the tokens in the bank

are placed on the items of the stack• We put one token on the item just

inserted and another token on an arbitrary item without a token

75

5 6

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

76

5 6 6 7 1 10 4 67 2 5 7

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

77

“hard” push

• Tokens from the bank “pay” for copying the array into the larger array

5 6 6 7 1 10 4 67 2 5 7

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

78

• Then you pay a token and put two in the bank as an “easy” push

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 A

t

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4

t

“hard” push

79

Need to prove• The balance is never negative:• When we get to an expensive push there

are enough tokens to pay for copying• By induction: prove that after the i-th push

following a copying there are 2i tokens in the bank

• N/2 easy pushes before the hard push N tokens are at bank at the hard push.

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4

t

Starting with empty

array

80

How many tokens we spent ?

• Each operation spent 3 tokens

81

Summary

• So the total # of tokens is 3m

THM: A sequence of m operations takes O(m) time !! (we finished the proof)

• Each operation takes O(1) amortized time

82

Theorem: A sequence of m operations on a stack takes O(m) time

proof (3) .

• We formalize the bank as a potential function

83

Amortized(op) = actual(op) +

Define

Let be a potential function

(can represent the amount in the bank)

Out of pocket = on Operation + on (bank)

84

Amortized(op1) = actual(op1) + 1- 0


Amortized(opm) = actual(opm) + m- (m-1)

……

+

iAmortized(opi) = iactual(opi) + m- 0

iAmortized(opi) iactual(opi) if m- 0 0

We need to find that gives a nontrivial statement

Out of pocket = on Operation + on (bank)

85

Example: Our extendable arrays

Define: the potential of the stack

2( 1) 2( 1) 0( )

0

t N if t NS

Otherwise

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4

t

86

Amortized(push) = actual(push) + =

1 + 2(t’+1) – N - (2(t+1) – N) = 1+2(t’-t) = 3

Amortized(push) = N + 1 + =

N + 1 + (2 - N) = 3

Can we gain (make money)? No! #pop ≤ # push amortz>= 2

With copying (hard):

Without copying (easy):

[ N + 1 + (after -

before)]Amortized(pop) = 1 + =

1 + 2(t-1) – N - (2t – N) = -1

Cs

upto here

87



Amortized(opn) = actual(opn) + n- (n-1)

……

+

iAmortized(opi) = iactual(opi) + n- 0

iAmortized(opi) iactual(opi) if n- 0 03m ≥

88

Summary

• So the total # of tokens is 3m

THM: A sequence of m operations starting from an empty stack takes O(m) time !!

• Each operations take O(1) amortized time

89

Queue

• Inject(x,Q) : Insert last element x into Q• Pop(Q) : Delete the first element in Q• Empty?(Q): Return yes if Q is empty• Front(Q): Return the first element in Q• Size(Q)• Make-queue()

90

The Queue Data Abstraction

inject

inject

91

The Queue Data Abstraction

inject

inject

inject

inject

popFirst in,First out (FIFO).

92

Using an array

12 1 4 2A 5

A[0] A[1]

t

pop(Q)

A[2] A[N-1]

93

Using an array

1 4 2 5A

A[0] A[1]

t

pop(Q)

A[2] A[N-1]

94

Using an array

1 4 2 5A

A[0] A[1]

t

A[2] A[N-1]

This would be inefficient if we insist that elements span a prefix of the array

USE a MOVING ARRAY

95

Using an array

12 1 4 2A 5

A[0] A[1]

r

A[2] A[N-1]

f

A

A[0] A[1]

r

A[2]

f

Empty queue f=r

96

Using an array

12 1 4 2A 5

A[0] A[1]

r

pop(Q)

A[2] A[N-1]

f

97

Using an array

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)

98

Using an array

1 4 2A 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)

99

Using an array

1 4 2A 5 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)

100

Using an array

2A 5 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

101

Using an array

A 5 5 5 5

A[0] A[1]

r

A[2] A[N-1]

f

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

Fall off the edge?

102

Make the array “circular”

5 5 A 5 5

A[0] A[1]

r

A[2] A[N-1]

f

Pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

103

Operations

empty?(Q): return (f = r)

top(Q): if empty?(Q) then error else return A[f]

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

104

Operations

size(Q): if (r >= f) then return (r-f) else return N-(f-r)

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

105

Operations

size(Q): if (r >= f) then return (r-f) else return N-(f-r)

5 5 A 5 5

A[0] A[1]

r

A[2] A[N-1]

f

106

Pop

pop(Q)

1 4 2A 5

A[0] A[1]

A[2]

f r

pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)

107

Pop

pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)

pop(Q)

1 4 2A 5

A[0] A[1]

A[2]

f r

108

Inject

inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N

inject(5,Q)

4 2A 5

A[0] A[1]

A[2]

f r

Why?

f=r empty

109

Inject

inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N

inject(5,Q)

4 2A 5 5

A[0] A[1]

A[2]

f r

Or could do

doubling

110


12 1 5

headsize=3

tail

inject(4,Q)

111


12 1 5

headsize=3

tail

inject(4,Q)

4

112


12 1 5

headsize=3

tail

inject(4,Q)

4

Complete the details by yourself

113

Continue in DS2….Slide 26

114

Double ended queue (deque)

• Push(x,D) : Insert x as the first in D• Pop(D) : Delete the first element of D• Inject(x,D): Insert x as the last in D• Eject(D): Delete the last element of D• Size(D)• Empty?(D)• Make-deque()

115

Implementation with doubly linked lists

5

head

size=2

tail

13

x.next

x.element

x.prev

x

116

Empty list

head

size=0

tail

We use two sentinels here to make the code simpler

117

Push

push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1

5

head

size=1

tail

118


5

head

size=1

tail

push(4,D)

4

119


5

head

size=1

tail

push(4,D)

4

120


5

head

size=1

tail

push(4,D)

4

121


5

head

size=2

tail

push(4,D)

4

122

Implementations of the other operations are similar

• Try by yourself

abstract data types stack, queue amortized analysis

Documents