introduction to algorithm analysis and design

7/24/2019 Introduction to algorithm analysis and design

1/18

MODULE 1

ALGORITHM

An algorithm is a finite set of instructions each of which may reuire one or more

o!erations that" if followe#" accom!lishes a !articular tas$%All algorithms must satisfy the foll

criteria&

1% Input& 'ero or more uantities are e(ternally)% Output & at least one uantity is !ro#uce#

*% Definiteness :each instruction is clear an# unam+iguous%

,% Finiteness &if we trace out the instructions of an algorithm" then for all cases"the

algorithm terminates after a fi(e# no%of ste!s%

-% Effectiveness &e.ery instruction must +e .ery +asic so that it can +e carrie# out in!rinci!le"+y a !erson using only !encil an# !a!er% It also must +e feasi+le%

/onsi#ering criteria * #irections such as 0a## or 2 to (3 or com!ute -45 are not

!ermitte#%/onsi#ering criteria - #irections such as 0a## or 2 to (3 or com!ute

-45 are not !ermitte#%

/ATEGORIE6 O7 ALGORITHM

7un#amentally algorithms can +e #i.i#e# into two categories%

o Iterati.e 8re!etiti.e9 algorithm

o Recursi.e algorithms%

Iterative algorithmsty!ically use loo!s an# con#itional statements%

Recursive algorithmsuse divide and conquerstrategy to +rea$ #own a large !ro+lem

into small chun$s an# se!arately a!!ly algorithm to each chun$%


2/18

:ER7ORMA;/E A;AL


3/18

If we $new the characteristics of the com!iler to +e use# "we coul# !rocee# to #etermine

the num+er of a##itions"su+tractions"multi!lications"#i.isions"com!ares"loa#s"an# so on%

e coul# o+tain an e(!ression fortP(n) of the form

tP(n)>

caADD8n9?

cs6UF8n9?

cmMUL8n9?

cdDI8n9?%

where n #enotes the instance characteristics an#ca "

cs " cm "

cd

"etc"res!ecti.ely #enote the time nee#e# for an

a##ition"su+traction"multi!lication"#i.ision etc"an# ADD"6UF"DI"MUL"etc are the

functions%

A !rogram ste! is a meaningful segment of a !rogram that has an e(ecution time that is

in#e!en#ent of the instance characteristics%

7or eg" comments an# #eclarati.e statements such as int" Jinclu#e" etc" count as 'ero

ste!s"an assignment statement which #oes not in.ol.e any calls to other !rograms is

counte# as one ste!" in an iterati.e statement such as for" while" we consi#er the ste!

counts only for the control !art of the statement%

The control !arts for for ha.e the foll forms&

7or8i>Ke(!rCiK>Ke(!r1Ci??9

The first e(ecution of the control !art of the forhas a ste! count eual to the sum of the

counts for Ke(!ran# Ke(!r1%Remaining e(ecutions of the for ha.e a ste! count of one %

*%AMORTIED /OM:LE=ITaiSCElse if8aiSKmin9

Min>aiSC

;ow the +est case occurs when the elements are in increasing or#er%

The num+er of element com!arisons is nB1%

The worst case occurs when the elements are in #ecreasing or#er%

In this case the num+er of element com!arisons is )8nB19%


7/18

A #i.i#eBan# Qconuer algorithm for this !ro+lem woul# !rocee# as follows&

Let : > 8n"aiS""aS9 #enote an instance of the !ro+lem%

Here n is the num+er of elements in the list aiS"%"aS%

In this case" the ma(imum an# minimum are aiS if n>1%

If n>)"the !ro+lem can +e sol.e# +y ma$ing one com!arison%

If the list has more than ) elements" : has to +e #i.i#e# into smaller instances%

7or e(am!le"we might #i.i#e : into the ) instancesP1=( n2, a [1 ] , .a[n2 ]) P2 >

8nBn4)"an4)?1S"%anS9%

After ha.ing #i.i#e# : into ) smaller su+ !ro+lems "we can sol.e them +y recursi.ely

in.o$ing the same #i.i#eBan# Qconuer algo%

How can we com+ine the solutions forP1 an#

P2 to o+tain a solution for :

If MA=8:9 an# MI;8:9 are the ma(imum an# minimum of the elements in :"then

MA=8:9 is the larger of MA=8 P1 9 an# MA=8

P2 9% Also"MI;8:9 is the smaller of

MI;8 P1 9 an# MI;8

P2 9%

Ma(Min is a recursi.e function that fin#s the ma(imum an# minimum of the set of

elements @a8i9"a8i?19"%%"a891%

The situation of set si'es one8i>9 an# two 8i>B19 are han#le# se!arately%

7or sets containing more than ) elements"the mi#!oint is #etermine#8ust as in +inary

search9an# ) new su+!ro+lems are generate#%

hen the ma(ima an# minima of these su+!ro+lems are #etermine#"the ) ma(ima

arecom!are# an# the two minima are com!are# to achie.e the solution for the entire set%

6u!!ose we simulate the function Ma(Min on the following V elements&


8/18

A goo# way of $ee!ing trac$ of recursi.e calls is to +uil# a tree +y a##ing a no#e each

time a new call is ma#e%

Each no#e has , items of information & I ""ma("min%

E(amining the figure "we see that the root no#e contains 1 an# V as the .alues i an#

corres!on#ing to the initial call to Ma(Min%

This e(ecution !ro#uces ) new calls to Ma(Min"where i an# ha.e the .alues 1"- an#

"V res!ecti.ely"an# thus s!lit the set into ) su+sets of a!!ro(imately the same si'e%

hat is the no%of element com!arisons nee#e# foe Ma(Min

If T8n9 re!resents this num+er "then the resulting recurrence relation is


9/18

T8n9 >

T(n/2)+T(n /2)+2>2

1n=2

{0n=1

7igure% Trees of recursi.e calls of Ma(Min

RA;DOMIED ALGORITHM & I;7ORMAL DE6/RI:TIO;6

A ran#omi'e# algo is one that ma$es use of a a ran#omi'er8such as a ran#om num+er

generator9%

6ome of the #ecisions ma#e in the algorithm #e!en# on the out!ut of the ran#omi'er%

6ince the out!ut of any ran#omi'er might #iffer run to run"the out!ut of a ran#omi'e#

algo coul# also #iffer from run to run for the same in!ut% The e(ecution time of a ran#omi'e# algorithm coul# also .ary from run to run for the

same in!ut%

Ran#omi'e# algorithms can +e categori'e# into ) classes& the first algo that always

!ro#uce the same out!ut for the same in!ut%These are calle# Las egas algorithms%

The e(ecution time of a Las egas algo #e!en#s on the out!ut of the ran#omi'er%


10/18

If we are luc$y"the algo might terminate fast"an# if not"it might run for a longer !erio# of

time%

The secon# is algorithms whose out!uts might #iffer from run to run%these are calle#

Monte /arlo algorithms%

/onsi#er any !ro+lem for which there are only ) !ossi+le answers"say


11/18

In contrast there is a sim!le an# elegant ran#omi'e# Las egas algorithm that ta$es only

~O (log n ) time%

It ran#omly !ic$s ) array elements an# chec$s if they come from ) #iff cells an# ha.e the

same .alue%

If they #o the re!eate# element has +een foun#%

If not" this +asic ste! of sam!ling is re!eate# as many times as it ta$es to i#entify the

re!eate# element%

In this algo" the sam!ling !erforme# is with re!etitionsC i%e%"the first an# secon# elements

are ran#omly !ic$e# from out of the n elements%

There is a !ro+a+ility that the same array element is !ic$e# each time%if we ust chec$ for

the euality of the ) elements !ic$e#"our answer might +e incorrect%

Therefore it is essential to ma$e sure that the two array in#ices !ic$e# are #ifferent an#

the ) array cells contain the same .alue%

:RIMALIT< /HE/WI;G8eg& Monte carlo ty!e9

Any integer greater than one is sai# to +e a !rime if its only #i.isors are 1 an# the integer

itself%

Fy con.ention" we ta$e 1 to +e a non!rime%


12/18

Then )"*"-"2"11"an# 1* are the first si( !rimes%

Gi.en an integer n"the !ro+lem of #eci#ing whether n is a !rime is $nown as !rimality

testing%

If a num+er n is com!osite "it must ha.e a #i.isor n

%

/onsi#er each num+er l in the inter.al)" n S an# chec$ whether l #i.i#es n%

If none of these num+ers #i.i#es n"then n is !rimeC otherwise it is com!osite%

e can #e.ise a Monte /arlo ran#omi'e# algorithm for !rimality testing that runs in time

O88n

log 2 %

The out!ut of this algorithm is correct with high !ro+a+ility%

If the in!ut is !rime the algorithm ne.er gi.es an incorrect answer%

If the in!ut num+er is com!osite "then there is a small !ro+a+ility that the answer may +eincorrect%

Algorithms of this $in# are sai# to ha.e oneBsi#e# error%

7ermats theorem suggests the following algorithm for !rimality testing& ran#omly

choose an a Kn an# chec$ whether an1 18mo# n9 %

If 7ermats euation is not satisfie# "n is com!osite%

If the euation is satisfie#"we try some more ran#om a$s%

If on each atrie#" 7ermats euation is satisfie#" we out!ut X n is !rimeCotherwise we

out!ut Xn is com!osite% Unfortunately"in our algorithm e.en if n is com!osite"7ermats euation may +e satisfie#

#e!en#ing on the achosen%

Moreo.er" there are com!osite num+ers for which e.ery a that is less than an# relati.ely

!rime to n will satisfy 7ermats euation%The num+ers -1 an# 115- are e(am!les%

The mo#ifie# !rimality testing algorithm is also $nown as Miller Ra+ins algorithm%

Miller Ra+ins algorithm will ne.er gi.e an incorrect answer if the in!ut is !rime"since

7ermats euation will always +e satisfie#%

If n is com!osite"the algorithm #etect the com!ositeness of n if either the ran#omly

chosen a lea#s to the #isco.ery of a nontri.ial suare root of 1 or it .iolates 7ermat3s

euation%

6TRA66E;6 MATRI= MULTI:LI/ATIO;

6u!!ose "we nee# to com!ute the matri( multi!lication of ) matrices A Y F of or#er n%


13/18

Let the resultant matri( +e /%i%e%"/>A%F 8where A"F"/ are nZn matrices9%

The normal matri( multi!lication !roce#ure is

Ci >

!=1

n

Ai!. "! 8where i""$ .aries from 1 to n 9

The running time of this !roce#ure is P8 n3 9%

6trassens shows how to re#uce this com!le(ity%8using #i.i#e an# conuer techniue%9

e wish to com!ute the !ro#uct />A%F where /"A"F are nZn matrices%

Assuming that n is an e(act !ower of )"we #i.i#e each nZn matrices into four n4)Zn4)

matrices as follows&

[r st u] > [a #c d] [e gf h ] 7rom a+o.e we get"

r > ae ? +f s > ag ? +h

t > ce ? #f

u > cg ? #h

Each of these four euations s!ecifies two multi!lications of n4)Zn4) matrices an# the

a##ition of their n4)Zn4) !ro#ucts%

Using these euations to #efine a straightBforwar# #i.i#eBan#Bconuer strategy"we #eri.e

the following recurrence for the time T8n9 to multi!ly two n4)Zn4) matrices%

T8n9>NT8n4)9?P8 n2

9

This recurrence euation has the solution

T8n9> P8 n3

9

6o this metho# is no faster than the or#inary one%

6trassen #isco.ere# a #ifferent recursi.e a!!roach that reuires only 2 recursi.e

multi!lications of n4)Zn4) matrices an# P8 n2

9 scalar a##itions an# su+tractions

yiel#ing the recurrence%

T8n9>2T8n4)9? P8 n2

9

> P8 nlog7 9

> O8 n2.81

9

6trassens metho# has , ste!s&

Di.i#e the in!ut matrices A an# F into n4)Zn4) su+matrices%


14/18

Using P8 n2

9 scalar a##itions an# su+tractions"com!ute 1,n4)Zn4) matrices%

A1 , "1 , A2 , "2 , . , A7 , "7

Recursi.ely com!ute the se.en matri( !ro#uctsPi=Ai . "i ,fori=1,2, .,7

/om!ute the #esire# su+matrices r"s"t"u of the resultant matri( / +y a##ing or4an#

su+tracting .arious com+inations ofPi matrices using only P8 n

2

9 scalar

a##itions an# su+tractions%

Let us guess that each matri( !ro#uctPi can +e written in the form

Pi=Ai . "i

e guess that each !ro#uct is com!ute# +y a##ing or su+tracting some of the su+matrices

of A" a##ing or su+tracting some of the su+matrices of F"an# then multi!lying the two

results together%

r >P1+P4P5+P7

s >P3+P5

t >P2+P4

u >P1+P3P2+P6

whereP1 (a+d )(e+h)P2=(c+d )e

P3=a ( fh )P4=d (ge)

P5=(a+# ) h P6=(ca )(e+f)

P7=(#d )(g+h)


15/18

These 2 su+matri( !ro#ucts can thus +e use# to com!ute the !ro#uct />A%F" which

com!letes the #escri!tion of 6trassens metho#%

FI;AR< 6EAR/H

Letai "1i n" +e a list of elements that are sorte# in non#ecreasing or#er%

/onsi#er the !ro+lem of #etermining whether a gi.en element ( is !resent in the list%

If ( is !resent" we are to #etermine a .alue such thata > (%

If ( is not in the list" then is to +e set to 'ero%

Let : > 8n"

ai , , al"("9 #enote an ar+itrary instance of this search !ro+lem8n is the

num+er of elements in the list" ai , , al is the list of elements an# ( is the element

searche# for9%

Di.i#eBan#Bconuer can +e use# to sol.e this !ro+lem%

Let 6mall8:9+e true if n > 1%

In this case"68:9will ta$e the .alue i if ( >ai " otherwise it will ta$e the .alue 5%

Then g8l9 > P819%

If : has more than one element"it can +e #i.i#e# 8or re#uce#9into a new su+!ro+lem asfollows%

:ic$ an in#e( 8in the range i "l S an# com!are ( witha$ %

There are three !ossi+ilities&

819 %a$ & In this case the !ro+lem : is imme#iately sol.e#%

8)9 & a' & In this case ( has to +e searche# for only in the su+list

ai , ai+1 , .. ,a$1 %Therefore" : re#uces to 8 Q i " ai , . , a$1 "(9

8*9 a'& In this case the su+list to +e searche# isa$+1, . al %

: re#uces to 8l B "a$+1 , .. , al "(9%

Any gi.en !ro+lem : gets #i.i#e# 8re#uce#9into one new su+!ro+lem%

This #i.ision ta$es only P819 time%


16/18

After a com!arison witha$ " the instance remaining to +e sol.e# 8if any9 can +e

sol.e# +y using this #i.i#eBan#Bconuer scheme again%

If is always chosen such thata$ is the mi##le element8that is" > 8n ? l94)9" then the

resulting search algorithm is $nown as +inary search%

;ote that the answer to the new su+!ro+lem is also the answer to the original !ro+lem:C

there is no nee# for any com+ining%

%Does Fin6earch terminate

e o+ser.e that low an# high are integer .aria+les such that each time through the loo!

either ( is foun# or low is increase# +y at least one or high is #ecrease# +y at least one%

Thus we ha.e two seuences of integers a!!roaching each other an# e.entually low

+ecomes greater than high an# causes termination in a finite num+er of ste!s if ( is not

!resent% /om!uting time of +inary search is


17/18


18/18

6ELE/TIO; O7 A OR6TB /A6E O:TIMAL ALGORITHM

introduction to algorithm analysis and design

Documents