minimum spanning t

8/7/2019 Minimum Spanning t

http://slidepdf.com/reader/full/minimum-spanning-t 1/50

Minimum Spanning TreesMinimum Spanning Trees



11/29/2006 Mr D C Jinwala, Algorithms & Computational Complexity, M Tech(CO), SVNIT, 200607 2

GraphsGraphs -- DefinitionsDefinitions GraphGraph

a set of a set of verticesvertices (nodes) and(nodes) and edgesedges connecting themconnecting them

we saywe say G = ( V, E )G = ( V, E ) wherewhere

V is a set of vertices:V is a set of vertices: V = { vV = { vii }} An edge connects two vertices:An edge connects two vertices: e = ( ve = ( vii ,, vvjj ))

E is a set of edges: E is a set of edges: (v(vii ,, vvjj )) }}

Vertices

Edges




GraphsGraphs -- DefinitionsDefinitions

PathPath –– p of length, k, is a sequence of connectedp of length, k, is a sequence of connected

verticesvertices

p = <vp = <v00,v,v

11,...,,...,vv

kk>> wherewhere (v(v

ii,v,v

i+1i+1)) ∈∈ EE

< i, c, f, g, h >Path of length 5

< a, b >

Path of length 2





CycleCycle

A graph contains noA graph contains no cyclescycles if there isif there is no no pathpath

p = <vp = <v00

,v,v11

,...,,...,vvk k

>> such that such that vv00

== vvk k

< i, c, f, g, i >is a cycle





Spanning TreeSpanning Tree

A A spanning treespanning tree is a set of is a set of |V||V|--11 edges that connect alledges that connect all

the vertices of a graphthe vertices of a graph

The red path connectsall vertices,

so it’s a spanning tree

The red path connectsall vertices,

so it’s a spanning tree




GraphsGraphs -- Minimum Spanning TreeMinimum Spanning Tree Generally there is more than one spanning treeGenerally there is more than one spanning tree

If a costIf a cost ccijij is associated with edgeis associated with edge eeijij=(=(v v ii,v ,v jj))

then thethen the minimum spanning treeminimum spanning tree is the set of edgesis the set of edges EEspanspan such thatsuch that

C=C=ΣΣ((ccijij|for|for allall eeijij belonging tobelonging to EspanEspan)) is a minimumis a minimum

The red tree is the

Min ST

Other ST’s can be

formed ..

• Replace 2 with 7

• Replace 4 with 11




Generic MSTGeneric MST Loop invariantLoop invariant

Prior to each iteration, A is a subset of some MST.Prior to each iteration, A is a subset of some MST.

Add the safe edgeAdd the safe edge

At each step, investigate (At each step, investigate (u,vu,v) that can be added to A without) that can be added to A withoutviolating the constraint.violating the constraint.

GENERICGENERIC--MST(G, w)MST(G, w)

1. A 1. A ←←←←←←←← ΦΦ2. while A does not form a spanning tree2. while A does not form a spanning tree

3.3. do find an edge (do find an edge (u,v u,v ) that is safe for A ) that is safe for A

4. A 4. A ←←←←←←←← A U {(A U {(u,v u,v )})}5. return A 5. return A




GraphsGraphs -- KruskalKruskal’’ss AlgorithmAlgorithm How to find the safe edge ?!!!How to find the safe edge ?!!!

Calculate the minimum spanning treeCalculate the minimum spanning tree

Put all the vertices into single node trees by themselvesPut all the vertices into single node trees by themselves Put all the edges in a priority queuePut all the edges in a priority queue

Repeat until weRepeat until we’ ’ ve constructed a spanning treeve constructed a spanning tree

Extract the cheapest edge Extract the cheapest edge

If it forms a cycle, ignore it,If it forms a cycle, ignore it,

else add it to the forest of treeselse add it to the forest of trees

(it will join two trees into a larger tree)(it will join two trees into a larger tree)

Return the spanning treeReturn the spanning tree




GraphsGraphs -- KruskalKruskal’’ss AlgorithmAlgorithm Calculate the minimum spanning treeCalculate the minimum spanning tree

Put all the vertices into single node trees by themselvesPut all the vertices into single node trees by themselves

Put all the edges in a priority queuePut all the edges in a priority queue

Repeat until weRepeat until we’ ’ ve constructed a spanning treeve constructed a spanning tree

Extract cheapest edge Extract cheapest edge

If it forms a cycle, ignore itIf it forms a cycle, ignore it

else add it to the forest of treeselse add it to the forest of trees(it will join two trees into a larger tree)(it will join two trees into a larger tree)

Return the spanning treeReturn the spanning tree

Note that this algorithm makes no attempt • to be clever

• to make any sophisticated choice of the next edge

• it just tries the cheapest one!




KruskalKruskal’’ss AlgorithmAlgorithm Cycle detectionCycle detection

Uses aUses a UnionUnion--findfind structurestructure

For which we need to understand aFor which we need to understand a partitionpartition of a setof a set

PartitionPartition

A set of sets of elements of a setA set of sets of elements of a set

Every element belongs to one of the subEvery element belongs to one of the sub--setssets

No element belongs to more than one subNo element belongs to more than one sub--setset

Formally:Formally:

Set,Set, S = {S = {ssii }}

Partition(SPartition(S) = { P) = { Pii }, where P}, where Pii = {= {ssii }}

•• ∀∀ ssii∈∈ S,S, ssii ∈∈ PPjj

∀∀ j, k j, k PPjj ∩∩ PPk k == ∅∅

S =S =∪∪ PPjj




KruskalKruskal’’ss AlgorithmAlgorithm PartitionPartition

The elements of each set of a partitionThe elements of each set of a partition

are related by anare related by an equivalence relationequivalence relation

equivalence relationsequivalence relations areare

reflexivereflexive

transitivetransitive

symmetricsymmetric

The sets of a partition areThe sets of a partition are equivalence classesequivalence classes

Each element of the set is related to every other elementEach element of the set is related to every other element

x ~ x

if x ~ y and y ~ z, then x ~ z

if x ~ y, then y ~ x




KruskalKruskal’’ss AlgorithmAlgorithm PartitionsPartitions

In the MST algorithm,In the MST algorithm,

the connected vertices form equivalence classesthe connected vertices form equivalence classes

““Being connectedBeing connected” ” is the equivalence relationis the equivalence relation

Initially, each vertex is in a class by itself Initially, each vertex is in a class by itself

As edges are added,As edges are added,

more vertices become relatedmore vertices become relatedand the equivalence classes growand the equivalence classes grow

Until finally all the vertices are in a single equivalenceUntil finally all the vertices are in a single equivalence

classclass




KruskalKruskal’’ss AlgorithmAlgorithm RepresentativesRepresentatives

One vertex in each class may be chosen as theOne vertex in each class may be chosen as the

representative of that classrepresentative of that class

We arrange the vertices in lists that lead to theWe arrange the vertices in lists that lead to therepresentativerepresentative

This is theThis is the unionunion--findfind structurestructure

Cycle determinationCycle determination




KruskalKruskal’’ss AlgorithmAlgorithm Cycle determinationCycle determination

If two vertices have the same representative,If two vertices have the same representative,

theythey’ ’ re already connected and adding a furtherre already connected and adding a further

connection between them is pointlessconnection between them is pointless Procedure:Procedure:

For each endFor each end--point of the edge that youpoint of the edge that you’ ’ re going to addre going to add

follow the lists and find its representativefollow the lists and find its representative if the two representatives are equal,if the two representatives are equal,

then the edge will form a cyclethen the edge will form a cycle




KruskalKruskal’’ss Algorithm in operationAlgorithm in operation

Each vertex is its

own representative

All the vertices are in

single element trees





The cheapest edge

is h-g

All the vertices are in

single element trees

Add it to the forest,

joining h and g into a

2-element tree




KruskalKruskal’’ss Algorithm in operationAlgorithm in operationThe cheapest edge

is h-g


joining h and g into a

2-element tree

Choose g as its

representative





The next cheapest edge

is c-i


joining c and i into a

2-element tree

Choose c as its

representativeOur forest now has 2 two-element trees

and 5 single vertex ones






is a-b

Add it to the forest,joining a and b into a

2-element tree

Choose b as its

representative

Our forest now has 3 two-element trees

and 4 single vertex ones






is c-f


merging two

2-element trees

Choose the rep of one

as its representative




KruskalKruskal’’ss Algorithm in operationAlgorithm in operationThe next cheapest edge

is g-iThe rep of g is c

∴ g-i forms a cycle

The rep of i is also c

It’s clearly not needed!




KruskalKruskal’’ss Algorithm in operationAlgorithm in operationThe next cheapest edge

is c-dThe rep of c is c

∴ c-d joins two

trees, so we add it

The rep of d is d

.. and keep c as the representative






is h-i

The rep of h is c

∴ h-i forms a cycle,

so we skip it

The rep of i is c






is a-hThe rep of a is b

∴ a-h joins two trees,

and we add it

The rep of h is c




KruskalKruskal’’ss AlgorithmAlgorithmAlgorithm Kruskal(Graph G(V,E),Weight_function w)

A ←←←←Φ

for each vertex v ∈∈∈∈ V[G]do MAKE-SET(v)

Sort the edges of E into nondecreasing order

by weight wfor each edge (u,v) ∈∈∈∈ E,

do if FIND-SET(u)≠≠≠≠ FIND-SET(u)

then A ←←←←

A U {(u,v)}UNION(u,v)

return A




Greedy AlgorithmsGreedy Algorithms At no stage did we attempt to At no stage did we attempt to “ “ look ahead look ahead ” ”

We simply made the naWe simply made the naï ï ve choiceve choice

Choose the cheapest edge!Choose the cheapest edge!

Therefore, MST is an example of aTherefore, MST is an example of a greedygreedy

algorithmalgorithm




Proving MSTProving MST MST Proof MST Proof

““Proof by contradictionProof by contradiction” ” is usually the best approach!is usually the best approach!

Note thatNote that

any edge creating a cycle is not neededany edge creating a cycle is not needed

∴∴ Each edge must join two subEach edge must join two sub--treestrees

Suppose that the next cheapest edge,Suppose that the next cheapest edge, eexx, would join, would join

treestrees TTaa andand TTbb

Suppose that instead of Suppose that instead of eexx we choosewe choose eezz -- a morea more

expensive edge, which joinsexpensive edge, which joins TTaa andand TTcc




Proving MSTProving MST

But we still need to joinBut we still need to join TTbb toto TTaa or some otheror some other

tree to whichtree to which TTaa is connectedis connected

The cheapest way to do this is to addThe cheapest way to do this is to add eexx

So we should have addedSo we should have added eexx instead of instead of eezz

Proving that the greedy approach is correct for Proving that the greedy approach is correct for

MST MST




MSTMST -- Time complexity Time complexity

StepsSteps

InitialiseInitialise forestforest O( |V| )O( |V| )

Sort edgesSort edges O( |E|O( |E|loglog|E| )|E| )

Check edge for cyclesCheck edge for cycles O( |V| )O( |V| ) xx

Number of edgesNumber of edges O( |V| )O( |V| ) O( |V|O( |V|22 ))

TotalTotal O( |V|+|E|O( |V|+|E|loglog|E|+|V||E|+|V|22 ))

SinceSince |E| = O( |V||E| = O( |V|22 )) O( |V|O( |V|22 loglog|V| )|V| )




MSTMST -- Time complexity Time complexity

Thus we would class MST asThus we would class MST as O( nO( n22 loglog n )n )

for a graph withfor a graph with nn verticesvertices

This is anThis is an upper bound upper bound ,,

some improvements on this are known ...some improvements on this are known ...

PrimPrim’ ’ ss AlgorithmAlgorithm can becan be O( |O( |E|+|V|E|+|V|loglog|V |V || ))

usingusing Fibonacci heapsFibonacci heaps

even better variants are known for restricted cases,even better variants are known for restricted cases,such assuch as sparse graphssparse graphs (( |E||E| ≈≈ |V||V| ))




PrimPrim’’ss AlgorithmAlgorithm Follows the natural greedy approachFollows the natural greedy approach

starting with the source vertex to create the spanningstarting with the source vertex to create the spanning

tree,tree,

add an edge to the tree that is attached at exactly oneadd an edge to the tree that is attached at exactly oneend to the tree & has minimum weight among all suchend to the tree & has minimum weight among all such

edges.edges.

Prim's algorithm starts from one vertex and growsPrim's algorithm starts from one vertex and growsthe rest of the tree an edge at a time.the rest of the tree an edge at a time.

As a greedy algorithm, which edge should we pick?As a greedy algorithm, which edge should we pick?




Why isWhy is PrimPrim’’ss algorithm correct?algorithm correct? Theorem: Let G be a connected, weighted graph and letTheorem: Let G be a connected, weighted graph and let

EE’ ’ ⊂⊂⊂⊂⊂⊂⊂⊂E be a subset of the edges in a MST T=(V, EE be a subset of the edges in a MST T=(V, Err). Let V'). Let V'

be the vertices incident with edges in E'. If (x,y) is an edgebe the vertices incident with edges in E'. If (x,y) is an edge

of minimum weight such that xof minimum weight such that x ЄЄ V V ’ ’ and y is not in V', thenand y is not in V', thenE U {x,y} is a subset of a minimum spanning tree.E U {x,y} is a subset of a minimum spanning tree.

Proof:Proof:

If the edge is in T, this is trivial.If the edge is in T, this is trivial.

Suppose (x,y) is not in T Then there must be a path in T from xSuppose (x,y) is not in T Then there must be a path in T from x to yto y

since T is connected.since T is connected.

If (v,w) is the first edge on this path with one edge in V', if If (v,w) is the first edge on this path with one edge in V', if wewe

delete it and replace it with (x, y) we get a spanning tree.delete it and replace it with (x, y) we get a spanning tree.

This tree must have smaller weight than T, since W(v,w)>W(x,y).This tree must have smaller weight than T, since W(v,w)>W(x,y).

Thus T could not have been the MST.Thus T could not have been the MST.




ApproachApproach In Kruskal,In Kruskal,

the selection functionthe selection function

uses a greedy approachuses a greedy approach……..

chooses an edge that in minimum weighted edgechooses an edge that in minimum weighted edge ……. then in. then inincreasing orderincreasing order…………..

does not worry too much about their connection to thedoes not worry too much about their connection to the

previously chosen edgespreviously chosen edges……except cycles.except cycles.

the result is a sort of forest of trees that grows haphazardly athe result is a sort of forest of trees that grows haphazardly andnd

later merge into a tree.later merge into a tree.

While in Prim,While in Prim,

the MST grows in a natural mannerthe MST grows in a natural manner……

staring from an arbitrary root.staring from an arbitrary root.

at each step, a new edge is added to a tree already constructed.at each step, a new edge is added to a tree already constructed.




Algorithm Prim(G=(V, E), weight)Algorithm Prim(G=(V, E), weight){initialization}{initialization}

TT ←←←←←←←← ∅∅∅∅∅∅∅∅

BB ←←←←←←←←

{ an arbitrary memebr of N}{ an arbitrary memebr of N}

while Bwhile B ≠≠≠≠≠≠≠≠ N doN do

find E = {u, v} of minimum find E = {u, v} of minimum

weight such thatweight such thatuu ЄЄ B and v B and v ЄЄ NN\\BB

TT ←←←←←←←← T U {u}T U {u}

BB ←←←←←←←← B U {v}B U {v}

return Treturn T




Kruskal and PrimKruskal and Prim –– OperationalOperational differencesdifferences

Consider the graph shown below:Consider the graph shown below:




RunningRunning KruskalKruskal on aboveon above

Step Edge consi Connected components

Init - {1} {2} {3} {4} {5} {6} {7}

1 {1 2} {1 2} {3} {4} {5} {6} {7}

2 {2 3} {1 2 3} {4} {5} {6} {7}

3 {4 5} {1 2 3} {4 5} {6} {7}

4 {6 7} {1 2 3} {4 5} {6 7}

5 {1 4} {1 2 3 4 5} {6 7}

6 {2 5} Rejected 7 {4 7} {1 2 3 4 5 6 7}




Running Prim on aboveRunning Prim on above

Step {u v} B

Init - {1}

1 {1 2} {1 2}

2 {2 3} {1 2 3}

3 {1 4} {1 2 3 4}4 {4 5} {1 2 3 4 5}

5 {4 7} {1 2 3 4 5 7}

6 {5} rejected 7 {4 7} {1 2 3 4 5 6 7}




AlgorithmAlgorithm Prim(G(V,EPrim(G(V,E), weight[1..n,1..n, r)), weight[1..n,1..n, r)

1.1. for each ufor each u ЄЄ V[G]V[G]

2.2. dodo key[ukey[u]] ←←←←←←←← ∞∞

3.3. ππππππππ[[u]u]←←←←←←←←NILNIL

4.4. key[root]key[root]←←←←←←←←005.5. QQ ←←←←←←←← V[G]V[G] //Q is a priority queue as min//Q is a priority queue as min--heap on keyheap on key

6.6. while Qwhile Q ≠≠≠≠≠≠≠≠ ∅∅∅∅∅∅∅∅

7.7. do udo u ←←←←←←←← EXTRACTEXTRACT--MIN(Q)MIN(Q)8.8. for each v for each v ЄЄ Adj[uAdj[u]]

9.9. do if v do if v ЄЄ QQ and(w(u,v and(w(u,v ) <) < key[v key[v ])])

10.10. thenthen ππππππππ[[v]v]←←←←←←←←uu11.11. key[v]key[v]←←←←←←←←w(v,uw(v,u))






i hi l i h bi hi l i h b




Running this algorithm on aboveRunning this algorithm on aboveStep Vertex

inspected

Edge

inspected

Key[v] Minimum

removed from Q

in Tree V – Q

Init- - ∞ - Nil All vertices

11 - 0 1 {1} V – {1}

21 1-2 key[2]=2 - -do- -do-

31 1-4 key[4]=4 - -do- -do-

32 - - 2 {1,2} V – {1 2}

42 2-3 key[3]=2 - -do- -do

52 2-5 key[5]=4 - -do- -do-

62 2-4 Rejected - -do- -do-

73 - - 3 {1 2 3} V- {1 2 3}

83 3-5 Rejected - -do- -do-

93 3-6 key[6]=6 - -do- -do-

10

4 - - 4 {1 2 3 4} V – {1 2 3 4}

114 4-5 key[5]=3 - -do- -do-

124 4-7 key[7]=4 - -do- -do-

i d f bi d f b




continued from abovecontinued from above

134 4-2 & 4-1 Rejected - -do- -do-

145 - - 5 {1 2 3 4 5} V - …

155 5-4, 5-7, 5-6 Rejected - -do- -do-

167 - - 7 {1 2 3 4 5 7} V - …

177 7-3 Key[6]=3 - -do- -do-

187 7-5, 7-4 Rejected - -do- -do-

19

6 - - 6 {1 2 3 4 5 7 6} NIL

20Since Q is null now no further iterations



Clustering Problem:Clustering Problem:Application of MSTApplication of MST

Th Cl t i P blTh Cl t i P bl




The Clustering ProblemThe Clustering Problem arises when a collection of objects are to bearises when a collection of objects are to be

classified or organized into coherent groups.classified or organized into coherent groups.

it is natural to look for how similar or dissimilarit is natural to look for how similar or dissimilar

each pair of objects iseach pair of objects is define a distance functiondefine a distance function

objects at a larger distance from each other are lessobjects at a larger distance from each other are lesssimilar to each other.similar to each other.

the distance may not be in terms of physical distancethe distance may not be in terms of physical distance

distance may take more abstract meaningdistance may take more abstract meaning

e.g.e.g.

distance between two speciesdistance between two species -- to be the no of years in theto be the no of years in thecourse of the evolutioncourse of the evolution

distance between two imagesdistance between two images –– number of correspondingnumber of correspondingpixels at which their intensity values differ by at least somepixels at which their intensity values differ by at least somethresholdthreshold

Th Cl t i P bl (Th Cl t i P bl ( tdtd))




The Clustering Problem (The Clustering Problem (contdcontd)) Given a distance function on the objectsGiven a distance function on the objects

divide the objects into groups so thatdivide the objects into groups so that

objects within the same group are closeobjects within the same group are close

objects within the different groups are far apart.objects within the different groups are far apart.

FormallyFormally

given a set U of n objects, labeledgiven a set U of n objects, labeled pp11,p,p22,p,p33,p,p44……ppnn for each pairfor each pair ppii andand ppjj, we have a numerical distance, we have a numerical distanced(pd(pii,p,pjj)) withwith d(pd(pii,p,pjj)=0)=0 andand d(pd(pii,p,pjj)>0)>0..

alsoalso d(pd(pii,p,pjj)=)=d(pd(pjj,p,pii))

given a parameter k, a k given a parameter k, a k --clustering of U is a partition of clustering of U is a partition of U into k nonempty sets CU into k nonempty sets C11, C, C22, C, C33, C, C44…….C.Ck k ..

spacing of a k spacing of a k --clustering is defined as the minimumclustering is defined as the minimum

distance between any two points lying in different clusters.distance between any two points lying in different clusters.

Th Cl t i P bl (Th Cl t i P bl ( tdtd))




The Clustering Problem (The Clustering Problem (contdcontd)) Given that points in different clusters should be farGiven that points in different clusters should be far

apart from each other,apart from each other,

a natural goal is to seek a k a natural goal is to seek a k --clustering with theclustering with the

maximum possible spacing.maximum possible spacing. How many k How many k --clustering of a set u can be there ?clustering of a set u can be there ?

exponentially manyexponentially many

finding out then one with maximum spacing.finding out then one with maximum spacing. How ?How ?

Al ith A hAlgorithm Approach




Algorithm ApproachAlgorithm Approach consider growing a graph on the vertex set U.consider growing a graph on the vertex set U.

connected components become the clusters.connected components become the clusters.

bring the nearby points into the same cluster.bring the nearby points into the same cluster.

draw an edge between the closest pair of pointsdraw an edge between the closest pair of points

add edges between pairs of points in order of increasingadd edges between pairs of points in order of increasing

distancedistance d(pd(pii,p,pjj).).

if pif pii andand ppjj belong to the same cluster, do not add thatbelong to the same cluster, do not add thatedge.edge.

single link clusteringsingle link clustering

a special case of agglomerative clusteringa special case of agglomerative clustering

How is this related toHow is this related to KruskalKruskal’ ’ ss MST.MST.

Algorithm Approach (Algorithm Approach (contdcontd))




Algorithm Approach (Algorithm Approach (contdcontd)) Follow the same method asFollow the same method as KruskalKruskal’ ’ ss algorithmalgorithm

but stop when obtaining k but stop when obtaining k --clustering.clustering.

How to stop at having k connected components ?How to stop at having k connected components ?

stop just before it adds the last k stop just before it adds the last k –– 1 edges.1 edges.

RunRun KruskalKruskal’ ’ ss MST on the given graph fully andMST on the given graph fully and

then delete the k then delete the k --1 most expensive edges1 most expensive edges

define the resulting structure to be k connecteddefine the resulting structure to be k connected

components.components.

an example of 3an example of 3--clusteringclustering

minimum spanning t

Documents