prims greedy strategy

November 9, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L16.1

Introduction to Algorithms6.046J/18.401J

LECTURE 16Greedy Algorithms (and

Graphs) Graph representation Minimum spanning trees Optimal substructure Greedy choice Prims greedy MST

algorithmProf. Charles E. Leiserson


Graphs (review)Definition. A directed graph (digraph)G = (V, E) is an ordered pair consisting of a set V of vertices (singular: vertex), a set E V V of edges.In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices.In either case, we have |E | = O(V 2). Moreover, if G is connected, then |E | |V | 1, which implies that lg |E | = (lgV). (Review CLRS, Appendix B.)


Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, , n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) E,0 if (i, j) E.


Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, , n}, is the matrix A[1 . . n, 1 . . n]given by

A[i, j] = 1 if (i, j) E,0 if (i, j) E.

22 11

33 44

A 1 2 3 41234

0 1 1 00 0 1 00 0 0 00 0 1 0

(V 2) storage denserepresentation.


Adjacency-list representationAn adjacency list of a vertex v V is the list Adj[v]of vertices adjacent to v.

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

22 11

33 44



Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

22 11

33 44

For undirected graphs, |Adj[v] | = degree(v).For digraphs, |Adj[v] | = out-degree(v).



Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

22 11

33 44

For undirected graphs, |Adj[v] | = degree(v).For digraphs, |Adj[v] | = out-degree(v).Handshaking Lemma: vV = 2 |E | for undirected graphs adjacency lists use (V + E) storage a sparse representation (for either type of graph).


Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E R. For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)


Minimum spanning trees

Input: A connected, undirected graph G = (V, E)with weight function w : E R. For simplicity, assume that all edge weights are

distinct. (CLRS covers the general case.)

=Tvu

vuwTw),(

),()( .

Output: A spanning tree T a tree that connects all vertices of minimum weight:


Example of MST

6 125

14

3

8

10

15

9

7


Optimal substructureMST T:

(Other edges of Gare not shown.)


Optimal substructure

u

vRemove any edge (u, v) T.


MST T:


Optimal substructure

u

vRemove any edge (u, v) T.

MST T: (Other edges of Gare not shown.)


u

vRemove any edge (u, v) T. Remove any edge (u, v) T. Then, T is partitioned into two subtrees T1 and T2.

T1T2

u

v




u

vRemove any edge (u, v) T. Remove any edge (u, v) T. Then, T is partitioned into two subtrees T1 and T2.

T1T2

u

v



Theorem. The subtree T1 is an MST of G1 = (V1, E1), the subgraph of G induced by the vertices of T1:

V1 = vertices of T1,E1 = { (x, y) E : x, y V1 }.

Similarly for T2.


Proof of optimal substructure

w(T) = w(u, v) + w(T1) + w(T2).Proof. Cut and paste:

If T1were a lower-weight spanning tree than T1 for G1, then T = {(u, v)} T1 T2 would be a lower-weight spanning tree than T for G.





Do we also have overlapping subproblems?Yes.





Great, then dynamic programming may work!Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.

Do we also have overlapping subproblems?Yes.


Hallmark for greedyalgorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.


Hallmark for greedyalgorithms

Greedy-choice propertyA locally optimal choice

is globally optimal.

Theorem. Let T be the MST of G = (V, E), and let A V. Suppose that (u, v) E is the least-weight edge connecting A to V A. Then, (u, v) T.


Proof of theoremProof. Suppose (u, v) T. Cut and paste.

A V A

u

v

(u, v) = least-weight edge connecting A to V A

T:



A V A

u

T:

(u, v) = least-weight edge connecting A to V A

v

Consider the unique simple path from u to v in T.



A V A

u(u, v) = least-weight edge connecting A to V A

vT:

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V A.



A V A

u(u, v) = least-weight edge connecting A to V A

vT :

Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V A.A lighter-weight spanning tree than T results.


Prims algorithmIDEA: Maintain V A as a priority queue Q. Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in A.Q Vkey[v] for all v Vkey[s] 0 for some arbitrary s Vwhile Q

do u EXTRACT-MIN(Q)for each v Adj[u]

do if v Q and w(u, v) < key[v]then key[v] w(u, v) DECREASE-KEY

[v] uAt the end, {(v, [v])} forms the MST.


Example of Prims algorithm

A V A

00

6 125

14

3

8

10

15

9

7



A V A

00

6 125

14

3

8

10

15

9

7



A V A

77

00

1010

1515

6 125

14

3

8

10

15

9

7



A V A

1212

55 77

00

1010

99

1515

6 125

14

3

8

10

15

9

7



A V A

66

55 77

1414 00

88

99

1515

6 125

14

3

8

10

15

9

7



A V A

66

55 77

33 00

88

99

1515

6 125

14

3

8

10

15

9

7


Analysis of PrimQ Vkey[v] for all v Vkey[s] 0 for some arbitrary s Vwhile Q


do if v Q and w(u, v) < key[v]then key[v] w(u, v)

[v] u





[v] u

(V)total





[v] u

(V)total

|V |times





[v] u

degree(u)times

|V |times

(V)total


Analysis of Prim

Handshaking Lemma (E) implicit DECREASE-KEYs.

Q Vkey[v] for all v Vkey[s] 0 for some arbitrary s Vwhile Q



[v] u

degree(u)times

|V |times

(V)total


Analysis of Prim

Handshaking Lemma (E) implicit DECREASE-KEYs.

Q Vkey[v] for all v Vkey[s] 0 for some arbitrary s Vwhile Q



[v] u

degree(u)times

|V |times

(V)total

Time = (V)TEXTRACT-MIN + (E)TDECREASE-KEY


Analysis of Prim (continued)

Time = (V)TEXTRACT-MIN + (E)TDECREASE-KEY



Time = (V)TEXTRACT-MIN + (E)TDECREASE-KEYTotalQ T TDECREASE-KEYEXTRACT-MIN




array O(V) O(1) O(V2)




array O(V) O(1) O(V2)binary heap O(lg V) O(lg V) O(E lg V)




array O(V) O(1) O(V2)binary heap O(lg V) O(lg V) O(E lg V)

Fibonacci heap

O(lg V)amortized

O(1)amortized

O(E + V lg V)worst case


MST algorithms

Kruskals algorithm (see CLRS): Uses the disjoint-set data structure (Lecture 10). Running time = O(E lg V).


MST algorithms

Kruskals algorithm (see CLRS): Uses the disjoint-set data structure (Lecture 10). Running time = O(E lg V).

Best to date: Karger, Klein, and Tarjan [1993]. Randomized algorithm. O(V + E) expected time.

Introduction to Algorithms6.046J/18.401JGraphs (review)Adjacency-matrix representationAdjacency-matrix representationAdjacency-list representationAdjacency-list representationAdjacency-list representationMinimum spanning treesMinimum spanning treesExample of MSTExample of MSTOptimal substructureOptimal substructureOptimal substructureOptimal substructureOptimal substructureProof of optimal substructureProof of optimal substructureProof of optimal substructureHallmark for greedy algorithmsHallmark for greedy algorithmsProof of theoremProof of theoremProof of theoremProof of theoremPrims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmExample of Prims algorithmAnalysis of PrimAnalysis of PrimAnalysis of PrimAnalysis of PrimAnalysis of PrimAnalysis of PrimAnalysis of Prim (continued)Analysis of Prim (continued)Analysis of Prim (continued)Analysis of Prim (continued)Analysis of Prim (continued)MST algorithmsMST algorithms

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 150 /GrayImageDepth -1 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 300 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects true /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /False

/DetectCurves 0.100000 /EmbedOpenType false /ParseICCProfilesInComments true /PreserveDICMYKValues true /PreserveFlatness true /CropColorImages true /ColorImageMinResolution 150 /ColorImageMinResolutionPolicy /OK /ColorImageMinDownsampleDepth 1 /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /GrayImageMinDownsampleDepth 2 /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /CheckCompliance [ /None ] /PDFXOutputConditionIdentifier () /Description >>> setdistillerparams> setpagedevice

prims greedy strategy

Documents