lecture 7: minimum spanning trees and prim's algorithm
TRANSCRIPT
Lecture 7: Minimum Spanning Trees andPrim’s Algorithm
CLRS Chapter 23
Outline of this Lecture
� Spanning trees and minimum spanning trees.
� The minimum spanning tree (MST) problem.
� The generic algorithm for MST problem.
� Prim’s algorithm for the MST problem.
– The algorithm
– Correctness
– Implementation + Running Time
1
Spanning Trees
Spanning Trees: A subgraph�
of a undirected graph� � ������ is a spanning tree of
�if it is a tree and
contains every vertex of�
.
Example:
� �� �
� �� � � �� � � �� �� �� � � �� �� �� �
� �� � � �� � � �� �
� �� �
� �� � � �� �� �� �
� �� � � �� �� �� �� �� �
� �� �
� �� �
� �� � � � � � � �� � � �����
������ � � � � � � �����
� � �� � � � � � � �� � � ������ � ����� � � �
a b
cd
e
a b
cd
e
a b
cd
e
a b
cd
e
Graph spanning tree 1
spanning tree 2 spanning tree 3
2
Spanning Trees
Theorem: Every connected graph has a spanningtree.
Question: Why is this true?
Question: Given a connected graph�
, how can youfind a spanning tree of
�?
3
Weighted Graphs
Weighted Graphs: A weighted graph is a graph, inwhich each edge has a weight (some real number).
Weight of a Graph: The sum of the weights of alledges.
Example:
� �� �
� �� � � �� � � �� �� �� � � �� �� �� �
� �� � � �� � � �� �
� �� �
� �� � � �� �� �� �
� �� � � �� �� �� �� �� �
� �� �
� �� �
� �� � � � � � � �� � � �����
������ � � � � � � �����
� � �� � � � � � � �� � � ������ � ����� � � �
a b
cd
e
a b
cd
e
a b
cd
e
a b
cd
e
10
9
7 32
2310 32
239
7 7
9
32
23
3223
10
weighted graph
Tree 2, w=71 Tree 3, w=72
Tree 1. w=74
Minimum spanning tree
4
Minimum Spanning Trees
A Minimum Spanning Tree in an undirected connectedweighted graph is a spanning tree of minimum weight(among all spanning trees).
Example:
� �� �
� �� � � �� � � �� �� �� � � �� �� �� �
� �� � � �� � � �� �
� �� �
� �� � � �� �� �� �
� �� � � �� �� �� �� �� �
� �� �
� �� �
� �� � � � � � � �� � � �����
������ � � � � � � �����
� � �� � � � � � � �� � � ������ � ����� � � �
a b
cd
e
a b
cd
e
a b
cd
e
a b
cd
e
10
9
7 32
2310 32
239
7 7
9
32
23
3223
10
weighted graph
Tree 2, w=71 Tree 3, w=72
Tree 1. w=74
Minimum spanning tree
5
Minimum Spanning Trees
Remark: The minimum spanning tree may not beunique. However, if the weights of all the edges arepairwise distinct, it is indeed unique (we won’t provethis now).
Example:
1
2
6724
1
2
6724
MST1 MST2weighted graph
1
2 2
100
6724
6
Minimum Spanning Tree Problem
MST Problem: Given a connected weighted undi-rected graph
�, design an algorithm that outputs a
minimum spanning tree (MST) of�
.
Question: What is most intuitive way to solve?
Generic approach: A tree is an acyclic graph.The idea is to start with an empty graph and try to addedges one at a time, always making sure that what isbuilt remains acyclic. And if we are sure every time theresulting graph always is a subset of some minimumspanning tree, we are done.
7
Generic Algorithm for MST problem
Let � be a set of edges such that � � �, where�
is a MST. An edge��� ���
is a safe edge for � , if� � � ��� ��� � is also a subset of some MST.
If at each step, we can find a safe edge��� ���
, wecan ’grow’ a MST. This leads to the following genericapproach:
Generic-MST(G, w)Let A=EMPTY;while A does not form a spanning treefind an edge (u, v) that is safe for Aadd (u, v) to A
return A
How can we find a safe edge?
8
How to find a safe edge
We first give some definitions. Let� � ��� � �
be aconnected and undirected graph. We define:
Cut A cut��� ��� � ��
of G is a partition of V.
Cross An edge��� ��� � �
crosses the cut��� � � �
��if one of its endpoints is in
�, and the other is
in� � �
.
Respect A cut respects a set � of edges if no edgein A crosses the cut.
Light edge An edge is a light edge crossing a cutif its weight is the minimum of any edge crossingthe cut.
9
How to find a safe edge
Lemma
Let� � ��� � �
be a connected, undirected graphwith a real-valued weight function � defined on
�. Let
� be a subset of�
that is included in some minimumspanning tree for
�, let
��� � � � ��be any cut of
�that
respects � , and let��� ���
be a light edge crossing thecut
��� � � � ��. Then, edge
��� ��� is safe for � .
It means that we can find a safe edge by
1. first finding a cut that respects � ,
2. then finding the light edge crossing that cut.
That light edge is a safe edge.
10
Proof
1. Let � � �, where
�is a MST. Suppose
��� ��� ���
.
2. The trick is to construct another MST� �
that con-tains both � and
��� ��� , thereby showing
��� ��� is
a safe edge for � .
11
3. Since�
, and�
are on opposite sides of the cut��� � � � ��, there is at least one edge in
�on the
path from�
to�
that crosses the cut. Let��� ���
besuch edge. Since the cut respects � ,
��� ��� �� � .
Since��� ���
is a light edge crossing the cut, wehave �
��� ��� ����� � �
.
a cut respects A
another MST T’
A
v
u
y
xA
MST T
v
u
y
x
12
4. Add��� ���
to�
, it creates a cycle. By removingan edge from the cycle, it becomes a tree again.In particular, we remove
��� ��� (
�� � ) to make anew tree
� �.
5. The weight of� �
is
�� � � �
�� � �
���� ��� ��
���� ���
��� �
6. Since�
is a MST, we must have �� � �
�� � �
,hence
� �is also a MST.
7. Since � � � ��� � � � is also a subset of� �
(a MST),��� ��� is safe for � .
13
Prim’s Algorithm
The generic algorithm gives us an idea how to ’grow’a MST.
If you read the theorem and the proof carefully, youwill notice that the choice of a cut (and hence thecorresponding light edge) in each iteration is imma-terial. We can select any cut (that respects the se-lected edges) and find the light edge crossing that cutto proceed.
The Prim’s algorithm makes a nature choice of the cutin each iteration – it grows a single tree and adds alight edge in each iteration.
14
Prim’s Algorithm : How to grow a tree
Grow a Tree
� Start by picking any vertex � to be the root of thetree.
� While the tree does not containall vertices in the graphfind shortest edge leaving the treeand add it to the tree .
Running time is� � ��� � � � ��� � ����� � � �
.
15
More Details
Step 0: Choose any element � ; set� � ��� � and
� � �. (Take � as the root of our spanning tree.)
Step 1: Find a lightest edge such that one endpointis in
�and the other is in
� � �. Add this edge to
� and its (other) endpoint to�
.
Step 2: If� � � � �
, then stop & output (minimum)spanning tree
��� � � . Otherwise go to Step 1.
The idea: expand the current tree by adding thelightest (shortest) edge leaving it and its endpoint.
e
2420
r
ab
c
d
26
f
g ir
ab
c
d e
f
g i
8 8
12
1614
new
242026
1614
12
23 23
new edge
1212
h h
16
Prim’s Algorithm
Worked Example
a
b
c
d
e
f
g
4
8
9
8
2
1
9
7
10
5
6
2
b
c
d
e
f
g
4
8
10
8
2
1
7
95
6
2
S={a}
Step 0
aV \ S = {b,c,d,e,f,g}
9
Connected graph
lightest edge = {a,b}
17
Prim’s Algorithm
Prim’s Example – Continued
b
c
d
e
f
g
4
8
9
8
2
1
9
7
10
5
6
2
c
d
e
f
g
8
9
10
8
2
1
7
95
6
2
a
a
Step 1.1
Step 1.1 after
4
S={a}
S={a,b}b
before
V \ S = {b,c,d,e,f,g}
V \ S = {c,d,e,f,g}
lightest edge = {a,b}
lightest edge = {b,d}, {a,c}
A={}
A={{a,b}}
18
Prim’s Algorithm
Prim’s Example – Continued
c
d
e
f
g
4
8
9
8
2
1
9
7
10
5
6
2
c
e
f
g
8
10
8
2
1
7
95
6
2
a
a
4b
beforeb Step 1.2 S={a,b}
Step 1.2 after
S={a,b,d}
d
V \ S = {c,d,e,f,g}
V \ S = {c,e,f,g}9
lightest edge = {b,d}, {a,c}
lightest edge = {d,c}
A={{a,b}}
A={{a,b},{b,d}}
19
Prim’s Algorithm
Prim’s Example – Continued
c
e
f
g
4
8
9
8
2
1
9
7
10
5
6
2
e
f
g
8
9
10
8
2
1
7
95
6
2
a
a
4b
beforeb
d
Step 1.3
Step 1.3 after
S={a,b,d}d
S={a,b,c,d}
V \ S = {c,e,f,g}
V \ S = {e,f,g}
c
lightest edge = {d,c}
lightest edge = {c,f}
A={{a,b},{b,d}}
A={{a,b},{b,d},{c,d}}
20
Prim’s Algorithm
Prim’s Example – Continued
e
f
g
4
8
9
8
2
1
9
7
10
5
6
2
e
g
8
9
10
8
2
1
7
95
6
2
a
a
4b
beforeb
d
d
c
S={a,b,c,d}
V \ S = {e,f,g}
c
S={a,b,c,d,f}V \ S = {e,g}
Step 1.4
Step 1.4 after
f
lightest edge = {c,f}
lightest edge = {f,g}
A={{a,b},{b,d},{c,d}}
A={{a,b},{b,d},{c,d},{c,f}}
21
Prim’s Algorithm
Prim’s Example – Continued
e
g
4
8
9
8
2
1
9
7
10
5
6
2
e
8
9
10
8
2
1
7
95
6
2
a
a
4b
beforeb
d
d
c
c
f
S={a,b,c,d,f}V \ S = {e,g}
f
Step 1.5
Step 1.5 after
S={a,b,c,d,f,g}V \ S = {e}
{f,g}}
g
lightest edge = {f,g}
lightest edge = {f,e}
A={{a,b},{b,d},{c,d},{c,f}}
A={{a,b},{b,d},{c,d},{c,f},
22
Prim’s Algorithm
Prim’s Example – Continued
e4
8
9
8
2
1
9
7
10
5
6
2
8
9
10
8
2
1
7
95
6
2
a
a
4b
beforeb
d
d
c
c
f
f
g
S={a,b,c,d,f,g}V \ S = {e}
{f,g}}
g
Step 1.6
Step 1.6 after
S={a,b,c,d,e,f,g}V \ S = {}
{f,g},{f,e}}
MST completed
e
lightest edge = {f,e}
A={{a,b},{b,d},{c,d},{c,f},
A={{a,b},{b,d},{c,d},{c,f},
23
Recall Idea of Prim’s Algorithm
Step 0: Choose any element � and set� � � ��� and � � �
.(Take � as the root of our spanning tree.)
Step 1: Find a lightest edge such that one endpoint is in�
andthe other is in � � . Add this edge to � and its (other)endpoint to
�.
Step 2: If � � � �, then stop and output the minimum span-
ning tree ��� �� .Otherwise go to Step 1.
Questions:
� Why does this produce a Minimum SpanningTree?
� How does the algorithm find the lightest edge andupdate � efficiently?
� How does the algorithm update�
efficiently?
24
Prim’s Algorithm
Question: How does the algorithm update�
efficiently?
Answer: Color the vertices. Initially all are white.Change the color to black when the vertex is movedto
�. Use color[
�] to store color.
Question: How does the algorithm find the lightestedge and update � efficiently?
Answer:(a) Use a priority queue to find the lightest edge.(b) Use pred[
�] to update � .
25
Reviewing Priority Queues
Priority Queue is a data structure (can be implementedas a heap) which supports the following operations:
insert(� ����� �
):Insert
�with the key value
��� �in � .
u = extractMin():Extract the item with the minimum key value in � .
decreaseKey(� �����
� -��� �
):Decrease
�’s key value to
���� -
��� �.
Remark: Priority Queues can be implemented so thateach operation takes time
� � ���� � � � . See CLRS!
26
Using a Priority Queue to Find the Lightest Edge
Each item of the queue is a triple��� ���
������ ���
,��� ��� ��
,where
��
is a vertex in� � �
,�
��� ��� ��is the weight of the lightest edge
from�
to any vertex in�
, and�
������� ���
is the endpoint of this edge in�
.The array is used to build the MST tree.
r
ab
c
d e
f
g ir
ab
c
d e
f
g i
242026
1614
8
242026
1614
8
1212
23 23
new edgekey[f] = 8, pred[f] = e
1212
key[i] = infinity, pred[i] = nil key[i] = 23, pred[i] = f
After adding the new edgeand vertex f, update the key[v] and pred[v] for each vertex v adjacent to f
key[g] = 16, pred[g] = c
key[h] = 24, pred[h] = b
f has the minimum key
h h
27
Description of Prim’s Algorithm
Remark: � is given by adjacency lists. The vertices in � �are stored in a priority queue with key=value of lightest edge tovertex in
�.
Prim( � ��� � � )�for each � � � initialize� ����� �� � � �
;������� � � �� � �;������ �� � �
; start at root� � ���� �� � � �! ;" �
new PriQueue( � ); put vertices in"
while("
is nonempty) until all vertices in MST�u=
"$#extraxtMin(); lightest edge
for each ( %&� ' �)(*� �� )�if (( ������� � � %! � � �
)&&(� � � � %! ,+ ����� %! ))���-.� %/ � � � � � %! ; new lightest edge"$#
decreaseKey( % � ���-.� %! );� � �0��� %! � � ;�������� � � �� � 1 2��
When the algorithm terminates," � �
and the MST is3 � � � % � � � �0��� %! �$4/%&� � � ��� � #The pred pointers define the MST as an inverted tree
rooted at � .28
Example for Running Prim’s Algorithm
a
b
c
d
e
f
1
2
3
4
5
1
103
4
u
key[u]
pred[u]
a b c d e f
29
Analysis of Prim’s Algorithm
Let� � � � �
and� � ��� �
. The data structure PriQueuesupports the following two operations: (See CLRS)
�� � � � � �
to extract each vertex from the queue.Done once for each vertex
� � � � ���� � ��
�� � � � � �
time to decrease the key value of neigh-boring vertex.Done at most once for each edge
� � � � � � � � ��
Total cost is then
� � � � � � ��� � �
30
Analysis of Prim’s Algorithm – Continued
Prim(G, w, r) { for each (u in V){
key[u] = +infinity; color[u] = white;
}
key[r] = 0; pred[r] = nil; Q = new PriQueue(V);
while (Q. nonempty()){
u = Q.extractMin(); for each (v in adj[u]){
if ((color[v] == white) &(w(u,v) < key[v])
key[v] = w(u, v); {
}}color[u] = black;
}}
1O(log n)1
11
pred[v] = u;
O(log n)
1
O(deg(u) log n)
1
[O(log n) + O(deg(u) log n)]u in V
11
2n
n
Q.decreaseKey(v, key[v]);
31
Analysis of Prim’s Algorithm – Continued
So the overall running time is
� � � ��� � � � � � �
� � �� � � ���� � � � ����� � ��� ����� � �
� � � � � � ���� � � � �
� � ��� � ��� � ��� ��
� � � � � � � � � ���� � � � � � � �� � � � ���� � � � � � � �� � � � ���� � � � � � �� � � ��� � � � ��� � ��� � � � � � �
32