1

Week 3: Minimum Spanning Trees

•Definition of MST•Generic MST algorithm•Kruskal's algorithm•Prim's algorithm

2

Problem: Laying Telephone Wire

Central office

3

Wiring: Naïve Approach

Central office

Expensive!

4

Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

5

Definition of MST• Input: connected, and undirected Graph G=(V,E)

– Each edge (u,v) in E carries a weight w(u,v)• Also referred to as cost (length of edge) to connect u and v.

• Output: A subset T of E that connects all of the vertices in V and whose total weight is minimized.

• Since the total weight is minimized, the subset T must be acyclic (no circuit). – Thus, T is a tree. We call it a spanning tree.

• The problem of determining the tree T is called the minimum-spanning-tree problem.

What is a Minimum-Cost Spanning Tree• Example: The graph

Has 16 spanning trees. Some are:

The graph has two minimum-cost spanning trees, each with a cost of 6:

All 16 of its Spanning TreesComplete Graph

Minimum Spanning Trees

The Minimum Spanning Tree for a given graph is the Spanning Tree of minimum cost for that graph.

5

7

2

1

3

4

2

1

3

Complete Graph Minimum Spanning Tree

9

Application of MST: an example

• In the design of electronic circuitry, it is often necessary to make a set of pins electrically equivalent by wiring them together.

• To interconnect n pins, we can use n-1 wires, each connecting two pins.

• We want to minimize the total length of the wires.• Minimum Spanning Trees can be used to model

this problem.

10

Electronic Circuits:

11

Electronic Circuits:

12

Here is an example of a connected graph and its minimum spanning tree:

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

Notice that the tree is not unique: replacing (b,c) with (a,h) yields another spanning tree with the same minimum weight.

13

Growing a MST(Generic Algorithm)

• Set A is always a subset of some minimum spanning tree. This property is called the invariant Property.

• An edge (u, v) is a safe edge for A if adding the edge to A does not destroy the invariant.

• A safe edge is just the CORRECT edge to choose to add to T.

GENERIC_MST(G,w)1 A:={}2 while A does not form a spanning tree do3 find an edge (u,v) that is safe for A4 A:=A∪{(u,v)}5 return A

14

V-S↓

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

S↑ ↑ S

↓ V-S

• This figure shows a cut (S,V-S) of the graph.

• The edge (d,c) is the unique light edge crossing the cut.

15

Safe edge

We need some definitions and a theorem.• A cut (S,V-S) of an undirected graph G=(V,E) is a

partition of V.• An edge crosses the cut (S,V-S) if one of its

endpoints is in S and the other is in V-S.• An edge is a light edge crossing a cut if its weight

is the minimum of any edge crossing the cut.

16

Theorem 1Given:

– G=(V, E) be a connected, undirected graph with a real-valued weight function w defined on E.

– A be a subset of E that is included in some minimum spanning tree for G.

– Let (S,V-S) be any cut of G such that for any edge e=(u, v) in A, {u, v} S or {u, v} (V-S).

– Let (x, y) be a light edge crossing (S,V-S).Then, edge (x, y) is safe for A.Proof: Let Topt be a minimum spanning tree. (Blue) A --a subset of Topt and (x, y)-- a light edge crossing (S, V-S)• If (x, y) is NOT safe, then (x, y) is not in T.

– (See the red edge in Figure)• There MUST be another edge (x’, y’) in Topt crossing (S, V-S).(Green)• We can replace (x’, y’) in Topt with (x, y) and get another tree T’opt • Since (x, y) is light , T’opt is also optimal. 1

2

11 1

1 1

1

17

CorollaryGiven: Let G=(V, E) be a connected, undirected graph with a real-valued weight function w defined on E.

– Let A be a subset of E that is included in some minimum spanning tree for G.

– Let C be a connected component (tree) in the forest GA=(V, A).– Let (u, v) be a light edge (shortest among all edges connecting C

with others components) connecting C to some other component in GA.

• Then, edge (u,v) is safe for A. (For Kruskal’s algorithm)

18

The algorithms of Kruskal and Prim• The two algorithms are elaborations of the generic

algorithm.• They each use a specific rule to determine a safe

edge in line 3 of GENERIC_MST.• In Kruskal's algorithm,

– The set A is a forest.– The safe edge added to A is always a least-weight edge

in the graph that connects two distinct components.• In Prim's algorithm,

– The set A forms a single tree.– The safe edge added to A is always a least-weight edge

connecting the tree to a vertex not in the tree.

19

Kruskal’s Algorithm

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Complete Graph

1

4

2

5

2

5

4

3

4

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

A AB D

B B

B

C D

J C

C

E

F

D

D H

J E G

F FG I

G GI J

H J JI

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Sort Edges (in reality they are placed in a priority queue - not sorted - but sorting them

makes the algorithm easier to visualize)

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Cycle

Don’t Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Cycle

Don’t Add Edge

2

5

2

5

4

3

4

4

10

1

6

3

3

2

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

B

B

D

J

C

C

E

F

D

D H

J

E G

F

F

G

I

G

G

I

J

H J

JI

1A D

4B C

4A B

Add Edge

4

1

2

2 1

3

32

4

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Minimum Spanning Tree Complete Graph

35

Kruskal's algorithm(basic part)1 (Sort the edges in an increasing order)2 A:={}3 while E is not empty do {3 take an edge (u, v) that is shortest in E and delete it from E4 if u and v are in different components then add (u, v) to A }Note: each time a shortest edge in E is considered.

36

Kruskal's algorithmMST_KRUSKAL(G,w)1 A:={}2 for each vertex v in V[G]3 do MAKE_SET(v)4 sort the edges of E by nondecreasing weight w5 for each edge (u,v) in E, in order by

nondecreasing weight6 do if FIND_SET(u) != FIND_SET(v)7 then A:=A {(u,v)}∪8 UNION(u,v)9 return A

37

Disjoint-Set • Keep a collection of sets S1, S2, .., Sk,

– Each Si is a set, e,g, S1={v1, v2, v8}.• Three operations

– Make-Set(x)-creates a new set whose only member is x.– Union(x, y) –unites the sets that contain x and y, say, Sx

and Sy, into a new set that is the union of the two sets.– Find-Set(x)-returns a pointer to the representative of

the set containing x.– Each operation takes O(log n) time.

38

• Our implementation uses a disjoint-set data structure to maintain several disjoint sets of elements.

• Each set contains the vertices in a tree of the current forest.

• The operation FIND_SET(u) returns a representative element from the set that contains u.

• Thus, we can determine whether two vertices u and v belong to the same tree by testing whether FIND_SET(u) equals FIND_SET(v).

• The combining of trees is accomplished by the UNION procedure.

• Running time O(|E| lg (|E|)).

39

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

The execution of Kruskal's algorithm (Moderate part)

• The edges are considered by the algorithm in sorted order by weight.

• The edge under consideration at each step is shown with a red weight number.

40

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

41

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

42

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

43

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

44

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

45

Prim’s Algorithm

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Complete Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

Old Graph New Graph

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

4

1

2

2 1

3

32

4

A

B C

D

E F

G

HI

J

Complete Graph Minimum Spanning Tree

4

1

2 3

2 1

3

5

3

4

2

5 6

4

4

10

A

B C

D

E F

G

HI

J

58

Prim's algorithm(basic part)

MST_PRIM(G,w,r)1. A={}2. S:={r} (r is an arbitrary node in V)3. Q=V-{r}; 4. while Q is not empty do {5 take an edge (u, v) such that (1) u S and v Q (v S ) and (u, v) is the shortest edge satisfying (1)

6 add (u, v) to A, add v to S and delete v from Q }

59

Prim's algorithmMST_PRIM(G,w,r)1 for each u in Q do2 key[u]:=∞3 parent[u]:=NIL4 key[r]:=0; parent[r]=NIL;5 QV[Q]6 while Q!={} do7 u:=EXTRACT_MIN(Q); if parent[u]Nil print (u,

parent[u])8 for each v in Adj[u] do9 if v in Q and w(u,v)<key[v]10 then parent[v]:=u11 key[v]:=w(u,v)

60

priority queue Q: three important operations– EXTRACT-MIN(Q) –takes log (n) times if

there are n items in Q.– Insert an element (key u) into Q- takes log(n)

time.– Update(u, k): update the key value of element u

into k. It takes Olog (n) time for adaptable priority queue (heap).

61

• Grow the minimum spanning tree from the root vertex r.

• Q is a priority queue, holding all vertices that are not in the tree now.

• key[v] is the minimum weight of any edge connecting v to a vertex in the tree.

• parent[v] names the parent of v in the tree.• When the algorithm terminates, Q is empty; the

minimum spanning tree A for G is thus A={(v,parent[v]):v∈V-{r}}.

• Running time: O(||E||lg |V|).

62

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

The execution of Prim's algorithm(basic part)

the root vertex

63

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

64

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

65

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

66

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

Bottleneck spanning tree: A spanning tree of G whose largest edge weight is minimum over all spanning trees of G. The value of the bottleneck spanning tree is the weight of the

maximum-weight edge in T.

Theorem: A minimum spanning tree is also a bottleneck spanning tree. (Challenge problem)