dag and minimum spanning trees

T l i l S tTopological SortMinimum Spanning Treesp g

M. Shoaib Farooq 1

Directed Acyclic Graphsy p

● A directed acyclic graph or DAG is a directedA directed acyclic graph or DAG is a directed graph with no directed cycles:

M. Shoaib Farooq 2

DFS and DAGs

● Argue that a directed graph G is acyclic if aArgue that a directed graph G is acyclic if a DFS of G yields no back edges:■ Forward: if G is acyclic will be no back edges■ Forward: if G is acyclic, will be no back edges

○ Trivial: a back edge implies a cycle

■ Backward: if no back edges, G is acyclic■ Backward: if no back edges, G is acyclic○ Argue contrapositive: G has a cycle ⇒ ∃ a back edge

Let v be the vertex on the cycle first discovered, and u be the d f h lpredecessor of v on the cycle

When v discovered, whole cycle is whiteMust visit everything reachable from v before returning from

M. Shoaib Farooq 3

DFS-Visit()So path from u→v is yellow→yellow, thus (u, v) is a back edge

Topological Sortp g

● Topological sort of a DAG:Topological sort of a DAG:■ Linear ordering of all vertices in graph G such that

vertex u comes before vertex v if edge (u, v) ∈ Gg ( , )■ TS can be viewed as an ordering of its vertices

along a horizontal line so that all directed edges go g g gfrom left to right.

● Example Given a set of jobs, courses, etc., with i it t i t t t th j b i d th tprerequisite constraints, output the jobs in an order that

does not violate any of the prerequisites.l ld l i d d

M. Shoaib Farooq 4

● Real-world example: getting dressed

Existence of Topological Sortp gLemma G can be topologically sorted iff it has no cycle, that

i iff it i d (di t d li h)is, iff it is a dag (directed acyclic graph).

Proof ⇒ If G has a cycle, then it cannot be topologically sorted.y p g y

a b

⇐ If G has no cycle, then it can be topologically sorted.

Constructive proof: An algorithm that sorts any dag.

M. Shoaib Farooq 5

Topological Sort of Digraphp g g p

Ordering < over V(G) such that u < v whenever (u, v) ∈ E(G).

a b Some topological sorts:

c d1. a, c, e, b, d, g, f2. a, b, c, d, g, f, ec

e f g

g f3. b, d, g, a, c, f, e

e f g

M. Shoaib Farooq 6

Getting Dressedg

UndeShorts 11/16 Socks 17/18UndeShorts 11/16

Shoes 13/14Pants 12/15

Watch 9/10

Belt 6/7

Shirt 1/8

Tie 2/5

Jacket 3/4Jacket 3/4

M. Shoaib Farooq 7

Getting Dressed

M. Shoaib Farooq 8

Topological Sort AlgorithmTopological Sort Algorithm

M. Shoaib Farooq 9

Topological Sort Algorithmp g g

Initialize a global queue L ← ⟨ ⟩ within DFS(G)

Add a line to DFS-visit

DFS(G)for each vertex u ∈ V[G]

color[u] = WHITE;Π[u]= NIl

time = 0;for each vertex u ∈ V[G]

if (color[u] == WHITE)if (color[u] == WHITE)DFS_Visit(u);

L ← insert(v, L) // insert v in the front of L

M. Shoaib Farooq 10

Topological Sort Algorithmp g g

● Depth-first search takes O(V+E) time and itDepth first search takes O(V E) time and it takes O(1) time to insert each of the |V| vertices onto the front of the link list.ve t ces o to t e o t o t e st.

Ti O(V+E)● Time: O(V+E)

M. Shoaib Farooq 11

Minimum Spanning Tree (MST)p g ( )Problem Select edges in a connected and undirected graph to

form a tree that connects all the vertices (spanning tree).minimize the total edge weight of the spanning tree.

da

27

1f

d

b

25

452

1f b g7

1 34

M. Shoaib Farooq 12

c e4Total weight of tree edges: 14

Applications of MSTpp

Telephone wiring (use as little wire as possible)p g ( p )

Electronic circuit board design

Cancer imaging (MSTs describe arrangements of nucleiin skin cells)

Biomedical image analysis (detect actin fibers in cell images)

Astronomy (find clusters of quasars and Seyfert galaxies)

Finding road networks in satellite and aerial imagery

M. Shoaib Farooq 13

Finding road networks in satellite and aerial imagery

Minimum Spanning Tree

M. Shoaib Farooq 14

Minimum Spanning Treep g

● Problem: given a connected, undirected,Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weightedges t at e t e tota we g t

6 45 9

14

5

2

9

102

15

M. Shoaib Farooq 15

3 8


● The phrase “ minimum spanning tree” is a shortened p p gform of the phrase “minimum-weight spanning tree”.

● All spanning trees have exactly |v| -1 edges.● MSTs satisfy the optimal substructure property: an

optimal tree is composed of optimal subtrees

M. Shoaib Farooq 16

Free TreeMi i S i T (T )Minimum Spanning Tree (Terms)

● A free tree is a tree with no vertex designated as the groot vertex.

● A free tree with n vertices has exactly n- 1 edges. ● There exists a unique path between any two vertices

of a free tree.● Adding any edge to a free tree creates a unique cycle.

● Breaking any edge on this cycle restores the free tree.

M. Shoaib Farooq 17

Viable SetMi i S i T (T )Minimum Spanning Tree (Terms)

● A subset A subset of E is viable if 1. A is a subset of edges of some MST 2. A cannot contains a cycle . ca ot co ta s a cyc e

M. Shoaib Farooq 18

Safe EdgeMi i S i T (T )Minimum Spanning Tree (Terms)

● A subset A subset of E is viable if ● A is a subset of edges of some MST. ● An edge (u, v) subset of { E- A} is safe if edge (u, v) subset o { } s safe● A [ {(u, v)} is viable. ● In other words the choice (u v) is a safe choice to● In other words, the choice (u, v) is a safe choice to

add so that A can still be extended to form a MST.OROR

● A edge is safe if it will not create a cycle in MST

M. Shoaib Farooq 19

CUTMi i S i T (T )Minimum Spanning Tree (Terms)

● A cut (S, V- S) is just a partition of vertices into two ( , ) j pdisjoint subsets.

● An edge (u, v) crosses the cut if one endpoint is in S and the other is in V-S.

● Given a subset of edges A, a cut respects A if no edge in A crosses the cut.

M. Shoaib Farooq 20

Growing a MSTg

GENERIC-MST(G, w)1. A ← Φ1. A Φ2. while A doesnot form a spanning tree3. do find an edge(u, v) that is safe for A4. A ← A U {(u, v)}5. return A

Growing MSTg

b c d8 7

94

a

9

ei 144

4

11

2

h g f 10

ei 46

8 7h g f

1 2

S V - S

Growing MSTg

b c d8 7

94

a

9

ei 144

4

11

2

h g f 10

ei 46

8 7h g f

1 2

Growing MSTg

b c d8 7

94

a

9

ei 144

4

11

2

h g f 10

ei 46

8 7h g f

1 2

Total Weight = 37


Two Algorithms are used to solve minimum spanning tree problem

Kruskal’s algorithm

Two Algorithms are used to solve minimum spanning tree problemLet A be a subset of edges included in some MST

A is a forest.The safe edge added to A has least weight among all

g

The safe edge added to A has least weight among alledges connecting its two components.

Prim’s algorithmPrim’s algorithmA is a tree. The safe edge is a light edge connecting a vertex in A

M. Shoaib Farooq 40

to one not in A.

Kruskal’s Algorithm (Operations)g

● Make-set(u): Create a set containing a singleMake set(u): Create a set containing a single item u.

● Find-set(u): Find the set that contains u● Find-set(u): Find the set that contains u

● Union(u,v): merge the set containing u and set containing v into a common set.

M. Shoaib Farooq 41

Kruskal’s Algorithmg

Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148

for each v ∈ VMakeSet(v);

sort E by increasing edge weight w

1

5

1321

sort E by increasing edge weight wfor each (u,v) ∈ E (in sorted order)

if FindSet(u) ≠ FindSet(v)if FindSet(u) ≠ FindSet(v)T = T U {{u,v}};Union(u, v);

M. Shoaib Farooq 42

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 43

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 44

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1?

5

1321



M. Shoaib Farooq 45

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 46

Return T


Kruskal()2? 19

Run the algorithm:

T = ∅;

2? 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 47

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 48

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5?

1725

148



1

5?

1321



M. Shoaib Farooq 49

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 50

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148?



1

5

1321



M. Shoaib Farooq 51

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 52

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9?

5

1725

148



1

5

1321



M. Shoaib Farooq 53

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 54

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

13?21



M. Shoaib Farooq 55

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 56

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

14?8



1

5

1321



M. Shoaib Farooq 57

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 58

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

17?25

148



1

5

1321



M. Shoaib Farooq 59

Return T


Kruskal()2 19?

Run the algorithm:

T = ∅;

2 19?

9

5

1725

148



1

5

1321



M. Shoaib Farooq 60

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321?



M. Shoaib Farooq 61

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725?

148



1

5

1321



M. Shoaib Farooq 62

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 63

Return T


Kruskal()2 19

Run the algorithm:

T = ∅;

2 19

9

5

1725

148



1

5

1321



M. Shoaib Farooq 64

Return T

Prim’s Algorithm – an Exampleg p

da

7

f

d

b

2 5

45

2choosen

f b g11 34

c e 74

M. Shoaib Farooq 65

edge candidates


da

7

f

d

b

2 5

45

2

f b g11 34

c e 74

M. Shoaib Farooq 66


da

7

f

d

b

2 5

45

2

f b g11 34

c e 74

M. Shoaib Farooq 67


da

7

f

d

b

2 5

45

2

f b g11 34

c e 74

M. Shoaib Farooq 68


da

7

f

d

b

2 5

45

2

f b g11 34

c e 74

M. Shoaib Farooq 69

Prim’s Algorithm – an ExampleTotal weight of the MST: 14

g p

da

7

f

d

b

2 5

45

2

f b g11 34

c e 74

M. Shoaib Farooq 70

Prim’s Algorithmg

MST-Prim(G, w, r)Q = V[G];for each u ∈ Q

key[u] = ∞;ykey[r] = 0;p[r] = NULL;while (Q not empty)while (Q not empty)

u = ExtractMin(Q);for each v ∈ Adj[u]

if (v ∈ Q and w(u,v) < key[v])p[v] = u;key[v] = w(u,v);

M. Shoaib Farooq 71

Prim’s Algorithmg

MST-Prim(G, w, r) 6 49Q = V[G];

for each u ∈ Qkey[u] = ∞; 14 10

5

2

9

ykey[r] = 0;p[r] = NULL;while (Q not empty)

10

3

15

8while (Q not empty)u = ExtractMin(Q);for each v ∈ Adj[u]

Run on example graph


M. Shoaib Farooq 72

Prim’s Algorithmg

MST-Prim(G, w, r) ∞6 49Q = V[G];

for each u ∈ Qkey[u] = ∞;

∞ ∞ ∞

14 10

5

2

9


∞ ∞ ∞

∞

10

3

15


Run on example graph


M. Shoaib Farooq 73

Prim’s Algorithmg



∞ ∞ ∞

14 10

5

2

9


0 ∞ ∞

∞

10

3

15

8

r

while (Q not empty)u = ExtractMin(Q);for each v ∈ Adj[u]

Pick a start vertex r


M. Shoaib Farooq 74

Prim’s Algorithmg



∞ ∞ ∞

14 10

5

2

9


0 ∞ ∞

∞

10

3

15

8

u


Red vertices have been removed from Q


M. Shoaib Farooq 75

Prim’s Algorithmg



∞ ∞ ∞

14 10

5

2

9


0 ∞ ∞

3

10

3

15

8

u


3

Red arrows indicate parent pointers


M. Shoaib Farooq 76

Prim’s Algorithmg



14 ∞ ∞

14 10

5

2

9


0 ∞ ∞

3

10

3

15

8

u


3


M. Shoaib Farooq 77

Prim’s Algorithmg



14 ∞ ∞

14 10

5

2

9


0 ∞ ∞

3

10

3

15


3u


M. Shoaib Farooq 78

Prim’s Algorithmg



14 ∞ ∞

14 10

5

2

9


0 8 ∞

3

10

3

15


3u


M. Shoaib Farooq 79

Prim’s Algorithmg



10 ∞ ∞

14 10

5

2

9


0 8 ∞

3

10

3

15


3u


M. Shoaib Farooq 80

Prim’s Algorithmg



10 ∞ ∞

14 10

5

2

9


0 8 ∞

3

10

3

15


3u


M. Shoaib Farooq 81

Prim’s Algorithmg



10 2 ∞

14 10

5

2

9


0 8 ∞

3

10

3

15


3u


M. Shoaib Farooq 82

Prim’s Algorithmg



10 2 ∞

14 10

5

2

9


0 8 15

3

10

3

15


3u


M. Shoaib Farooq 83

Prim’s Algorithmg

MST-Prim(G, w, r) ∞6 49

u

Q = V[G];for each u ∈ Q

key[u] = ∞;

10 2 ∞

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 84

Prim’s Algorithmg

MST-Prim(G, w, r) ∞6 49

u


key[u] = ∞;

10 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 85

Prim’s Algorithmg

MST-Prim(G, w, r) 46 49

u


key[u] = ∞;

10 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 86

Prim’s Algorithmg


u


key[u] = ∞;

5 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 87

Prim’s Algorithmg


u


key[u] = ∞;

5 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 88

Prim’s Algorithmg


u


key[u] = ∞;

5 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 89

Prim’s Algorithmg


u


key[u] = ∞;

5 2 9

14 10

5

2

9


0 8 15

3

10

3

15


3


M. Shoaib Farooq 90

Prim’s Algorithmg

MST-Prim(G, w, r) 46 49Q = V[G];


5 2 9

14 10

5

2

9

uykey[r] = 0;p[r] = NULL;while (Q not empty)

0 8 15

3

10

3

15


3


M. Shoaib Farooq 91

Review: Prim’s Algorithmg


key[u] = ∞;ykey[r] = 0;p[r] = NULL;while (Q not empty)

What is the hidden cost in this code?while (Q not empty)



M. Shoaib Farooq 92



key[u] = ∞;ykey[r] = 0;p[r] = NULL;while (Q not empty)while (Q not empty)


if (v ∈ Q and w(u,v) < key[v])p[v] = u;DecreaseKey(v, w(u,v));

M. Shoaib Farooq 93



key[u] = ∞; How often is ExtractMin() called?ykey[r] = 0;p[r] = NULL;while (Q not empty)

How often is ExtractMin() called?How often is DecreaseKey() called?


if (v ∈ Q and w(u,v) < key[v])p[v] = u;DecreaseKey(v, w(u,v));

M. Shoaib Farooq 94



key[u] = ∞;What will be the running time?A: Depends on queuey

key[r] = 0;p[r] = NULL;while (Q not empty)

A: Depends on queuebinary heap: O(E lg V)Fibonacci heap: O(V lg V + E)



M. Shoaib Farooq 95

dag and minimum spanning trees

Technology

e e f g

directed edges gog g

ec e f g g f

directed graph g

shoaib farooq

dfs of g yields

t e o t o t e

e t e tota