completing kosaraju’s algorithm introduction to greedy algorithms scheduling monday, july 7th
DESCRIPTION
Completing Kosaraju’s Algorithm Introduction to Greedy Algorithms Scheduling Monday, July 7th. Announcements. Scoryst : Annotate which pages answer which problems. OH room B24A: A is the desk number. The room is Gates B24. Night office hours: Yiming : Thursday 7- 9pm - PowerPoint PPT PresentationTRANSCRIPT
Completing Kosaraju’s Algorithm
Introduction to Greedy Algorithms
Scheduling
Monday, July 7th
Announcements
Scoryst: Annotate which pages answer which problems.
OH room B24A: A is the desk number. The room is
Gates B24.
Night office hours:
Yiming: Thursday 7-9pm
Billy: Wednesdays 7-9pm
Slides are posted on the website on the schedule tab.
Last Wednesday
1. DAGs
2. Topological Sorting
3. Strongly Connected Components
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Recap: DAGs
Job Scheduling in Operating Systems
grep
tee
wc
awk
cat
job1
job2
echo
DAG Terminology
grep
tee
wc
awk
cat
job1
job2
echo
source: no incoming edges
sink: no outgoing edges
Topological Sorting of A DAGInput:
CS 106A
CS 106B CS 107 CS 110
CS 103 CS 109 CS 161
CS 143
106A 106B 107 110 103 109 161 143
Output:
OR
106A 103 106B 109 161 107 143 110
OR…Output is not unique
Topological Sorting Algorithm #1
procedure topologicalSort1(DAG G): let result empty list while G is not empty: 1. let v be a source node in G 2. add v to result 3. remove v from G return result/reverse
Note: You can also pick a sink
iteratively!
Pseudocode TS Algorithm #2 (Using DFS) procedure topologicalSort2(DAG G): run DFS(G) & put each finished vertex into an array in reverse orderRuntime = Runtime of DFS = O(n + m)
Correctness due to Key lemma about finishing times:
If u v, then f[u] > f[v]
Recap: Strongly Connected Components
L
I
K
J A
D
H
EF
G
B
SCC1 SCC2
SCC3
SCC4
C
SCC1 SCC2
SCC3
SCC4
Claim: GSCC has to be a DAG!
Two-tiered Structure
Kosaraju’s Algorithm: High-level Goal
Use BFS/DFS to peel off the components by label propagation
by picking start nodes from sink SCCs iteratively.
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
EF
G
B
C
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
E
F
G
B
C
E
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
EE
G
B
C
EF
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
EE
E
B
C
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
EE
E
B
C
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
EE
E
B
C
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
B
C
EE
E
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
B
C
B
EE
E
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
B
B
B
C
EE
E
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
B
B
B
C
EE
E
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J A
D
H
BB
BC
EE
E
EF
G
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
D
H
A
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
A
HA
A
D
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
A
A
A
D
H
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
A
A
A
D
H
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
A
A
A
D
H
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
JA
A
A
A
D
H
EE
E
EF
G
B
BC
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
J
I
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
L
I
K
I
I
J
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
L
I
I
I
I
J
K
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
I
II
I
IJ
K
L
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
I
I
I
IJ
K
L
I
EE
E
EF
G
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
I
I
I
IJ
K
L
I
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
I
I
I
IJ
K
L
I
EE
E
EF
G
B
BC
A
A
A
A
D
H
Goal: Use BFS/DFS to Peel-off SCCs
I
I
I
IJ
K
L
I
EE
E
EF
G
B
BC
A
A
A
A
D
H
How To Pick The Source Nodes?
Run DFS on Grev and pick source nodes
in
decreasing order of finishing times
Kosaraju’s Algorithm
procedure kosarajusSCC(DAG G): 1. run DFS on Grev and compute finishing times f 2. run DFS/BFS in G:
pick start vertices by decreasing f times from the first DFS propagate the ID of each new start vertex during traversalRun-time: O(n + m)!
DFS & Finishing Times on Grev
L
I
K
J A
D
H
EF
G
B
C
L
I
K
J A
D
H
F
G
B
C
E
DFS & Finishing Times on Grev
L
I
K
J A
D
H
F
B
C
E
G
DFS & Finishing Times on Grev
L
I
K
J A
D
H
B
C
E
G
F
DFS & Finishing Times on Grev
L
I
K
J A
D
H
1
B
C
E
G
F
DFS & Finishing Times on Grev
L
I
K
J A
D
H
1
2
B
C
E
G
F
DFS & Finishing Times on Grev
L
I
K
J A
H
1
2
B
C
E
G
F
D
DFS & Finishing Times on Grev
L
I
K
J
H
1
2
B
C
E
G
F
D
A
DFS & Finishing Times on Grev
L
I
K
J
1
2
B
C
E
G
F
D
A
H
DFS & Finishing Times on Grev
L
I
K
J
3
1
2
B
C
E
G
F
D
A
H
DFS & Finishing Times on Grev
L
I
K3
1
2
B
C
E
G
F
D
A
H
J
DFS & Finishing Times on Grev
L
K3
1
2
B
C
E
G
F
D
A
H
J
I
DFS & Finishing Times on Grev
K3
1
2
B
C
E
G
F
D
A
H
J
I
L
DFS & Finishing Times on Grev
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
4 3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
4 3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
6
4 3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
6
4
7
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
6
4
7 8
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
10
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
10
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
10
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
10
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
B
C
E
G
F
D
A
H
J
I
L
K
10
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
C
E
G
F
D
A
H
J
I
L
K
10
B
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
Why Does DFS on Grev Work?
Key Lemma:
In GSCC if SCCi SCCj, then ***f[SCCi] > f[SCCj]***
Define: f[SCCi] = maxv∈SCCi f[v]
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
DFS & Finishing Times on Grev
SCC4
SCC3
SCC2
SCC1 f[SCC1] = 7 f[SCC2] = 9
f[SCC3] =10
f[SCC4] = 12
SCC4 > SCC3 > SCC1SCC2 >
Topological Ordering of SCCs
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
Let’s go back to the original graph G
SCC4
SCC3
SCC2
SCC1 f[SCC1] = 7 f[SCC2] = 9
f[SCC3] =10
f[SCC4] = 12SCC4 > SCC3 > SCC1SCC2 >
Kosaraju’s Algorithm Simulation
5
6
4
7 8
9
3
1
2
E
GD
A
H
J
I
L
K
10
B
C11
12
First Step: DFS on GrevF
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
Kosaraju’s Algorithm Simulation
Second Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
B
C
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
B
C
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
C
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EF
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
F
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
G
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
D
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
DD
H
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J A
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
J AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
K
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
L
I
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
I
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Kosaraju’s Algorithm SimulationSecond Step: DFS/BFS on
G
12
B C11
E10
D9
A8
J7
I6
L5
K4
H3
G2
F1
LJ
IJ
KJ
JJ
AD
DD
HD
EE
FE
GE
BB
CB
Proof of Key Lemma:In GSCC if SCCi SCCj, then f[SCCi] > f[SCCj] SCCi SCCj
Let u be the first vertex visited in SCCi or SCCj
Case 1: u is in SCCi =>DFS starts at umoves to SCCjfinishes
SCCjfinishes uf[u] > f[v] ∀v ∈SCCj
=> f[SCCi] ≥ f[u] ≥ f[SCCj]Case 2: u is in SCCj => DFS starts at ufinishes SCCjstarts SCCifinishes SCCi
f[u] < s[y] ∀y ∈SCCi => f[u] < f[y] ∀y ∈SCCi => f[SCCi] > f[SCCj]
u u
Proof of Correctness of Kosaraju
Claim: Kosaraju finds SCCs in topological order TO* of (Grev)SCC output by the first DFS.
5
6
4
7 8
9
3
1
2
E
G
F
D
A
H
J
I
L
K
10
B
C11
12
TO* For Our Example
SCC4
SCC3
SCC2
SCC1 f[SCC1] = 7 f[SCC2] = 9
f[SCC3] =10
f[SCC4] = 12
SCC4 > SCC3 > SCC1SCC2 >
Topological Ordering of SCCs
Correctness of Kosaraju (1)
Claim: Kosaraju finds SCCs in topological order TO* of (Grev)SCC output by the first DFS.
Proof Sketch: Induct on the SCCs in the topological order.
Let the TO* be SCC1 SCC2 … SCCm
Base Case: by definition of TO*, the highest ft vertex is in
SCC1 AND SCC1 is a sink SCC in G
BFS/DFS can only discover SCC1
Correctness of Kosaraju (2)
Induc. Hypothesis: Assume Kosaraju finds first k-1 SCCs
in TO*
By definition of TO* and inductive hypothesis, the next
highest ft vertex that is unvisited, say u, will be from
SCCk.
By key lemma: BFS/DFS discovers only vertices in
SCCk && SCC1 … SCCk-1
But (by inductive hypothesis) all vertices in the k-1 SCCs
are already “peeled-off”, i.e. finished
only SCCk will be discovered during a BFS/DFS from u.
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Interesting Shortest Paths PhenomenonMilgram’s 1967 “six-degrees of separation” experiment
Humans are extremely well connected
On average 6 hops apart
Latest studies range from ~4.7 to ~5.8
Surprising because we form an extremely sparse graph
On FB > 1 Billion users: 109
~300 average edges per user
Interesting fact: Same Milgram who performed the
“obedience to authority” experiment
Interesting Conn. Comp. PhenomenonReal Graphs: Very Skewed Distribution in component
sizes
Checkout Jure Leskovec’s Graph Data:
http://snap.stanford.edu/data/
Bow-tie web graph from Broder et. al. 2000
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Greedy Algorithms
Algorithms that iteratively make
“short-sighted”, “locally optimum looking” decisions
hoping to output a good solution (hopefully
optimum)
Example: Dijkstra’s SSSP Algorithm
Greedy vs Divide-And-Conquer Algorithms
Greedy Divide and Conquer
easy to design difficult to design
Greedy vs Divide-And-Conquer Algorithms
Greedy Divide and Conquer
easy to design difficult to design
easy to analyze run-time difficult to analyze run-time
Greedy vs Divide-And-Conquer Algorithms
Greedy Divide and Conquer
easy to design difficult to design
easy to analyze run-time difficult to analyze run-time
difficult to prove correctness
easy to prove correctness
Two Correctness Proof Techniques
Call greedy’s solution Sg, and let S be any other
solution
1. “Greedy stays ahead”
Argue Sg is optimal/better than S at each step
Proof by induction
Example: Dijkstra’s correctness proof
2. Exchange Arguments
Argue any S can be transformed into Sg step by step
and without getting worse along the way
Warning: They’re common but not applicable to every
greedy algorithm!
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Scheduling Problem 1 Input: A set of n jobs J. Each jobi has length li
l1Job 1
l2Job 2
lnJob n
…
Output: A schedule of the jobs on a processor
s.t:
is minimum over all possible n! schedules.
completion time of job
i
Completion Time of Job iDefinition: time when jobi finishes
i.e., sum of scheduled job lengths up to and including jobi
S1 1
3J15J2
1J4
1J3
J3 J2 J1 J45 3 1
time 1 6 9 10
Total Cost of S1:1 + 6 + 9 + 10 = 26
0
Another Example Schedule
S1 1
J3 J2 J1 J45 3 1
time 1 6 9 10
Total Cost of S1: 1 + 6 + 9 + 10 = 26
S2 1
J3 J25
J13
J41
time 1 4 5 10
Total Cost of S2:1 + 4 + 5 + 10 = 20
Goal is to find the min cost schedule!
Let’s Start Simple
3J15J2
What are all possible schedules?
S1
J25
J13
time 3 8
Total Cost:3 + 8 = 11
S2
J25
J13
time 5 8
Total Cost:5 + 8 = 13
Why Put One Job In Front of Another?
Observation:
Shorter jobs have less impact on the
completion times of future jobs
Greedy Algorithm 1
Schedule jobs by increasing lengths
3J15J2
1J4
1J3
Ex:
Sg 1
J3 J25
J13
J41
time 51 2 1
0
Total Cost of Sg:1 + 2 + 5 + 10 = 18
Run-time O(nlog(n))!
Comparing Sg to Previous Schedules
Sg 1
J3 J25
J13
J41
time 51 2 1
0
S1 1
J3 J2 J1 J45 3 1
time 1 6 9 10
S2 1
J3 J25
J13
J41
time 51 4 10
26
20
18
Proof of Correctness (1)
“Greedy stays ahead” proof:
Induct on the cost of the first k jobs executed
Argue Sg beats everyone else at each step
Let S[i]: the ith job that a schedule S executes
E.g., Sg[1] is the first job Sg executes
Let Cost(S, i): be the sum of the costs of the first i jobs
that schedule S executes.
E.g., Cost(Sg, 3) is the sum of completion times
Sg[1], Sg[2], Sg[3]: Sg[1] + (Sg[1]+Sg[2]) +
(Sg[1]+Sg[2]+Sg[3])
Goal: Argue ∀S, Cost(Sg, n) ≤ Cost(S, n) by inducting
on i
Proof of Correctness (2)
Base Case: ∀S, Cost(Sg, 1) = Sg[1] ≤ Cost(S, 1) since
Sg[1] is the shortest length job
Inductive Hypothesis: Cost(Sg, k-1) ≤ Cost(S, k-1)
By inductive hypothesis
By greedy criterion of Sg
QED
Outline For Today
1. Complete Kosaraju’s Algorithm
2. Interesting Phenomenon About Shortest Paths And
Connected Components
3. Introduction to Greedy Algorithms
4. Scheduling Problem Variant 1 & Greedy Algorithm 1
5. Scheduling Problem Variant 2 & Greedy Algorithm 2
Scheduling Problem 2 Input: Now each jobi has length li AND weight wi
l1, w1Job 1
l2, w2Job 2
ln,wnJob n
…
Output: A schedule of the jobs on a processor
s.t:
is minimum over all possible n! schedules.
weighted completion time of
job i
Example Schedule And Cost
S1
l=3 w=1J1l=5 w=2J2
l=1 w=1J4
l=1 w=3J3
J3 J2 J1 J4
time 1 6 9 10
Total Cost of S1:3*1 + 2*6 + 1*9 + 1*10 = 34
l=1 w=3 l=5 w=2 l=3 w=1
l=1 w=1
Q1: What To Do When Weights Are Same?
Same As Un-weighted Case Shorter lengths
first
Previous Greedy Algorithm is Optimal
l=3 w=1J1l=5 w=1J2
l=1 w=1J4
l=1 w=1J3
Q2: What To Do When Lengths Are Same?
Higher weights first
l=3 w=2J1l=3 w=3J2
l=3 w=4J4
l=3 w=1J3
minimize:
Q3: What To Do With Mixed L & W?
Say J1 is shorter and also has less weight than J2?
l=3 w=1J1l=5 w=2J2
minimize:
Unclear:
Intuition for Q1 says J1 should come first
Intuition for Q2 says J2 should come first
Ideal Scenario: Combine l and w into a
single score that combines both intuitions
and we could order by that score
Possible Combined Scores?
The combined score f(li, wi) should satisfy:
1. If weights same shorter lengths get smaller
scores
2. If lengths same larger weights get smaller
scores
Guess 1: f1(li, wi) = li – wi
Guess 2: f2(li, wi) = li / wi
Is either one correct?
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
J2 (l=5, w=2)
SCHEDULE
TOTAL COST
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2
J2 (l=5, w=2)
SCHEDULE
TOTAL COST
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2
J2 (l=5, w=2)
3
SCHEDULE
TOTAL COST
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2
J2 (l=5, w=2)
3
SCHEDULE J1::J2TOTAL COST
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2
J2 (l=5, w=2)
3
SCHEDULE J1::J2TOTAL COST
1*3 + 2*8 = 19
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2 3
J2 (l=5, w=2)
3
SCHEDULE J1::J2TOTAL COST
1*3 + 2*8 = 19
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2 3
J2 (l=5, w=2)
3 2.5
SCHEDULE J1::J2TOTAL COST
1*3 + 2*8 = 19
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2 3
J2 (l=5, w=2)
3 2.5
SCHEDULE J1::J2 J2::J1TOTAL COST
1*3 + 2*8 = 19
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1 = li - wi f2 = li/wi
J1 (l=3, w=1)
2 3
J2 (l=5, w=2)
3 2.5
SCHEDULE J1::J2 J2::J1TOTAL COST
1*3 + 2*8 = 19
2*5 + 1*8 = 18
Guess 1 is certainly not optimal.
Is Guess 2 optimal?
Greedy Algorithm (Weighted Scheduling)Schedule jobs by increasing li/wi
scores Run-time
O(nlog(n))!
S1
l=3 w=1J1l=5 w=2J2
l=1 w=1J4
l=1 w=3J3
J3 J2J1J4
time 1 2 7 10
Total Cost of Sg:3*1 + 1*2 + 2*7 + 1*10 = 29
l=1 w=3 l=5 w=2 l=3 w=1
l=1 w=1
score:3
score:2.5
score:1/3
score:1
Proof of Correctness (1)
By Exchange Argument:
Argue any S can be transformed into Sg step by
step and without getting worse along the way
Let’s rename jobs so that Sg schedules jobs in order:
J1, J2, …, Jn
i.e., J1 happens to be the job with smallest l/w ratio
Therefore Sg = J1::J2::…::Jn
Proof of Correctness (2)
Consider any other schedule S ≠ Sg
Claim: In S=Js1::Js2::…::Jsn there is a job k, right
after a job i where k < i.
Sg
J1l1 w1
J2l2 w2
J3l3 w3
Jnln wn…
SJ1
l1 w1
J9l9 w9
……
J6l6 w6
Exchange J9 with J6
Proof of Correctness (3)
S`J1
l1 w1
J6l6 w6…
J9l9 w9
Exchange J11 with J8
J11
l11 w11 …
J8l8 w8
S``J1
l1 w1
J6l6 w6…
J9l9 w9
J8l8 w8
…J11
l11 w11
Exchange J8 with J9…
J1
l1 w1
J2
l2 w2
J3
l3 w3
Jn
ln wn…
Sg
…
Completing the Proof
Recall Claim: In S=Js1::Js2::…::Jsn there is a job k, right
after a job i where k < i.
Q: How does the cost of S change when we swap i
and j?
wi * lk
Jili wi
Jklk wk
Jili wi
Jklk wk
wk * li
Prefix Jobs
Prefix Jobs
Suffix Jobs
Suffix Jobs
By renaming of jobs and k < i => lk/wk ≤ li/wi => wilk ≤
wkliQED
On Wednesday We Will Look at Minimum Spanning Trees
BACKUP SLIDES
Let’s First Try To Eliminate One Guess
l=3 w=1J1l=5 w=2J2
f1(li, wi) = li – wi f1(J1)= 3 – 1 =2, f1(J2)= 5 – 2 =3
f1 schedules them as J1::J2
Score: 1*3 + 2*8 = 19
f2(li, wi) = li / wi f2(J1)= 3/1 =3, f1(J1)= 5/2 =2.5
f1 schedules them as J2::J1
Score: 2*5 + 1*8 = 18
f1=li - wi f2 = li/wi
J1 (l=3, w=1)
2 3
J2 (l=5, w=2)
3 2.5
SCHEDULE J1::J2 J2::J1TOTAL COST
1*3 + 2*8 = 19
2*5 + 1*8 = 18
Directed Acyclic Graphs (DAGs)
Graphs to model “dependency”/“prerequisite”
relationships
directed & acyclic
CS 106A
CS 106B CS 107 CS 110
CS 103 CS 109 CS 161
CS 143
Topological Sorting of a DAG
Input: A G(V, E) which is a DAG
Output: vertices in sorted order s.t
each vertex v appears after its dependencies
A More Formal Definition:
Given an input DAG G(V, E) order the nodes
such that
if (u, v) ∈ E, then u appears before v
ExampleInput:
CS 106A
CS 106B CS 107 CS 110
CS 103 CS 109 CS 161
CS 143
106A 106B 107 110 103 109 161 143
Output:
OR
106A 103 106B 109 161 107 143 110
OR…Output is not unique
Two Topological Sorting Algorithms (1)
procedure topologicalSort1(DAG G): let result empty list while G is not empty: 1. let v be a source node in G 2. add v to result 3. remove v from G return result/reverse
Two Topological Sorting Algorithms (2) procedure topologicalSort2(DAG G): run DFS(G) & put each finished vertex into an array in reverse orderRuntime = Runtime of DFS = O(n + m)
Correctness depended on Key Claim:
In a DAG if u v, then f[u] > f[v]