advanced algorithms
DESCRIPTION
Advanced Algorithms. Piyush Kumar (Lecture 2: Max Flows). Welcome to COT5405. Slides based on Kevin Wayne’s slides. Announcements. Preliminary Homework 1 is out Scribing is worth 5% extra credit. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/1.jpg)
Advanced AlgorithmsAdvanced AlgorithmsAdvanced AlgorithmsAdvanced Algorithms
Piyush KumarPiyush Kumar(Lecture 2: Max Flows)(Lecture 2: Max Flows)
Welcome to COT5405 Slides based on Kevin Wayne’s slides
![Page 2: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/2.jpg)
Announcements
• Preliminary Homework 1 is out• Scribing is worth 5% extra credit.• My compgeom.com web pages might
not be accessible from inside campus(for now you can use cs.fsu.edu/~piyush pages.
![Page 3: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/3.jpg)
Today
• More about the programming assignment.
• On Max Flow Algorithms
![Page 4: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/4.jpg)
Soviet Rail Network, 1955
Reference: On the history of the transportation and maximum flow problems.Alexander Schrijver in Math Programming, 91: 3, 2002.
“The Soviet rail system also roused the interest of the Americans, and again it inspired fundamental research in optimization.”
-- Schrijver
G. Danzig* 1951…First soln…
Again formulted byHarris in 1955 for theUS Airforce (“unclassified in 1999”)What were they looking for?
![Page 5: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/5.jpg)
Maximum Flow and Minimum Cut
• Max flow and min cut.– Two very rich algorithmic problems.– Cornerstone problems in combinatorial
optimization.– Beautiful mathematical duality.
![Page 6: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/6.jpg)
Applications– Network reliability.
– Distributed computing.
– Egalitarian stable matching.
– Security of statistical data.
– Network intrusion detection.
– Multi-camera scene reconstruction.
– Many many more . . .
- Nontrivial applications / reductions.- Data mining.- Open-pit mining. - Project selection.- Airline scheduling.- Bipartite matching.- Baseball elimination.- Image segmentation.- Network connectivity.
![Page 7: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/7.jpg)
Some more history
![Page 8: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/8.jpg)
• Flow network.– Abstraction for material flowing through the edges.– G = (V, E) = directed graph, no parallel edges.– Two distinguished nodes: s = source, t = sink.– c(e) = capacity of edge e.
Minimum Cut Problem
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4
capacity
source sink
![Page 9: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/9.jpg)
• Def. An s-t cut is a partition (A, B) of V with s A and t B.
• Def. The capacity of a cut (A, B) is:
Cuts
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4
Capacity = 10 + 5 + 15 = 30
A
cap( A, B) c(e)e out of A
![Page 10: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/10.jpg)
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 A
• Def. An s-t cut is a partition (A, B) of V with s A and t B.
• Def. The capacity of a cut (A, B) is:
Cuts
cap( A, B) c(e)e out of A
Capacity = 9 + 15 + 8 + 30 = 62
![Page 11: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/11.jpg)
• Min s-t cut problem. Find an s-t cut of minimum capacity.
Minimum Cut Problem
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 A
Capacity = 10 + 8 + 10 = 28
![Page 12: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/12.jpg)
• Def. An s-t flow is a function that satisfies:
– For each e E:
(capacity)– For each v V – {s, t}:
(conservation)
• Def. The value of a flow f is:
Flows
4
0
0
0
0 0
0 4 4
0
0
0
Value = 40
capacity
flow
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
f (e)e in to v f (e)
e out of v
0 f (e) c(e)
v( f ) f (e) e out of s
.
4
![Page 13: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/13.jpg)
Flows as Linear Programs
• Maximize:
Subject to:
v( f ) f (e) e out of s
.
f (e)e in to v f (e)
e out of v
0 f (e) c(e)
![Page 14: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/14.jpg)
Flows
10
6
6
11
1 10
3 8 8
0
0
0
11
capacity
flow
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
Value = 24
4
• Def. An s-t flow is a function that satisfies:
– For each e E:
(capacity)– For each v V – {s, t}:
(conservation)
• Def. The value of a flow f is:
f (e)e in to v f (e)
e out of v
0 f (e) c(e)
v( f ) f (e) e out of s
.
![Page 15: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/15.jpg)
• Max flow problem. Find s-t flow of maximum value.
Maximum Flow Problem
10
9
9
14
4 10
4 8 9
1
0 0
0
14
capacity
flow
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
Value = 28
![Page 16: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/16.jpg)
• Flow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.
Flows and Cuts
10
6
6
11
1 10
3 8 8
0
0
0
11
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
Value = 24
f (e)e out of A
f (e)e in to A
v( f )
4
A
![Page 17: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/17.jpg)
• Flow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.
Flows and Cuts
10
6
6
1 10
3 8 8
0
0
0
11
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
f (e)e out of A
f (e)e in to A
v( f )
Value = 6 + 0 + 8 - 1 + 11 = 24
4
11
A
![Page 18: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/18.jpg)
• Flow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.
Flows and Cuts
10
6
6
11
1 10
3 8 8
0
0
0
11
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0
f (e)e out of A
f (e)e in to A
v( f )
Value = 10 - 4 + 8 - 0 + 10 = 24
4
A
![Page 19: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/19.jpg)
Flows and Cuts• Flow value lemma. Let f be any
flow, and let (A, B) be any s-t cut. Then
• Pf.
f (e)e out of A
f (e) v( f )e in to A
.
v( f ) f (e)e out of s
v A f (e)
e out of v f (e)
e in to v
f (e)e out of A
f (e).e in to A
by flow conservation, all termsexcept v = s are 0
![Page 20: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/20.jpg)
Flows and Cuts
• Weak duality. Let f be any flow, and let (A, B) be any s-t cut. Then the value of the flow is at most the capacity of the cut.
Cut capacity = 30 Flow value 30
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4
Capacity = 30
A
![Page 21: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/21.jpg)
Weak duality. Let f be any flow. Then, for any s-t cut
(A, B) we have v(f) cap(A, B).
• Pf.
Flows and Cuts
v( f ) f (e)e out of A
f (e)e in to A
f (e)e out of A
c(e)e out of A
cap(A, B)s
t
A B
7
6
8
4
![Page 22: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/22.jpg)
Certificate of Optimality
• Corollary. Let f be any flow, and let (A, B) be any cut.If v(f) = cap(A, B), then f is a max flow and (A, B) is a min cut.
Value of flow = 28Cut capacity = 28 Flow value 28
10
9
9
14
4 10
4 8 9
1
0 0
0
14
s
2
3
4
5
6
7
t
15
5
30
15
10
8
15
9
6 10
10
10 15 4
4 0A
![Page 23: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/23.jpg)
Towards a Max Flow Algorithm
• Greedy algorithm.– Start with f(e) = 0 for all edge e E.– Find an s-t path P where each edge has f(e) < c(e).– Augment flow along path P.– Repeat until you get stuck.
s
1
2
t
10
10
0 0
0 0
0
20
20
30
Flow value = 0
![Page 24: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/24.jpg)
Towards a Max Flow Algorithm
• Greedy algorithm.– Start with f(e) = 0 for all edge e E.– Find an s-t path P where each edge has f(e) < c(e).– Augment flow along path P.– Repeat until you get stuck.
s
1
2
t
20
Flow value = 20
10
10 20
30
0 0
0 0
0
X
X
X
20
20
20
![Page 25: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/25.jpg)
Towards a Max Flow Algorithm• Greedy algorithm.
– Start with f(e) = 0 for all edge e E.– Find an s-t path P where each edge has f(e) < c(e).– Augment flow along path P.– Repeat until you get stuck.
greedy = 20
s
1
2
t
20 10
10 20
30
20 0
0
20
20
opt = 30
s
1
2
t
20 10
10 20
30
20 10
10
10
20
locally optimality global optimality
![Page 26: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/26.jpg)
Residual Graph• Original edge: e = (u, v) E.
– Flow f(e), capacity c(e).
• Residual edge.– "Undo" flow sent.– e = (u, v) and eR = (v, u).– Residual capacity:
• Residual graph: Gf = (V, Ef ).
– Residual edges with positive residual capacity.
– Ef = {e : f(e) < c(e)} {eR : c(e) > 0}.
u v 17
6
capacity
u v 11
residual capacity
6
residual capacity
flow
c f (e) c(e) f (e) if e E
f (e) if eR E
![Page 27: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/27.jpg)
DemoDemoDemoDemo
![Page 28: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/28.jpg)
Augmenting Path Algorithm
Augment(f, c, P) { b bottleneck(P) foreach e P { if (e E) f(e) f(e) + b else f(eR) f(e) - b } return f}
Ford-Fulkerson(G, s, t, c) { foreach e E f(e) 0 Gf residual graph
while (there exists augmenting path P) { f Augment(f, c, P) update Gf
} return f}
forward edge
reverse edge
![Page 29: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/29.jpg)
Max-Flow Min-Cut Theorem• Augmenting path theorem. Flow f is a max flow iff there are no augmenting paths.
• Max-flow min-cut theorem. [Ford-Fulkerson 1956] The value of the max flow is equal to the value of the min cut.
• Proof strategy. We prove both simultaneously by showing the TFAE:
(i) There exists a cut (A, B) such that v(f) = cap(A, B).
(ii) Flow f is a max flow.
(iii) There is no augmenting path relative to f.
• (i) (ii) This was the corollary to weak duality lemma.
• (ii) (iii) We show contrapositive.– Let f be a flow. If there exists an augmenting path, then we can improve f by
sending flow along path.
![Page 30: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/30.jpg)
Proof of Max-Flow Min-Cut Theorem• (iii) (i)
– Let f be a flow with no augmenting paths.– Let A be set of vertices reachable from s in
residual graph.– By definition of A, s A.– By definition of f, t A.
v( f ) f (e)e out of A
f (e)e in to A
c(e)e out of A
cap(A, B)
original network
s
t
A B
![Page 31: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/31.jpg)
Running Time• Assumption. All capacities are integers between 1 and U. The cut
containing s and rest of the nodes has C capacity.
• Invariant. Every flow value f(e) and every residual capacities cf (e) remains an integer throughout the algorithm.
• Theorem. The algorithm terminates in at most v(f*) C iterations.• Pf. Each augmentation increase value by at least 1. ▪
• Corollary. If C = 1, Ford-Fulkerson runs in O(m) time.
• Integrality theorem. If all capacities are integers, then there exists a max flow f for which every flow value f(e) is an integer.
• Pf. Since algorithm terminates, theorem follows from invariant. ▪
Total running time?
![Page 32: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/32.jpg)
32
7.3 Choosing Good 7.3 Choosing Good Augmenting PathsAugmenting Paths
![Page 33: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/33.jpg)
Ford-Fulkerson: Exponential Number of Augmentations
• Q. Is generic Ford-Fulkerson algorithm polynomial in input size?
• A. No. If max capacity is C, then algorithm can take C iterations.
s
1
2
t
C
C
0 0
0 0
0
C
C
1 s
1
2
t
C
C
1
0 0
0 0
0X 1
C
C
X
X
X
1
1
1
X
X
1
1X
X
X
1
0
1
m, n, and log C
![Page 34: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/34.jpg)
Choosing Good Augmenting Paths• Use care when selecting augmenting paths.
– Some choices lead to exponential algorithms.– Clever choices lead to polynomial algorithms.– If capacities are irrational, algorithm not guaranteed to
terminate!
• Goal: choose augmenting paths so that:– Can find augmenting paths efficiently.– Few iterations.
• Choose augmenting paths with: [Edmonds-Karp 1972, Dinitz 1970]
– Max bottleneck capacity.– Sufficiently large bottleneck capacity.– Fewest number of edges.
![Page 35: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/35.jpg)
Capacity Scaling• Intuition. Choosing path with highest bottleneck
capacity increases flow by max possible amount.– Don't worry about finding exact highest bottleneck
path.– Maintain scaling parameter .
– Let Gf () be the subgraph of the residual graph consisting of only arcs with capacity at least .
110
s
4
2
t 1
170
102
122
Gf
110
s
4
2
t
170
102
122
Gf (100)
![Page 36: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/36.jpg)
Capacity ScalingScaling-Max-Flow(G, s, t, c) { foreach e E f(e) 0 smallest power of 2 greater than or equal to max c Gf residual graph
while ( 1) { Gf() -residual graph while (there exists augmenting path P in Gf()) { f augment(f, c, P) update Gf() } / 2 } return f}
![Page 37: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/37.jpg)
Capacity Scaling: Correctness
• Assumption. All edge capacities are integers between 1 and C.
• Integrality invariant. All flow and residual capacity values are integral.
• Correctness. If the algorithm terminates, then f is a max flow.
• Pf.
– By integrality invariant, when = 1 Gf() = Gf.
– Upon termination of = 1 phase, there are no augmenting paths. ▪
![Page 38: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/38.jpg)
Capacity Scaling: Running Time• Lemma 1. The outer while loop repeats 1 + log2 C times.
• Pf. Initially < C. drops by a factor of 2 each iteration and never gets below 1.
• Lemma 2. Let f be the flow at the end of a -scaling phase. Then the value of the maximum flow is at most v(f) + m .
• Lemma 3. There are at most 2m augmentations per scaling phase.– Let f be the flow at the end of the previous scaling phase.– L2 v(f*) v(f) + m (2).– Each augmentation in a -phase increases v(f) by at least
. • Theorem. The scaling max-flow algorithm finds a max flow
in O(m log C) augmentations. It can be implemented to run in O(m2 log C) time. ▪
proof on next slide
![Page 39: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/39.jpg)
Capacity Scaling: Running Time• Lemma 2. Let f be the flow at the end of a -scaling
phase. Then value of the maximum flow is at most v(f) + m .
• Pf. (almost identical to proof of max-flow min-cut theorem)– We show that at the end of a -phase, there exists a
cut (A, B) such that cap(A, B) v(f) + m .– Choose A to be the set of nodes reachable from s in
Gf().
– By definition of A, s A.– By definition of f, t A.
v( f ) f (e)e out of A
f (e)e in to A
(c(e)e out of A
) e in to A
c(e)e out of A
e out of A
e in to A
cap(A, B) - m original network
s
t
A B
![Page 40: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/40.jpg)
• Bipartite matching. Can solve via reduction to max flow.
• Flow. During Ford-Fulkerson, all capacities and flows are 0/1. Flow corresponds to edges in a matching M.
• Residual graph GM simplifies to:
– If (x, y) M, then (x, y) is in GM.
– If (x, y) M, the (y, x) is in GM.
• Augmenting path simplifies to:– Edge from s to an unmatched node x X.– Alternating sequence of unmatched and matched edges.– Edge from unmatched node y Y to t.
Bipartite Matching
s t
1 1
1
YX
![Page 41: Advanced Algorithms](https://reader036.vdocuments.mx/reader036/viewer/2022081506/56814655550346895db36c5d/html5/thumbnails/41.jpg)
References
• R.K. Ahuja, T.L. Magnanti, and J.B. Orlin. Network Flows. Prentice Hall, 1993. (Reserved in Dirac)
• R.K. Ahuja and J.B. Orlin. A fast and simple algorithm for the maximum flow problem. Operation Research, 37:748-759, 1989.
• K. Mehlhorn and S. Naeher. The LEDA Platform for Combinatorial and Geometric Computing. Cambridge University Press, 1999. 1018 pages.
• On the history of the transportation and maximum flow problems, Alexander Schrijver.