tutorial 1 recurrence: divide and conquer strategy and analysis
TRANSCRIPT
![Page 2: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/2.jpg)
1. Math Background
![Page 3: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/3.jpg)
Computer Algorithms Design and Analysis
P21-27Harmonic Series:
Arithmetic-Geometric Series:
Stirling’s Formula:
1
1 ln( )n
in
iγ
=
≈ +∑
1
1
2 ( 1)2 2k
i k
i
i k +
=
= − +∑
! 2 ( )nnn ne
π≈
![Page 4: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/4.jpg)
Computer Algorithms Design and Analysis
Example: Asymptotic order:
![Page 5: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/5.jpg)
Computer Algorithms Design and Analysis
P52 Theorem 1.13Polynomial Series:
Geometric Series:
Logarithmic Series:
Polynomial- logarithmic Series:1
nd d+
~ (lar
1log( ) ~ ( log( ))
ii i n nθ
=∑
gest)b
i
i ar θ
=∑
1
log( ) ~ ( log( ))n
ii n nθ
=∑
1
1~ ( )
nd d
ii nθ +
=∑
![Page 6: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/6.jpg)
Computer Algorithms Design and Analysis
Example: Maximum Subsequence Sum
The problem: Given a sequence S of integer, find the largest sum of a consecutive subsequence of S. (0, if all negative items)
An example: -2, 11, -4, 13, -5, -2; the result 20: (11, -4, 13)
A brute-force algorithm: MaxSum = 0;for (i = 0; i < N; i++)for (j = i; j < N; j++){ThisSum = 0;for (k = i; k <= j; k++)ThisSum += A[k];if (ThisSum > MaxSum)MaxSum = ThisSum;
}return MaxSum;
……
i=0i=1
i=2
i=n-1
k
j=0 j=1 j=2
j=n-1
in O(n3)
the sequence
![Page 7: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/7.jpg)
Computer Algorithms Design and Analysis
More Precise Complexity
623
1)23(21)
23(
21
2)1)(2(
2))(1(
2))(1()(...21)1(
11
1:iscost totalThe
231
2
11
2
1
1
0
1
1
0
1
nnn
nnini
inininin
inininij
ij
n
i
n
i
n
i
n
i
n
i
n
ij
j
ik
n
i
n
ij
j
ik
++=
++++−=
+−+−=
−+−
−+−=−+++=+−
+−=
∑∑∑
∑∑
∑
∑
∑ ∑ ∑
===
=
−
=
−
=
=
−
=
−
= =
![Page 8: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/8.jpg)
Computer Algorithms Design and Analysis
Decreasing the number of loops
An improved algorithmMaxSum = 0;for (i = 0; i < N; i++){ThisSum = 0;for (j = i; j < N; j++){ThisSum += A[j];if (ThisSum > MaxSum)MaxSum = ThisSum;
}}return MaxSum;
the sequence
i=0i=1
i=2
i=n-1
j
in O(n2)
![Page 9: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/9.jpg)
Computer Algorithms Design and Analysis
Power of Divide-and-Conquer
Part 1 Part 2the sub with largest sum may be in:
Part 1 Part 2or:
Part 1 Part 2
recursion
The largest is the result
in O(nlogn)
![Page 10: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/10.jpg)
Computer Algorithms Design and Analysis
Divide-and-Conquer: the Procedure
Center = (Left + Right) / 2;MaxLeftSum = MaxSubSum(A, Left, Center); MaxRightSum = MaxSubSum(A, Center + 1, Right);
MaxLeftBorderSum = 0; LeftBorderSum = 0;for (i = Center; i >= Left; i--){
LeftBorderSum += A[i];if (LeftBorderSum > MaxLeftBorderSum) MaxLeftBorderSum = LeftBorderSum;
}
MaxRightBorderSum = 0; RightBorderSum = 0;for (i = Center + 1; i <= Right; i++){
RightBorderSum += A[i];if (RightBorderSum > MaxRightBorderSum) MaxRightBorderSum = RightBorderSum;
}return Max3(MaxLeftSum, MaxRightSum,
MaxLeftBorderSum + MaxRightBorderSum);
Note: this is the core part of the procedure, with base case and wrap omitted.
Note: this is the core part of the procedure, with base case and wrap omitted.
![Page 11: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/11.jpg)
Computer Algorithms Design and Analysis
A Linear Algorithm
ThisSum = MaxSum = 0;for (j = 0; j < N; j++){ThisSum += A[j];if (ThisSum > MaxSum)MaxSum = ThisSum;
else if (ThisSum < 0)ThisSum = 0;
}return MaxSum;
the sequence
j
Negative item or subsequence cannot be a prefix of the subsequence we want.
This is an example of “online algorithm”
in O(n)
![Page 12: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/12.jpg)
Computer Algorithms Design and Analysis
Comparison
Brute forceO(n3)
Improved brute forceO(n2)
Divide and ConquerO(n log n)
Double pointerO(n)
![Page 13: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/13.jpg)
2. Analysis to Recurrence
![Page 14: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/14.jpg)
Computer Algorithms Design and Analysis
Recursive Algorithm and Recurrence equationHow to analyze?
Guess and ProvingRecursion tree
![Page 15: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/15.jpg)
Computer Algorithms Design and Analysis
Guess and Proving
Example: T(n)=2T(⎣n/2⎦) +nGuess
T(n)∈O(n)? T(n)≤cn, to be proved for c large enough
T(n)∈O(n2)? T(n)≤cn2, to be proved for c large enough
T(n)∈O(nlogn)? T(n)≤cnlogn, to be proved for c large enough
Try to prove T(n)≤cn:T(n)=2T(⎣n/2⎦)+n ≤ 2c(⎣n/2⎦)+n
≤ 2c(n/2)+n = (c+1)n, Fail!
However: T(n) = 2T(⎣n/2⎦)+n ≥ 2c⎣n/2⎦+n
≥ 2c[(n-1)/2]+n = cn+(n-c) ≥ cn
T(n) = 2T(⎣n/2⎦)+n ≤ 2(c⎣n/2⎦ log (⎣n/2⎦))+n≤ cnlog (n/2)+n= cn log n – cn log 2 +n= cn log n – cn + n≤ cn log n for c≥1
![Page 16: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/16.jpg)
Computer Algorithms Design and Analysis
Recursion Tree
T(n) n
T(n/4) n/4T(n/4) n/4T(n/4) n/4T(n/4) n/4
T(n/2) n/2T(n/2) n/2
Work copy: T(k)=T(k/2)+T(k/2)+kWork copy: T(k)=T(k/2)+T(k/2)+k
At this level: T(n)=n+2(n/2)+4T(n/4)=2n+4T(n/4)At this level: T(n)=n+2(n/2)+4T(n/4)=2n+4T(n/4)
n/2d(size →1)
T(n)=nlgn
![Page 17: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/17.jpg)
Computer Algorithms Design and Analysis
2.1 T(n)=bT(n/c)+f(n)
Master TheoremLoosening the restrictions on f(n)
Case 1: f(n)∈O(nE‐ε), (ε>0), then:T(n)∈Θ(nE)
Case 2: f(n)∈Θ(nE), as all node depth contribute about equally:T(n)∈Θ(f(n)log(n))
case 3: f(n)∈Ω(nE+ε), (ε>0), and f(n)∈O(nE+δ), (δ≥ε), then:T(n)∈Θ(f(n))
The positive ε is critical, resulting gaps between cases as well
The positive ε is critical, resulting gaps between cases as well
This is regular condition, we can use bf(n/c) ≤ rf(n) (r<1) instead
![Page 18: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/18.jpg)
Computer Algorithms Design and Analysis
Recursion Tree for T(n)=bT(n/c)+f(n)
f(n)
T(1) T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)
f(n/c2)f(n/c2) f(n/c2) f(n/c2) f(n/c2)f(n/c2) f(n/c2) f(n/c2) f(n/c2)
…… ……
f(n/c) f(n/c) f(n/c)
logcn
f(n)
)/( cnbf
)/( 22 cnfb
…
( )bcnlogΘ
Note: bn cc nb loglog =
b
b
Total ?
…
18
![Page 19: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/19.jpg)
Computer Algorithms Design and Analysis
Examples(1) T(n)=4T(n/2)+n
case 1,T(n)∈Θ(n2)(2) T(n)=4T(n/2)+n2
case 2,T(n) ∈Θ(n2lgn)(3) T(n)=4T(n/2)+n3
case 3, T(n) ∈Θ(n3)(4) T(n)=4T(n/2)+n2/lgn
none of the three cases, gap
![Page 20: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/20.jpg)
Computer Algorithms Design and Analysis
2.2 T(n)=bT(n-c)+f(n) (P139)
![Page 21: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/21.jpg)
Computer Algorithms Design and Analysis
2.3 T(n)=r1T(n-1)+r2T(n-2)
Characteristic EquationIf the characteristic equation of the recurrence relation has two distinct roots s1 and s2, then
where u and v depend on the initial conditions, is the explicit formula for the sequence.
22
212211 vsusfandvsusf +=+=
nnn vsusa 21 +=
0212 =−− rxrx
2211 −− += nnn arara
![Page 22: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/22.jpg)
3. Divide and Conquer
![Page 23: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/23.jpg)
Computer Algorithms Design and Analysis
Basic Strategy
Divide: devide the problem into smaller instances of the same problemConquer: solve the smaller problem recursivelyCombine: combine the solutions to obtain the solution for the original input
![Page 24: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/24.jpg)
Computer Algorithms Design and Analysis
Example: Matrix Multiplication
![Page 25: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/25.jpg)
Computer Algorithms Design and Analysis
Standard Algorithm – by definition
Run time = Θ (n3)
![Page 26: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/26.jpg)
Computer Algorithms Design and Analysis
Divide‐and‐conquer Algorithm
Idea: n*n matrix = 2*2 of (n/2) * (n/2) sub‐matrices:
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡
2221
1211
2221
1211
2221
1211
BBBB
AAAA
CCCC
2112111111 BABAC +=
2212121112 BABAC +=
2122112121 BABAC +=
2222122122 BABAC +=
![Page 27: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/27.jpg)
Computer Algorithms Design and Analysis
Analysis:8 muls of (n/2)*(n/2) submatrices4 adds of (n/2)*(n/2) submatricesT(n)=8T(n/2)+n2
Still, T(n)=O(n3). No improvement!
![Page 28: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/28.jpg)
Computer Algorithms Design and Analysis
Strassen Algorithm
)( 2212111 BBAM −=
2212112 )( BAAM +=
1122213 )( BAAM +=)( 1121224 BBAM −=
))(( 221122115 BBAAM ++=))(( 222122126 BBAAM +−=))(( 121121117 BBAAM +−=
624511 MMMMC +−+=
2112 MMC +=
4321 MMC +=
731522 MMMMC −−+=
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡
2221
1211
2221
1211
2221
1211
BBBB
AAAA
CCCC
![Page 29: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/29.jpg)
Computer Algorithms Design and Analysis
Analysis:7 muls of (n/2)*(n/2) submatrices18 adds of (n/2)*(n/2) submatricesT(n)=7T(n/2)+n2
T(n)=O(nlog(7))=O(n2.81). Great improvement!
![Page 30: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/30.jpg)
Computer Algorithms Design and Analysis
Example: Number of Valid Strings
String to be transmitted on the channelLength n
Consisting of symbols ‘a’, ‘b’, ‘c’
If “aa” exists, cannot be transmitted
E.g. strings of length 2: ‘ab’, ‘ac’, ‘ba’, ‘bb’, ‘bc’, ‘ca’, ‘cc’, ‘cb’
Number of valid strings ?
![Page 31: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/31.jpg)
Computer Algorithms Design and Analysis
Divide and conquer
f(n)=2f(n‐1)+2f(n‐2), n>2f(1)=3, f(2)=8
b
a
c
b a c
n-1 n-1
n-2 n-2
![Page 32: Tutorial 1 Recurrence: Divide and Conquer Strategy and Analysis](https://reader030.vdocuments.mx/reader030/viewer/2022021307/62074b5349d709492c2ffa4b/html5/thumbnails/32.jpg)
Computer Algorithms Design and Analysis
Analysis of the D&C solution
0222 =−− xxCharacteristic equation
Solution
nnnf )31(32
32)31(3232)( −
+−++
+=