devide and conquar
TRANSCRIPT
-
8/9/2019 Devide and Conquar
1/24
2 S T R A T E G Y D I V I D E - A N D - C O N Q U E R
D i v i d e P r o b l e m P i n t o s m a l l e r p r o b l e m
P
1
P
2
: : : P
k
S o l v e p r o b l e m s P
1
P
2
: : : P
k
t o o b t a i n s o l u -
t i o n s S
1
S
2
: : : S
k
C o m b i n e s o l u t i o n S
1
S
2
: : : S
k
t o g e t t h e n a l
s o l u t i o n .
S u b p r o b l e m s P
1
P
2
: : : P
k
a r e s o l v e d r e c u r s i v e l y
u s i n g d i v i d e - a n d - c o n q u e r .
E x a m p l e s : Q u i c k s o r t a n d m e r g e s o r t .
1
-
8/9/2019 Devide and Conquar
2/24
3 S T R A T E G Y G R E E D Y
S o l u t i o n
f o r i 1 t o n d o
S E L E C T t h e n e x t i n p u t x
I f f x g S o l u t i o n i s F E A S I B L E t h e n
s o l u t i o n C O M B I N E ( S o l u t i o n , x )
S E L E C T a p p r o p r i a t e l y n d s t h e n e x t i n p u t t o
b e c o n s i d e r e d .
A F E A S I B L E s o l u t i o n s a t i s e s t h e c o n s t r a i n t s
r e q u i r e d f o r t h e o u t p u t .
C O M B I N E e n l a r g e s t h e c u r r e n t s o l u t i o n t o
i n c l u d e a n e w i n p u t .
E x a m p l e s : M a x n d i n g , S e l e c t i o n S o r t , a n d
K r u s k a l ' s S m a l l e s t E d g e F i r s t a l g o r i t h m f o r M i n -
i m u m S p a n n i n g T r e e .
2
-
8/9/2019 Devide and Conquar
3/24
4 S T R A T E G Y D Y N A M I C P R O G R A M M I N G
F i b o n a c c i N u m b e r s :
F
n
= F
n ; 1
+ F
n ; 2
F
1
= F
0
= 1
| R e c u r s i v e s o l u t i o n r e q u i r e s e x p o n e n t i a l t i m e :
h a s o v e r l a p p i n g s u b p r o b l e m s .
| B o t t o m - u p i t e r a t i v e s o l u t i o n i s l i n e a r {
c o m p u t e o n c e , s t o r e , a n d u s e m a n y
t i m e s .
3
-
8/9/2019 Devide and Conquar
4/24
4 . 1 M a t r i x S e q u e n c e M u l t i p l i c a t i o n
e g . 1 :
A
3 0 x 1
X B
1 x 4 0
X C
4 0 x 1 0
X D
1 0 x 2 5
X E
2 5 x 1
| L e f t t o r i g h t e v a l u a t i o n r e q u i r e s m o r e t h a n
1 2 K m u l t i p l i c a t i o n s .
| ( A X ( ( B X C ) X ( D X E ) ) ) n e e d s o n l y
6 9 0 m u l t i p l i c a t i o n s ( m i n i m u m n e e d e d ) .
| G r e e d y A l g o r i t h m : L a r g e s t C o m m o n
D i m e n s i o n F i r s t
e g . 2 :
A
1 x 2
X B
2 x 3
X C
3 x 4
X D
4 x 5
X E
5 x 6
| L a r g e s t C o m m o n D i m e n s i o n F i r s t i m p o s e s
f o l l o w i n g o r d e r :
( A X ( B X ( C X ( D X E ) ) ) )
w h i c h n e e d s 2 4 0 m u l t i p l i c a t i o n s .
4
-
8/9/2019 Devide and Conquar
5/24
| B e s t o r d e r :
( ( ( ( A X B ) X C ) X D ) X E )
w h i c h n e e d s 6 8 m u l t i p l i c a t i o n s .
| A n o t h e r G r e e d y A l g o r i t h m : S m a l l -
e s t C o m m o n D i m e n s i o n F i r s t b u t d i d
n o t w o r k f o r e g . 1 .
5
-
8/9/2019 Devide and Conquar
6/24
4 . 2 D i v i d e a n d C o n q u e r S o l u t i o n
I n p u t :
A
1
* A
2
: : : * A
n
d
0
d
1
d
1
d
2
: : : d
n ; 1
d
n
O u t p u t : A p a r a n t h e s i z a t i o n o f t h e i n p u t s e q u e n c e
r e s u l t i n g i n m i n i m u m n u m b e r o f m u l t i p l i c a t i o n s
n e e d e d t o m u l t i p l y t h e n m a t r i c e s .
S u b g o a l : I g n o r e S t r u c t u r e o f O u t p u t ( o r d e r
o f p a r e n t h e s i z a t i o n ) , f o c u s o n o b t a i n i n g a n u -
m e r i c a l s o l u t i o n ( m i n i m u m n u m b e r o f m u l t i p l i -
c a t i o n s )
D e n e M i , j ] = t h e m i n i m u m n u m b e r o f m u l t i -
p l i c a t i o n s n e e d e d t o c o m p u t e
A
i
A
i + 1
A
j
f o r i j n
S u b g o a l i s t o o b t a i n M 1 n ]
6
-
8/9/2019 Devide and Conquar
7/24
e . g . F o r ,
A 1
3 0 x 1
X A 2
1 x 4 0
X A 3
4 0 x 1 0
X A 4
1 0 x 2 5
X A 5
2 5 x 1
M 1 1 ] = 0 M 1 2 ] = 1 2 0 0 M 1 5 ] = 6 9 0
7
-
8/9/2019 Devide and Conquar
8/24
4 . 3 R e c u r s i v e F o r m u l a t i o n o f M i j ]
A 1 X A 2 = ( A 1 ) X ( A 2 )
| P a r t i t i o n a t k = 1 : S u b p r o b l e m s ( A 1 ) a n d ( A 2 )
| c o s t o f ( A 1 ) i s M 1 1 ] a n d t h a t o f ( A 2 ) i s
M 2 2 ]
| c o s t o f c o m b i n i n g ( A 1 ) a n d ( A 2 ) i n t o o n e i s
d
0
d
1
d 2
| M 1 2 ] = M 1 1 ] + M 2 2 ] + d
0
d
1
d 2
| M 1 2 ] = 0 + 0 + 1 2 0 0 = 1 2 0 0 .
A 2 X A 3 X A 4 = ( A 2 ) X ( A 3 X A 4 ) ( k = 2 )
O r , = ( A 2 X A 3 ) X A 4 ( k = 3 )
| k = 2 : c o s t = M 2 2 ] + M 3 4 ] + d 1 d 2 d 4
| k = 3 : c o s t = M 2 3 ] + M 4 4 ] + d 1 d 3 d 4
| M 2 4 ] = m i n ( M 2 2 ] + M 3 4 ] + d 1 d 2
d 4 M 2 3 ] + M 4 4 ] + d 1 d 3 d 4 )
8
-
8/9/2019 Devide and Conquar
9/24
I n s h o r t , M 2 4 ] =
m i n
2 k 3
( M 2 k ] + M k 4 ] + d
1
d
k
d
4
)
A 2 X A 3 X A 4 X A 5
= ( A 2 ) X ( A 3 X A 4 X A 5 ) ( k = 2 )
O r , = ( A 2 X A 3 ) X ( A 4 X A 5 ) ( k = 3 )
O r , = ( A 2 X A 3 X A 4 ) X ( A 5 ) ( k = 4 )
M 2 5 ] = ( M 2 2 ] + M 3 5 ] + d
1
d
2
d
5
M 2 3 ] +
M 4 5 ] + d
1
d
3
d
5
M 2 4 ] + M 5 5 ] + d
1
d
4
d
5
)
I n g e n e r a l , b y f a c t o r i n g ( A
i
A
i + 1
A
j
)
a t k t h i n d e x p o s i t i o n i n t o ( A
i
A
i + 1
A
k
)
a n d ( A
k + 1
A
j
) n e e d M i k ] + M k + 1 j ]
m u l t i p l i c a t i o n s a n d c r e a t e s m a t r i c e s o f d i m e n -
s i o n s d
i ; 1
d
k
a n d d
k
d
j
. T h e s e t w o m a t r i c e s
n e e d a d d i t i o n a l d
i ; 1
d
k
d
j
m u l t i p l i c a t i o n s t o
c o m b i n e .
9
-
8/9/2019 Devide and Conquar
10/24
4 . 4 R e c u r s i v e F o r m u l a a n d T i m e T a k e n
R e c u r s i v e l y ,
M i j ] =
m i n
i k j ; 1
M i k ] + M k + 1 j ] + d
i ; 1
d
k
d
j
!
M ( i i ) = 0
O p t i m a l S u b s t r u c t u r e
1 0
-
8/9/2019 Devide and Conquar
11/24
W e c a n r e c u r s i v e l y s o l v e f o r
M 1 n ] =
m i n
1 k n ; 1
( M 1 k ] + M k + 1 n ] + d
0
d
k
d
n
)
= m i n M 1 1 ] + M 2 n ] + d
0
d
1
d
n
,
M 1 2 ] + M 3 n ] + d
0
d
2
d
n
,
M 1 3 ] + M 4 n ] + d
0
d
3
d
n
,
M 1 n ; 1 ] + M n n ] + d
0
d
n ; 1
d
n
T i m e C o m p l e x i t y :
T
n
= n + T
1
+ T
n ; 1
+ T
2
+ T
n ; 2
+ T
3
+ T
n ; 3
+ T
n ; 2
+ T
2
+ T
n ; 1
+ T
1
1 1
-
8/9/2019 Devide and Conquar
12/24
T
n
= n + 2 T
1
+ 2 T
2
+ + 2 T
n ; 1
( 1 )
T
n ; 1
= n ; 1 + 2 T
1
+ 2 T
2
+ + 2 T
n ; 2
( 2 )
S u b t r a c t i n g ( I ) - ( I I ) y i e l d s
T
n
; T
n ; 1
= 1 + 2 T
n ; 1
T
n
= 1 + 3 T
n ; 1
= 1 + 3 ( 1 + 3 T
n ; 2
)
T
n
= 1 + 3 + 3
2
+ 3
3
+ + 3
n ; 1
T
1
= 1 + 3 + 3
2
+ 3
3
+ + 3
n ; 2
1 2
-
8/9/2019 Devide and Conquar
13/24
R e c u r s i v e S o l u t i o n i s e x p o n e n t i a l t i m e ( 3
n ; 2
)
S p a c e O ( n ) s t a c k d e p t h .
O v e r l a p p i n g s u b p r o b l e m s : e . g . R e c u r s i o n
t r e e f o r M 1 4 ]
2 6 r e c u r s i v e c a l l s f o r j u s t 1 0 s u b p r o b l e m s
M 1 1 ] M 2 2 ] M 3 3 ] M 4 4 ] M 1 2 ] M 2 3
1 3
-
8/9/2019 Devide and Conquar
14/24
S o w e t u r n t o d y n a m i c P r o g r a m m i n g ,
t h e s a m e f o r m u l a t i o n
a p p r o a c h t h e p r o b l e m b o t t o m - t o - t o p
n d a s u i t a b l e t a b l e t o s t o r e t h e s u b - s o l u t i o n s .
H o w m a n y s u b - s o l u t i o n s d o w e h a v e ?
M 1 1 ] M 2 2 ] M 3 3 ] M n n ] n
M 1 2 ] M 2 3 ] M n ; 1 n ] n ; 1
M 1 3 ] M 2 4 ] M n ; 2 n ] n ; 2
M 1 n ; 1 ] M 2 n ] 2
M 1 n ] 1
n ( n ; 1 )
2
) w e n e e d O ( n
2
) s p a c e
1 4
-
8/9/2019 Devide and Conquar
15/24
4 . 5 M a t r i x P a r e n t h e s i z a t i o n O r d e r
M , F a c t o r : M a t r i x
f o r i 1 t o n d o M i i ] 0
/ * m a i n d i a g o n a l * /
f o r d i a g o n a l 1 t o n ; 1 d o
f o r i 1 t o n ; d i a g o n a l d o
j = i + d i a g o n a l
M i j ] = m i n
i k j ; 1
( M i k ] + M k + 1 j ]
+ d
i ; 1
d
k
d
j
)
F a c t o r i j ] = k t h a t g a v e t h e m i n i m u m v a l u e
f o r M i j ]
e n d f o r
e n d f o r
1 5
-
8/9/2019 Devide and Conquar
16/24
4 . 6 W o r k o u t
A 1
3 0 x 1
X A 2
1 x 4 0
X A 3
4 0 x 1 0
X A 4
1 0 x 2 5
X A 5
2 5 x 1
M 1 2 ] = m i n
1 k 1
M i k ] + M k + 1 j ] +
d
i ; 1
d
k
d
j
]
= m i n M 1 1 ] + M 2 2 ] + d
0
d
1
d
2
]
= 0 + 0 + 3 0 1 4 0
= 1 2 0 0
M 1 3 ] = m i n
1 k 3 ; 1
M i k ] + M k + 1 j ] +
d
i ; 1
d
k
d
j
]
= m i n M 1 1 ] + M 2 3 ] + d
0
d
1
d
3
M 1 2 ] + M 3 3 ] + d
0
d
2
d
3
]
= m i n 0 + 4 0 0 + 3 0 1 1 0
1 2 0 0 + 0 + 3 0 4 0 1 0 ]
= m i n 7 0 0 1 2 0 0 0 + 1 2 0 0 ]
= 7 0 0
1 6
-
8/9/2019 Devide and Conquar
17/24
M 2 4 ] = m i n M i k ] + M k + 1 j ] + d
i ; 1
d
k
d
j
]
2 k 3
= m i n M 2 2 ] + M 3 4 ] + d
1
d
2
d
4
M 2 3 ] + M 4 4 ] + d
1
d
3
d
4
]
= m i n 0 + 1 0 0 0 0 + 1 4 0 2 5 4 0 0 + 0 + 1 1 0 2 5 ]
= m i n 1 0 1 0 0 6 5 0 ] = 6 5 0
M 3 5 ] = m i n M i k ] + M k + 1 j ] + d
i ; 1
d
k
d
j
]
3 k 4
= m i n M 3 3 ] + M 4 5 ] + d
2
d
3
d
5
M 3 4 ] + M 5 5 ] + d
2
d
4
d
5
]
= m i n 0 + 2 5 0 + 4 0 1 0 1 1 0 0 0 0 + 0 + 4 0 2 5 1 ]
= m i n 6 5 0 ; ] = 6 5 0
M 1 4 ] = m i n M 1 1 ] + M 2 4 ] + d
0
d
1
d
4
M 1 2 ] + M 3 4 ] + d
0
d
2
d
4
M 1 3 ] + M 4 4 ] + d
0
d
3
d
4
]
= m i n 0 + 6 5 0 + 3 0 1 2 5
1 2 0 0 + 1 0 0 0 0 + 3 0 4 0 2 5
7 0 0 + 0 + 3 0 1 0 2 5 ]
= m i n 1 4 0 0 ; ; ] = 1 4 0 0
1 7
-
8/9/2019 Devide and Conquar
18/24
1 8
-
8/9/2019 Devide and Conquar
19/24
5 D Y N A M I C P R O G R A M M I N G R E Q U I R E M E N T S
R e q u i r e m e n t s : a ) O p t i m a l S u b s t r u c t u r e
b ) O v e r l a p p i n g s u b p r o b l e m
S t e p s : 1 ) C h a r a c t e r i z e t h e s t r u c t u r e o f a n o p t i -
m a l s o l u t i o n
2 ) F o r m u l a t e a r e c u r s i v e s o l u t i o n
3 ) C o m p u t e t h e v a l u e o f a n o p t . s o l u t i o n
b o t t o m - u p . ( g e t v a l u e r a t h e r t h a n t h e s t r u c -
t u r e )
4 ) C o n s t r u c t a n o p t i m a l s o l u t i o n ( s t r u c t u r e )
f r o m c o m p u t e d i n f o m a t i o n .
M e m o i z a t i o n : T o p - d o w n , c o m p u t e a n d s t o r e r s t
t i m e , r e u s e s u b s e q u e n t t i m e s .
1 9
-
8/9/2019 Devide and Conquar
20/24
5 . 1 O b s e r v a t i o n 1 : O p t i m a l S u b s t r u c t u r e
T h e o p t i m a l s o l u t i o n c o n t a i n i n g s o p t i m a l s u b s o l u -
t i o n s .
R e c u r s i o n T r e e ( d o n o t w i d e y e t )
D e p t h ? ( m + n )
o u t d e g r e e 3 ) n u m b e r o f n o d e s a m o u n t o f w o r k
i n r e c u r s i v e c a l l s i s ( 3
m + n
)
2 0
-
8/9/2019 Devide and Conquar
21/24
5 . 2
O b s e r v a t i o n 2 : O v e r l a p p i n g S u b p r o b l e m s
w i d e s o m e r e p e a t e d p r o b l e m s , a s a b o v e .
a f e w p r o b l e m s , b u t m a n y r e c u r s i v e i n s t a n c e s
u n l i k e g o o d d i v i d e - a n d - c o n q u e r w h e r e p r o b l e m s
a r e i n d e p e n d e n t .
L C S h a s a n m n d i s t i n c t p r o b l e m s .
2 1
-
8/9/2019 Devide and Conquar
22/24
5 . 3 M e m o r i z e
( t o d e a l w i t h o v e r l a p p i n g p r o b l e m s )
a f t e r c o m p u t i n g s o l u t i o n t o a s u b p r o b l e m , s o r t
i n a t a b l e . S u b s e q u e n t c a l l - d o t a b l e l o o k u p .
T i m e O ( m n )
| e a c h p r o b l e m i s s o l v e d o n c e a n d u s e d t w i c e .
| s e e i n g u r e f o r 1 2 o r 2 3
| m i g h t n o t n e e d t o s o l v e a l l s u b p r o b l e m , o n l y
t h o s e n e e d e d .
2 2
-
8/9/2019 Devide and Conquar
23/24
5 . 4 D y n a m i c P r o g r a m m i n g I m p l i m e n t a t i o n
C o m p u t e t a b l e b o t t o m - u p i n s t e a d o f s t a r t i n g a t
( m n ) , s t a r t a t ( 1 1 )
D e m o n s t r a t e a l g o r i t h m
| t i m e = ( m n )
| s p a c e = ( m i n ( m n ) )
I n i t i a l i z e t o p r o w & l e f t c o l u m n t o 0
2 3
-
8/9/2019 Devide and Conquar
24/24
P r o d u c e f r o m t o p r o w , l e f t t o r i g h t , x i ] = y j ]
l l d i a g o n a l n e i g h b o r + 1 & c h a w . s l r e l l m a x
o f t h e o t h e r n e i g h b o r s .
2 4