cs262 problem session - stanford university · 2016. 1. 12. · !"#$%&'(f*1,*:, (! %...
TRANSCRIPT
![Page 1: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/1.jpg)
!"#$%&'()
! *+,(-./012(34&(56'(#7(8&9:4;(#7(#6;:#9<:(&=:&5('65;(56'(;#(#<&2(><?#'9<:(&=:&5@(56'(4+5(<#("&5;"9?;9#<52
! *$,(-./012(34&(&=:&5(*5AB;,(=&<#;&5(;4&(C"#$+$9%9;D(!(:9E&<("2
! *?,(-./012(!*FG(H,(A(!*H),!*F
)IH
),!*H
JIH
),222
! *=,(3KL12(M9;&"$9(=#&5(N.O(#7(<6'$&"5(849%&(-#"8+"=(=#&5(0LN(#7(<6'$&"52(09<?&(+%%(<6'$&"5(+"&(<#<P<&:+;9E&G(;4&(56'(95(:"&+;&"(;4+<(#"(&Q6+%(;#(;4&('+F2
! *&,(3KL12(34&D(+"&($#;4(R*STSTU,(84&"&(S(A(V(5;+;&5(+<=(U(A(5&Q6&<?&(%&<:;42
! *7,(-./012(>;(:9E&5(;4&(C"#$+$9%9;D(=95;"9$6;9#<(#E&"(5;+;&5(+;(&+?4(C#59;9#<2! *:,(-./012(M+<9%%+(WNN5(+"&(:##=(7#"(9<;"#<9?(+<=(<#;(5#(:##=(7#"(&F#<9?("&:9#<52(R<%D(;4&(9<;"#<9?("&:9#<5@(%&<:;4(5&&'5(;#(4+E&(+(=95;"9$6;9#<(;4+;(95(<&+"%D(:&#'&;"9?2
! *4,(-./012(X&(?+<(65&(49:4&"P#"=&"(WNN5(;#('#=&%(*?#<=9;9#<(#<,(;8#(#"('#"&(C"&E9#65(;9'&5;&C52
! *9,(-./012(><=&&=(1N(95(:6+"+<;&&=(;#(?#<E&":&(;#(+(5#%6;9#<G(;4#6:4(<#;(<&?&55+"9%D(;4&(:%#$+%%D(#C;9'+%(#<&2
!
![Page 2: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/2.jpg)
Problem 2(a)
00
00101 11
iki
iki
klkl
i
lilk
i
lilklkkl
AAAa
xPxPibxeif
xPibxeaifA
![Page 3: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/3.jpg)
Problem 2(b) • Baum‐Welch: Suppose Forward: Similar for Backward
bebeaaaa kkkllklkkl ,,
ifaifxe
aifxeif
kl
kllik
llklikk
1
1
![Page 4: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/4.jpg)
Problem 2(b) • Baum‐Welch:
bEibifibifbE
AxP
ibxeaifxP
ibxeaifA
AxP
ibxeaifxP
ibxeaifA
kbxi
kkbxi
kkk
kli
kikkll
i
kiklkllk
lki
lillkk
i
lilklkkl
ii
11
11
11
11
![Page 5: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/5.jpg)
Problem 2(b) • Baum‐Welch:
becE
bEcE
bEbe
aA
AA
Aa
aA
AA
Aa
k
ck
k
ck
kk
kl
iki
kl
iik
lklk
lk
iik
lk
iki
klkl
![Page 6: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/6.jpg)
Problem 2(b) • Baum‐Welch: Given Inductive step: After training:
000000 ,, kkkllklkkl ebeaaaa
111111 ,,
,,
ik
ik
ikl
ilk
ilk
ikl
ik
ik
ikl
ilk
ilk
ikl
ebeaaaa
ebeaaaa
Nk
Nk
Nkl
Nlk
Nlk
Nkl ebeaaaa ,,
![Page 7: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/7.jpg)
Problem 2(b) • Viterbi:
Viterbi parse may arbitrarily choose state k over state
k’ Akl Ak’l a’kl a’k’l
xkP
xexexeaaaaa
xexexeaaaaaxkP
i
Nikkk
Nikkki
NNNii
NNNii
, where
, where
10
10
1111211
1111211
π
π
![Page 8: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/8.jpg)
Problem 2(c)
1P(x)=1P(y)=0
2P(x)=0P(y)=1
1/21 xxyx
1121
akl l=1 2
k=0 1 0
1 1/2 1/2
2 1 0
ek(b) b=x y
k=1 1 0
2 0 1
Akl l=1 2
k=0 1 0
1 1 1
2 1 0
Ek(b) b=x y
k=1 3 0
2 0 1
![Page 9: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/9.jpg)
Problem 2(c)
x y x x
0
1 .9 .045 .3645 .1640
2 0 .405 0 0
Viterbi
xxyx1121
akl l=1 2
k=0 1
1 1/2 1/2
2 1
ek(b) b=x y
k=1 1 0
2 0 1
![Page 10: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/10.jpg)
Problem 2(c) x y x x
0
1 .75 .1688 .1139 .0769
2 0 .0375 .0084 .0057
Viterbi xxyx1111
akl l=1 2
k=0 1
1 0.9 0.1
2 1
ek(b) b=x y
k=1 0.75 0.25
2 0.5 0.5
akl l=1 2
k=0 1
1 1 0
2 1
ek(b) b=x y
k=1 0.75 0.25
2 ? ?
![Page 11: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/11.jpg)
Problem 3(a)
most likely sequence of states π, given the observed sequence x
sequence of states π for which the observed sequence x was most likely emitted
![Page 12: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/12.jpg)
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
x = TTT
π = ?
Loaded Loaded Loaded? Fair Fair Fair?
![Page 13: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/13.jpg)
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
![Page 14: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/14.jpg)
Problem 3(a)
Fair P(H) = 0.5 P(T) = 0.5
Loaded P(H) = 0 P(T) = 1
1 – 10‐googol 10‐googol
1 – 10‐googol 10‐googol
10‐googol
1 – 10‐googol
![Page 15: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/15.jpg)
Problem 3(b)
πi* πi Win(πi*, πi)
F F +CF F L ‐WL
L F ‐WF
L L +CL
![Page 16: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/16.jpg)
Problem 3(b)
![Page 17: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/17.jpg)
Problem 3(c)(i) 1 2 3 …
fF[0]
fL[0]
fF[1]
fL[1]
fF[2]
fL[2]
fF[599]
fL[599]
fF[600]
fL[600]
: :
Fair Loaded
“generate first i characters of x with exactly k loaded rolls, ending in state Fair.”
“generate first i characters of x with exactly k loaded rolls, ending in state Loaded.”
![Page 18: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/18.jpg)
Problem 3(c)(i) … 998 999 1000
bF[0]
bL[0]
bF[1]
bL[1]
bF[2]
bL[2]
bF[599]
bL[599]
bF[600]
bL[600]
: :
Fair Loaded
“start state i in Fair, generate remaining characters of x with exactly k more loaded rolls.”
“start state i in Loaded, generate remaining characters of x with exactly k more loaded rolls.”
![Page 19: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/19.jpg)
Problem 3(c)(i) “generate first i characters of x with exactly k loaded rolls, ending in state Fair.”
“generate first i characters of x with exactly k loaded rolls, ending in state Loaded.”
“start state i in Fair, generate remaining x with exactly k more loaded rolls.”
“start state i in Loaded, generate remaining x with exactly k more loaded rolls.”
Running time? Sequence length S # loaded rolls R
![Page 20: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/20.jpg)
Problem 3(c)(ii)
Same as problem 3(b): Independently maximize expected payoff at each position.
![Page 21: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/21.jpg)
Problem 4(a)
x \ y A C G T –
A pm ps ps ps pd
C ps pm ps ps pd
G ps ps pm ps pd
T ps ps ps pm pd
– pd pd pd pd 0
MIJ
1
pair‐HMM NW
pm m
ps ‐s
pd ‐d
![Page 22: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/22.jpg)
Problem 4(b)
X prefix P(xi)
Y prefix P(yj)
X suffix P(xi)
Y suffix P(yj)
Overlap P(xi,yj)
start end τ
(1‐τ)/2
(1‐τ)/2
β
β β
β
α
τ
(1‐α‐τ)/2
(1‐α‐τ)/2 1‐β
1‐β
1‐β
1‐β
![Page 23: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/23.jpg)
Problem 4(c)
M P(xi,yj)
I1 P(xi)
I2 P(xi)
IL P(xi) δ
ε ε
1‐ε 1‐ε 1
J1 P(yj)
J2 P(yj)
JL P(yj) ε ε
δ
1‐ε 1‐ε 1
1‐2δ
![Page 24: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/24.jpg)
Problem 4(d)
k
γ(k)
d2
d3
e1
e2
e3
![Page 25: CS262 Problem Session - Stanford University · 2016. 1. 12. · !"#$%&'(f*1,*:, (! % ::< % ::: % 2777 %=!678 % = &678 % =!628 % = &628 % =!638 % = &638 % =!69::8 % = &69::8 % =!6;778](https://reader036.vdocuments.mx/reader036/viewer/2022071023/5fd76ad0e66f2626e435bea4/html5/thumbnails/25.jpg)
Problem 4(d)
M s(xi,yj)
I1 s(xi)
J1 s(yj)
1‐d1‐…‐ds‐d’1‐….‐d’s
e1
e’1
d1
I2 s(xi)
J2 s(yj)
e’2
d2
d’2
e2
Is s(xi)
es
Js s(yj)
e’s
ds
d’s
d’1
1‐e’1
1‐e’2 1‐e’s
1‐es 1‐e2
1‐e1