統計解析 19 09 - keio universityweb.sfc.keio.ac.jp/~maunz/dsb19/dsb19_09.pdf=ln 1 2nf& 4...
Post on 19-Jan-2020
1 Views
Preview:
TRANSCRIPT
Ni
c
( )
(
, )
* v a~ x b
* x v b
- .
( ) (
i c j
g
•• a
•• d a i
a a jaa a
• d a g a• g d9F T a NIL
••
•
u L•• TV u d a
a• s• wd a• s wd7 VI a
•• C D d g• s w d a
• g• 9F T a NIL
• i j•• NIL• 9F T
•
•
•−2.5
0.0
2.5
5.0
−3 −2 −1 0 1 2 3X
Y
a i• ! " #$, #&, … , #( )
x * = 1,… , )
!$ = -. + -$#$$ + ⋯+ -(#$( + 1$!& = -. + -$#&$ + ⋯+ -(#&( + 1&
⋮!3 = -. + -$#3$ + ⋯+ -(#3( + 13
⋮!4 = -. + -$#4$ + ⋯+ -(#4( + 14
a i• x w a
5 =
!$!&⋮!3⋮!4
, 6 =
1 #$$ ⋯1⋮1
#&$⋮#3$
⋯⋱⋯
⋮1
⋮#4$
⋱⋯
#$(#&(⋮#3(⋮#4(
, 8 =
-.-$-&⋮-(
, 9 =
1$1&⋮13⋮14
• x5 = 68 + 9v w
!3 = 63:8 + 13
a i• 9 = 5 − 68 < a
< = 5 − 68 : 5 − 68 = 5:5 − 28:6:65 + 8:6:68• a
><>8
= −26:5 + 26:68 = 0
•
6:68 = 6:5
• 6:6 @$ a A8
A8 = 6:6 @$6:5
i d s lt• ax ~ w
• 6 Ba Σ x6~E B, Σ a68 + 9~E 68 + 9, 8Σ8:
• 9 0a F& x9~E 0, F&G~ w b
• 5 68a F&G x
5 = 68 + 9~E 68, F&G
a• 8c 9c 5 aA8c H5c I x
•A8 = 6:6 @$6:5~E 8, F& 6:6 @$
•H5 = 6A8 = 6 6:6 @$6:5 = J5~E 68, F&J
J = 6 6:6 @$6:
•I = 5 − H5 = G − J 5~E 0, F& G − J
J
a• x
A8~E 8, F& 6:6 @$
• a x
A8 − 8
F& 6:6 @$~E 0, 1
Rg• g 6a g 8a F& y
a 5 wx• * #3 = 1, #3$, … , #3( a !3
a x
K !3|#3; 8, F& =1
2NF&O#K −
!3 − #38 &
2F&
Rg• z
•P !3|#3; 8, F& = #38
•Q !3|#3; 8, F& = F&
E yi | xi ;β,σ2( ) = xiβV yi | xi ;β,σ
2( ) =σ 2
Rg• * w
K 5|6; 8, F& =R3S$
4
K !3|#3; 8, F&
=R3S$
41
2NF&O#K −
!3 − #38 &
2F&
=1
2NF&
4
O#K −∑3S$4 !3 − #38 &
2F&
og Rg•
ln K 5|6; 8, F& = ln R3S$
4
K !3|#3; 8, F&
= ln1
2NF&
4
O#K −∑3S$4 !3 − #38 &
2F&
= −)2ln2N −
)2lnF& −
12F&
W3S$
4!3 − #38 &
e•
g
• a
W3S$
4!3 − #38 &
yi − xiβ( )2
i=1
n∑
-4 -2 0 2 4
-12
-10
-8-6
-4-2
Values of parameter
Log
likel
ihoo
d
山登り
対数尤度関数が最大となる点を点推定
e N• g
xW
3S$
4!3 − #38 &
•A8 =
∑3S$4 #3!3∑3S$4 #3
&
XF& =1
) − " + 1W3S$
4
!3 − #3A8&
/ ) − (" + 1)
Rg E• g 6a g 8a F& y a
! a ax
K 5|6; 8, F& =1
2NF&O#K −
5 − 68 &
2F&
Rg E• z
•P 5|6; 8, F& = 68
•Q 5|6; 8, F& = F&
E y | X ;β,σ 2( ) = Xβ
Rg E• g w
K 5|6; 8, F& =1
2NF&O#K −
5 − 68 &
2F&
=1
2NF&O#K −
5 − 68 : 5 − 682F&
og Rg E• g w
ln K 5|6; 8, F& = ln1
2NF&O#K −
5 − 68 : 5 − 682F&
= −)2ln2N −
)2lnF& −
5 − 68 : 5 − 682F&
e N E• r
A8 = 6:6 @$6:5
XF& =1[5 − 6A8
:5 − 6A8
[ = ) − " + 1 hp y
e N z• A8 6:6 @$ ~
wA8 = 6:6 @$6:5
• 6 w a 6:6 @$ ~ w• ) < " i w j• w v• w v a
E• k " y ln g " w
g
• 183 a 2( NP a
• a g w i jw
• _ ~ x wx
vy
( s
• a ~ w
r• _
wx• y x
_ =
11
1
g• _• ak w l
wwa rank _ v
o
• w a g~ a
• w w• w• w• 6:6 @$ x ok l wx
nL ) ( (• 6:6 @$ x a x 6:6
~ w• y a g c G(d$ x w
a8e = 6:6 + cG(d$ @$6:5
• a fe 8 x8 w
fe 8 = ! − 68 : ! − 68 + c8:8
n• a )×) _wx
_h = G4 = h_
• aG4 )×) ah )×)
ij nL• y
5 = 68 + 9
• x a ij ~ wx
fe 8 = 5 − 68 & + c 8 k = 5 − 68 : 5 − 68 + cWlS$
(
-lk
• c 8 k = c∑lS$( -l
k wxim ≥ 0, c ≥ 0j• c HTR NX] FVFR X Vb w i
j a w• fe 8 a 8
ij nL• ij ~ fe 8 8
a w
arg min8∈ℝtuv
fe 8 = arg min8∈ℝtuv
5 − 68 & + c 8 k
= arg min8∈ℝtuv
5 − 68 : 5 − 68 + cWlS$
(
-lk
= arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
+ cWlS$
(
-lk
arg min
• max x # d x #• arg max x # d x # #
• min x # d x #• arg min x # d x # #
• d arg min
ik nL• ik ~ fe 8 a
arg min8∈ℝtuv
fe 8 = arg min8∈ℝtuv
! − 68 & + c 8 k
• m = 2 NIL• m = 1 9F T• 0 < y < 1 y w c 8 kx y a5 F XNH S X 5 y
cWlS$
(
2y -l + 1 − y -l&
(• ik ~ fe 8
arg min8∈ℝtuv
fe 8 = arg min8∈ℝtuv
! − 68 & + c 8 k
• NIL m = 2 • m
• 9F T m = 1 • m
• 5 F XNH S X 0 < y < 1 • m m
i& nL (• m = 2 fe 8 x
A8z{|}~ = arg min8∈ℝtuv
fe 8
= arg min8∈ℝtuv
! − 68 & + c 8 &
= arg min8∈ℝtuv
! − 68 : ! − 68 + cWlS$
(
-l&
= arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
+ cWlS$
(
-l&
i& nL (• NIL A8z{|}~ a
v
A8z{|}~ = arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å.WlS$
(
-l& ≤ Å
9
i& nL (• x ~
A8z{|}~ = arg min8∈ℝtuv
fe 8 = arg min8∈ℝtuv
5 − 68 : 5 − 68 + c8:8
• a NIL g
A8z{|}~ = 6:6 + cG(d$ @$6:5
i& nL (•
fe 8 = 5 − 68 & + c 8 &= 5 − 68 : ! − 68 + c8:8= 5:5 − 28:É:5 + 8:É:É8 + c8:8
• 8fe 8>8
= −2É:5 + 2É:É8 + 2c8
• ÑÖ 8Ü8
= 0A8z{|}~ = 6:6 + cG(d$ @$6:5
i& nL (• A8z{|}~
A8z{|}~ = arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å.WlS$
(
-l& ≤ Å
• Å -l x
• i& w c w ag
0 500 1000 1500 2000
−200
020
040
0
L1 Norm
Coe
ffici
ents
10 10 10 10 10
1
2
3
4
5
6
7
8
9
10
Å = WlS$
(
-l&
-l
0 500 1000 1500 2000
−200
020
040
0
L1 Norm
Coe
ffici
ents
10 10 10 10 10
1
2
3
4
5
6
7
8
9
10
i& nL (• i& w c w a g
2 4 6 8 10
−200
020
040
0
Log Lambda
Coe
ffici
ents
10 10 10 10 10
1
2
3
4
5
6
7
8
9
10-l
i& nL (• fe 8
arg min8∈ℝtuv
! − 68 & + c 8 &
• c g 3A : 5i jx w
2 4 6 8 10
3000
3500
4000
4500
5000
5500
6000
log(Lambda)
Mea
n−Sq
uare
d Er
ror
10 10 10 10 10 10 10 10 10 10 10 10 10
2 4 6 8 10
−200
020
040
0
Log Lambda
Coe
ffici
ents
10 10 10 10 10
1
2
3
4
5
6
7
8
9
10
i& nL (• g 3A : 5i j
c -l ”
i$ nL• m = 1 fe 8 x
A8áàââä = arg min8∈ℝtuv
fe 8
= arg min8∈ℝtuv
! − 68 & + c 8 $
= arg min8∈ℝtuv
! − 68 : ! − 68 + cWlS$
(
-l
= arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
+ cWlS$
(
-l
i$ nL• 9F T A8áàââä a
v
A8áàââä = arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å. WlS$
(
-l ≤ Å
i$ nL• 9F T A8áàââä
A8áàââä = arg min8∈ℝtuv
W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å. WlS$
(
-l ≤ Å
• Å ≥
• ã- ~a Å. =∑lS$( ã-l a Å < Å.
(• NIL 9F T a 8 k
ww
• w g -$ -& y a 8 kx
• 9F T d -$ + -& ≤ Åi j• NIL d -$
& + -&& ≤ Åi j
(• ã- = ã-$, ã-&• ã-z -$ -&
a x
9F T NIL
8 $ = -$ + -& ≤ Å 8 & = -$& + -&
& ≤ Å
7F XN X F ,
! − 68 & ! − 68 &
fe 8
8 k
• 8 k = ∑lS$( -l
k
x
• 5 F XNH S X c ∑lS$( 1 − y -l& + 2 -l x
7F XN X F ,
i$ nL• i$ w e
• i$ x 8 s go w ~
• f w ay
i$ nL• 9F T A8áàââä
min8∈ℝtuv
12W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å. WlS$
(
-l ≤ Å
• a #3l a w
i$ nL• a 9F T A8áàââä a
x -
x - =12W3S$
4
!3 −-. −WlS$
(
-l#3l
&
+ åWlS$
(
-l
~a å å ≥ 0
i$ nL• ã-
• a x - A8áàââä å ã- åw x
A8áàââä å = sin ã- ã- − åd
i$ nL• c 8 $ 8 w a x
w 8 w
• 3TTVINSFX I 4 H SX 1 LTVNXMR• MXX / [[[ O XFX T X TVL FVXNH N [ ((N
• 9 F X 1SL LV NTS 91 • MXX / VTO HX H NI TVL H NI FT -( ,-.(
• 8X VFXN MVNSPFL MV MT INSL 1 LTVN R 8 1 • MXX / T V SS NSVNF V 3 IVNH 7 V X 3 IVNH 7 V X FV E RNSF
V 5SXV E1E6F XE8X VFXN E MVNSPFLMV MT INSLE1 LTVNXMR TVE9NS FVE8S V E=VT R E 1 E2 HP E: E T E N 2V HPE . I
• 1 X VSFXNSL 4NV HXNTS : XMTI T : XN N V 14:: • MXX / [ XFS TVI I _ T]I F V I FIRREIN XVE XFX I
i$ nL• 9F T A8áàââä
min8∈ℝtuv
12W3S$
4
!3 −-. −WlS$
(
-l#3l
&
�. Å. WlS$
(
-l ≤ Å
• Å -lx
• i$ w c w ag
0 500 1000 1500 2000 2500 3000
−600
−400
−200
020
040
060
0
L1 Norm
Coe
ffici
ents
0 2 4 6 8 10 10
1
2
3
4
5
6
7
8
9
10
-l
Å = WlS$
(
-l
−4 −2 0 2 4
−600
−400
−200
020
040
060
0
Log Lambda
Coe
ffici
ents
10 10 7 4 0
1
2
3
4
5
6
7
8
9
10
0 500 1000 1500 2000 2500 3000
−600
−400
−200
020
040
060
0
L1 Norm
Coe
ffici
ents
0 2 4 6 8 10 10
1
2
3
4
5
6
7
8
9
10-l
i$ nL• i$ w c w a g
i$ nL• fe 8
A8áàââä = arg min8∈ℝtuv
! − 68 & + c 8 $
• c g 3A : 5i j
x w−4 −2 0 2 4
3000
3500
4000
4500
5000
5500
6000
log(Lambda)
Mea
n−Sq
uare
d Er
ror
10 10 10 9 10 8 8 8 7 7 7 5 4 4 3 2 0
i$ nL• g 3A : 5i j
c -l ”
−4 −2 0 2 4
−600
−400
−200
020
040
060
0
Log Lambda
Coe
ffici
ents
10 10 7 4 0
1
2
3
4
5
6
7
8
9
10
i• FV g v INF X g
• MXX / [ XFS TVI I _MF XN XFX9 FVS FV NX]E N 41 1 INF X MXR• )) g• i jdFL 2:8 2TI] RF NSI RF
• d
• XH e g • I 949 g d 0 g • MI 749 g d 0 g • XHM g • XL d • L d g
• g ] dt• Bd g
top related