manifold learning on probabilistic graphical models 概率图上的流形学习
DESCRIPTION
Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习. 答辩人 : 邵元龙 导师 : 鲍虎军 教授 & 何晓飞 教授 浙江大学 CAD&CG 国家重点实验室 2010 年 3 月 5 日. Outline. Background & Motivation Function Learning v.s. Statistical Modeling Manifold Regularized Variational Inference Algorithm Design & Examples - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/1.jpg)
Manifold Learning on Probabilistic Graphical Models概率图上的流形学习
答辩人 : 邵元龙导师 : 鲍虎军 教授 & 何晓飞 教授浙江大学 CAD&CG 国家重点实验室2010 年 3 月 5 日
![Page 2: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/2.jpg)
Outline
Background & Motivation Function Learning v.s. Statistical Modeling
Manifold Regularized Variational Inference Algorithm Design & Examples In Depth Analysis Implementation Experimental Results
![Page 3: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/3.jpg)
Function Learning Given data points , and a function space , find the optimal function , such that
Regularization is Important!!
N 1,
N
i i ix y
2*
1
arg min ,N
i if i
f V y f x f
F*f F
![Page 4: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/4.jpg)
Statistical Modeling All quantities, no matter given or to be
estimated, are random variables. Then we model the joint distribution.
, ,,|
, ,
p dpp
p p d d
H V Θ θ θH V
H VV H h V Θ θ h θ
![Page 5: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/5.jpg)
e.g. Gaussian Mixture Model
![Page 6: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/6.jpg)
Difficulties
How many components are there? Should there be any “components” ?
![Page 7: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/7.jpg)
Difficulties (continued)
What if data reside on a non-trivial manifold
![Page 8: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/8.jpg)
Efforts towards Non-Parametric, but …
![Page 9: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/9.jpg)
What we want…
![Page 10: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/10.jpg)
Review GMM
Function Learning embedded. :f X Z
![Page 11: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/11.jpg)
Problem Formulation
What to regularize? Where to regularize?
![Page 12: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/12.jpg)
Manifold Learning
![Page 13: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/13.jpg)
Manifold Assumption Y changes smoothly with X, and we have
so should be small over manifold Minimizing it over the manifold,
f x x f x
f xx
f x
2minx
f dx
MM
![Page 14: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/14.jpg)
Manifold Regularization
2
2
2,
min
1x
ij i ji j
f dx
S f x f xN
S
MM
![Page 15: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/15.jpg)
Manifold Regularization
2
2
2,
2
2,
min
1
1
x
iji
i
j
ij j
i
j
j
i
f dx
f x f xSN
S yN
y
S
S
MM
Transductive Learning
![Page 16: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/16.jpg)
Problem Formulation
What to regularize? Where to regularize?
![Page 17: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/17.jpg)
Variational Inference For , define , a var. dist.
Approximate the true posterior with it
by minimizing the KL divergence
H H q H
|H
q q H p
H
H H V
* arg min || |q
q KL q pH
H H H V
![Page 18: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/18.jpg)
Manifold Regularized Variational Inference
* arg minq
q H
H F
2
,
|| |
,ij i ji j
KL q p
S d q Z q Z
H H V
S
F
![Page 19: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/19.jpg)
How to Optimize?
* arg minH
H q Hq H
H
HF
2
,
|| |
,ij i ji j
KL q p
S d q Z q Z
H H V
S
F
![Page 20: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/20.jpg)
Optimization Algorithm
* arg mini
iii
q ZZ Zq Z F
2
,: ,
|| | , 2 ,ii iZ Zi i Z ij i j
j i j j i
KL q Z p Z S d q Z q Z
MB S
F
1
*
for ,...,
arg min || |N
Hq H
H Z Z
q H KL q H p H
MB
![Page 21: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/21.jpg)
An Illustration
0.2 2 200
![Page 22: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/22.jpg)
Works Done
Example Distribution Types Convergence Proof Convexity Analysis (More TODO) Computational Complexity Numerical Stability A Flexible Inference Engine
![Page 23: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/23.jpg)
YASIE (Yet Another Statistical Inference Engine)
Interface Design Inference Scheduling Type-Free Mixture Model Design Issues (e.g. Balance of Memory & Comp.
Time)
![Page 24: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/24.jpg)
Experiments
Data Clustering Gaussian Mixture Model
Image Annotation Link Mixture of Unigram
![Page 25: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/25.jpg)
Image Annotation Model Link Mixture of Unigram
![Page 26: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/26.jpg)
Image Similarity Graph
“?” should be something like “Barcelona”
?
![Page 27: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/27.jpg)
Image Annotation Performances
![Page 28: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/28.jpg)
Image Annotation Examples
![Page 29: Manifold Learning on Probabilistic Graphical Models 概率图上的流形学习](https://reader033.vdocuments.mx/reader033/viewer/2022061323/56816750550346895ddc01bc/html5/thumbnails/29.jpg)
Any Question? 实验室的老师们:鲍虎军老师,何晓飞老师,蔡登老师,刘新国老师,章国锋老师,黄劲老师…… 师兄师弟师妹们:董子龙,姜翰青,周源,张驰原,林斌斌,薛维,瞿新泉,姚冠红……感谢你们一直以来给我的帮助!