random swap em algorithm for gmm and image segmentation qinpei zhao, ville hautamäki, ismo...
TRANSCRIPT
![Page 1: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/1.jpg)
Random Swap EM algorithm for GMM and Image Segmentation
Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti
Speech & Image Processing UnitDepartment of Computer Science, University of Joensuu
Box 111, Fin-80101 JoensuuFINLAND
![Page 2: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/2.jpg)
Outline
Background & StatusRS-EMApplication
![Page 3: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/3.jpg)
Background: Mixture Model
![Page 4: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/4.jpg)
Background: EM algorithm
EM algorithm -> {α, Θ} E-step (Expectation):
M-step (Maximization):
Iterate E,M step until convergence α- mixing coefficient
Θ- model parameters, eg. {μ,∑}
( 1) ( 1)( , ) [log ( , | ) | , ]i iQ E p X Y X
( ) ( 1)argmax ( , )i iQ
![Page 5: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/5.jpg)
Local MaximaLet’s describe it as mountain climbing……
![Page 6: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/6.jpg)
600km
2160m 3099m
![Page 7: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/7.jpg)
![Page 8: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/8.jpg)
Initialization Effect
Initialization and Result(1) Initialization and Result(2)
![Page 9: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/9.jpg)
Sub-optimal Example
The situation of local maxima trap
![Page 10: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/10.jpg)
Status
Standard EM for Mixture Models(1977) Deterministic Annealing EM (DAEM) (1998) Split-Merge EM (SMEM) (2000) Greedy EM (2002) RS-EM coming…
![Page 11: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/11.jpg)
Outline
Background & StatusRS-EM (Random Swap)Application
![Page 12: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/12.jpg)
RSEM: Motivations Random manner Prevent from staying near the unstable or
hyperbolic fixed points of EM. Prevent from its stable fixed points corresponding
to insignificant local maxima of the likelihood function
Avoid the slow convergence of EM algorithm Less sensitive to its initialization
![Page 13: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/13.jpg)
Formulas SMEM
Greedy EM
RSEM
![Page 14: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/14.jpg)
Random Swap EMAfter EM
Afte
r Sw
ap
After EM
![Page 15: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/15.jpg)
Comparisons(1)
![Page 16: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/16.jpg)
Comparisons(2)
Q1 Q2 S1 S4
![Page 17: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/17.jpg)
Outline
Background & StatusRS-EMApplication
![Page 18: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/18.jpg)
Application Image Segmentation Color Quantization Image Retrieval ……
![Page 19: Random Swap EM algorithm for GMM and Image Segmentation Qinpei Zhao, Ville Hautamäki, Ismo Kärkkäinen, Pasi Fränti Speech & Image Processing Unit Department](https://reader035.vdocuments.mx/reader035/viewer/2022062404/551bf9f2550346a34f8b487a/html5/thumbnails/19.jpg)
Conclusion Introduce Randomization into algorithm Performs better Without heavy time complexity Wider applications
Thanks!☺