random walk with restart (rwr) for image segmentation sungsu lim aalab, kaist
TRANSCRIPT
![Page 1: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/1.jpg)
Random Walk with Restart (RWR) for Image Segmentation
Sungsu Lim
AALAB, KAIST
![Page 2: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/2.jpg)
Image Segmentation Computer vision: make machine to see or to under-
stand/ interpret the scenes (images & videos) like human do.
Image segmentation is one of the most challenging issues in computer vision.
Two major difficulties of conventional algorithms: weak boundary problem & texture problem.
Semi-supervised segmentation approaches are pre-ferred since user inputs can reduce the ambiguity in images.2011-02-08 RWR for image segmentation 2
![Page 3: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/3.jpg)
Random Walks for Image Segmenta-tion RW (L. Grady, PAMI2006): In image segmentation,
random walks are used to determine the labels (i.e., “object” or “background”) to associate with each pixel.
K-way image segmentation: given user-defined seeds indicating regions of the image belonging to k ob-jects. Each seed specifies a location with a user-de-fined label.
We can use hitting time or commute time as rele-vance score between two nodes (seed and unlabeled pixel). By assigning each pixel to the label for which the best value is calculated.2011-02-08 RWR for image segmentation 3
![Page 4: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/4.jpg)
4
1
4
3
2
56
7
910
8
11
12
Random walk with restart
![Page 5: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/5.jpg)
Example of RW
What if we start at a different node?
Start node
2011-02-08 RWR for image segmentation 5
![Page 6: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/6.jpg)
RWR for Classification
RW with start nodes being
labeled points in class A
RW with start nodes being
labeled points in class B
Nodes frequented more by RW(A) belongs to class A,
otherwise they belong to B
Simple idea: use RW for classification
2011-02-08 6
![Page 7: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/7.jpg)
RWR for Image Segmentation Limitation of RW: only considers the local rela-
tionship between the pixel and that border. (more prone to hit popular nodes)
RWR (Kim, Lee and Lee, ECCV2008): a new generative image segmentation algorithm based on Random Walks with Restart (Pan,Yang and Faloutsos, KDD2004)
Most previous semi-supervised image seg-mentation algorithms focus on the inter-label discrimination, but it introduce a generative model for image segmentation.2011-02-08 RWR for image segmentation 7
![Page 8: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/8.jpg)
Generative Model
8
![Page 9: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/9.jpg)
Random Walk with Restart
Imagine a network, and starting at a specific node, you follow the edges randomly.
But (perhaps you’re afraid of wondering too far) with some probability, you “jump” back to the starting node (restart!).
If you record the number of times you land on each
node, what would that dis-tribution look like?
9
![Page 10: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/10.jpg)
Random Walk with Restart The walk distribution r satisfies a simple equa-
tion:
ePππ cc )1(
Transition ma-trix (relevance
vector)
Seed vec-tor
(start nodes)
“Keep-go-ing” proba-
bility (damp-ing factor)
Restart probabil-
ity
Equivalent to the well-known Google PageR-ank if all nodes
are start nodes! (e is
uniform)Rank-
ing vector
2011-02-08 RWR for image segmentation 10
![Page 11: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/11.jpg)
11
1
43
2
5 6
7
9 10
811
12
0.13 0 1/3 1/3 1/3 0 0 0 0 0 0 0 0
0.10 1/3 0 1/3 0 0 0 0 1/4 0 0 0
0.13
0.22
0.13
0.050.9
0.05
0.08
0.04
0.03
0.04
0.02
0
1/3 1/3 0 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 1/4 0 0 0 0 0 0 0
0 0 0 1/3 0 1/2 1/2 1/4 0 0 0 0
0 0 0 0 1/4 0 1/2 0 0 0 0 0
0 0 0 0 1/4 1/2 0 0 0 0 0 0
0 1/3 0 0 1/4 0 0 0 1/2 0 1/3 0
0 0 0 0 0 0 0 1/4 0 1/3 0 0
0 0 0 0 0 0 0 0 1/2 0 1/3 1/2
0 0 0 0 0 0 0 1/4 0 1/3 0 1/2
0 0 0 0 0 0 0 0 0 1/3 1/3 0
0.13 0
0.10 0
0.13 0
0.22
0.13 0
0.05 00.1
0.05 0
0.08 0
0.04 0
0.03 0
0.04 0
2 0
1
0.0
n x n n x 1n x 1
Example of RWRIterative update until convergence
ePππ cc tt 1)1(
2011-02-08 RWR for image segmentation 11
![Page 12: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/12.jpg)
Use of RWR Linear solution:
It can be reformulated as
( )
It considers all relations at all scales in the image.
ePπ 1))1(( cIc
0
)1(t
ttcc ePπ 10 c
As t increases weight becomes
smaller.
Weighted av-erage of all probability
Restart probabil-
ity
2011-02-08 RWR for image segmentation 12
![Page 13: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/13.jpg)
Use of RWR
As restarting probability c decreases, coarser scale is more emphasized in likelihood term.
0
)1(t
ttcc ePπ
2011-02-08 RWR for image segmentation 13
![Page 14: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/14.jpg)
Energy minimization framework Quadratic energy (cost) minimization:
similar to the formulation of RWR
( )
)(minarg* πππ
E
i
iji
jiijPE 2
,
2 )()()( eππππ
0eπPπππ
)(E
ePππ cc )1(
1
c
2011-02-08 RWR for image segmentation 14
![Page 15: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/15.jpg)
Experimental Results
2011-02-08 RWR for image segmentation 15
![Page 16: Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST](https://reader035.vdocuments.mx/reader035/viewer/2022062321/56649e935503460f94b98b18/html5/thumbnails/16.jpg)
Applications 1. Data-Driven RWR (Kim, Lee and Lee,
ICIP2009)It use the restart probability matrix. The restarting probability of each pixel depend on its edgeness, generated by Canny edge detector.
2. High-order RWR (multi-layer graph model)
2011-02-08 RWR for image segmentation 16