陈 静 2009.09.04 jdl 视觉建模与识别组. 文献列表 标题: combining powerful local and...
TRANSCRIPT
![Page 1: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/1.jpg)
陈 静2009.09.04JDL视觉建模与识别组
![Page 2: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/2.jpg)
文献列表
标题: Combining powerful local and global statistics for texture description作者: Y. Xu, S.B. Huang, H. Ji, C. Fermuller
Paper I — #0872 Paper I — #0872
Paper II — #0600 Paper II — #0600
标题: Appearance-based Keypoint Clustering作者: F. Estrada, P.Fua, V.Lepetit, S. Susstrunk
![Page 3: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/3.jpg)
Appearance-based Keypoint Clustering Paper I
![Page 4: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/4.jpg)
1st Author——Francisco J. Estrada
• Biography– University of Toronto at Scarborough
• Department of Computer and Mathematical Sciences
– Ecole Polytechnique Federale de Lausanne (EPFL)
• Images and Visual Representation Group (IVRG)
• Computer Vision Lab (CVLAB)
– Centre for Vision Research (CVR) at York University
• Elderlab
– University of Toronto (UofT)• Computational Vision group in the
department of Computer Science过去
现在
From: http://www.cs.utoronto.ca/~strider
![Page 5: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/5.jpg)
1st Author——Francisco J. Estrada
• Research Interest & Publications– Perceptual Grouping
F. J. Estrada, P. Fua, V. Lepetit, and , S. Süsstrunk, “Appearance Based Keypoint Clustering”,In CVPR, 2009.
– Image SegmentationF. J. Estrada, and A. D. Jepson, “Benchmarking Image Degmentation Algorithms”, In IJCV,2009.
– Image ProcessingF. J. Estrada, D. J. Fleet, and A. D. Jepson, “ Stochastic Image Denoising”,In BMCV, 2009.
C. Fredembach, F. Estrada, and S. Süsstrunk, “Memory Colour Segmentation and Classification Using Class-specific Eigenregions”, accepted to IEEE TIP, 2009
R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk, “Frequency-tuned Salient Region Detection”, In CVPR, 2009
C. Fredembach, F. Estrada, and S. Süsstrunk, “Segmenting Memory Colours”, Color Imaging Conference, pp. 315-320, 2008
R. Achanta, F. Estrada, P. Wils, and S. Süsstrunk, “Salient Region Detection and Segmentation”, International Conference on Computer Vision Systems, pp. 66-75, 2008
– Single View ReconstructionP. Denis, J. Elder, and F. Estrada, “Efficient Edge-Based Methods for Estimating Manhattan Frames in Urban Imagery”, In ECCV, 2008
2009-8-27 中国科学院计算技术研究所 JDL实验室
5
From: http://www.cs.utoronto.ca/~strider
![Page 6: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/6.jpg)
2nd Author-Pascal Fua
• Biography :– Ecole Polytechnique , Paris, 1984.– PhD, Université d'Orsay, 1989.– Chair for Computer Vision Lab,since 2002
• Current Research Interests– Shape modeling and motion recovery.– Human body modeling.– Optimization algorithms for image processing and image
registration.– Automated feature extraction.– Robotics and Augmented Reality applications.
• Publication :– 2009 : PAMI 5’, CVPR 7’, IJCV 1’, ICCV 2’
2009-8-27 中国科学院计算技术研究所 JDL实验室
6
![Page 7: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/7.jpg)
3rd Author and 4th Author
第四作者: Sabine SusstrunkDirectory of the Images and Visual Representation Group (IVRG) since 1999.Reserch areas:Computational photography, color imaging, image quality metrics, image indexing and archiving
第三作者: Vincent LepetitResearch and Teaching Associate IN CVLabResearch Area:Object Detection and RecognitionRigid and Deformable 3D Tracking and Registration,Biomedical Applications, Augmented Reality.
![Page 8: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/8.jpg)
英文摘要• We present an algorithm for clustering sets of detected
interest points into groups that correspond to visually distinct structure.
• Through the use of a suitable colour and texture representation, our clustering method is able to identify key points that belong to separate objects or background regions.
• These clusters are then used to constrain the matching of key points over pairs of images, resulting in greatly improved matching under difficult conditions.
• We present a thorough evaluation of each component of the algorithm and show its usefulness on difficult matching problems.
2009-8-27 中国科学院计算技术研究所 JDL实验室
8
![Page 9: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/9.jpg)
中文摘要• 主要内容
– 聚类算法:将各关键点按视觉结构来进行分组• 关键技术
– Suitable 颜色和纹理表示 – 谱嵌入
• 主要应用– 关键点匹配
• 实验设计– 算法的各个组成部分的评估– 在匹配问题上的评估
![Page 10: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/10.jpg)
本文要解决的问题• Keypoints Perceptual Grouping
![Page 11: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/11.jpg)
Motivation
• lead to more efficient and reliable algorithms for– Object detection– Tracking– Scene reconstruction
![Page 12: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/12.jpg)
总的流程
KeypointsDetectionKeypointsDetection
Feature RepresentationFeature Representation
Spectral Embedding
Mean-shift ClustingMean-shift Clusting
![Page 13: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/13.jpg)
局部表观表示• 颜色信息表示
– RGB 颜色空间– 标准直方图
ip''''''''''''''
![Page 14: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/14.jpg)
局部表观表示• 纹理信息表示
2 2 2
2
1
1
:
: the number of pixels in
, , , ,
1
i
i
T
j jij p
i
j
j
j p
C v v v vN
where
N p
I I I I Iv
x y x y x y
v vN
''''''''''''''
''''''''''''''
''''''''''''''
ip''''''''''''''
说明:1 。这种纹理表示方法有很好的性能,特 别是在物体分类问题2 。本文的贡献在于,将它引入多线索视觉分类算法中
![Page 15: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/15.jpg)
相似度计算
2
2 2 ,
:
: colour histogram for patch
: colour histogram for patch
, 12
i
i jijk
i i
j j
i j
H k H kH H
H k
where
H p
H p
H k H kH k k n
''''''''''''''''''''''''''''''''''''''''''''''''''''''''
''''''''''''''
''''''''''''''''''''''''''''
''''''''''''''''''''''''''''
''''''''''''''''''''''''''''''''''''''''''
52
1
1...5
, ln
:
: ( 1...5)
0, 0
ij i j kk
k i j kk
k k kk i j
C C
where
C C k
C u C u u
和 的广义特征值,即 满足:
:ij i jd p p''''''''''''''''''''''''''''
块 和 中心间的欧式距离
22 22
2 2 2
ijij ijd
ija e
参数 通过最大化 F 来获得
*
.5* .5*
:
: :
p rF
p r
where
p r
正确率, 召回率
![Page 16: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/16.jpg)
谱嵌入• 基本思想
1
2
4
.6
.15
3 .7
.15
.6
.2
.6 .2
.05 .0
基本假设:1.如果转移概率能表征节点间的相似度,那么Random walk 将会 favous 那些和它相似的节点2.属于相同物体的图像块的 Kernel diffusion 很相似
节点 Vi :对应于图像块边 E(i, j) :表示块 和块 之间的相似度jp
''''''''''''''ip
'''''''''''''' ip''''''''''''''
Normalizing each column of EM: Markov matrix
0j t jtd M d
''''''''''''''''''''''''''''
![Page 17: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/17.jpg)
谱嵌入
Figure 2. Left: Original image and first 3 dimensions of the embedding.Second and third columns: similarity of sampled image patches (blue dots) with regard to a selected patch shown in red, brightness is proportional to similarity.
![Page 18: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/18.jpg)
实 验
• Figure 5. Tracking of a lightly textured object on a heavily cluttered background.
• Left Column: Unconstrained matching between image pairs.
• Right Column: Our approach to
• matching.
• Bottom Row: Keypoint clusters produced by our algorithm for the four target images.
2009-8-27 中国科学院计算技术研究所 JDL实验室
18
![Page 19: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/19.jpg)
实 验
2009-8-27 中国科学院计算技术研究所 JDL实验室
19
Figure 6. Tracking a cheetah against a complex background.
Left column: Standard unconstrained matching between a video frame and the reference image that appears in the top-left corner of Fig.1.
Middle column: Foreground/background clusters produced by our method for the target video frame.
Right column: Constrained matching results.
![Page 20: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/20.jpg)
实 验
Figure 7. Additional examples of constrained SIFT matching. Top row: conventional SIFT matching.Bottom row: constrained matching using the clusters detected by our method.
![Page 21: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/21.jpg)
Combining powerful local and global statistics texture description
Paper II
![Page 22: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/22.jpg)
英文摘要• A texture descriptor is proposed, which combines local highly
discriminative features with the global statistics of fractal geometry to achieve high descriptive power, but also invariance to geometric and illumination transformations.
• As local measurements SIFT features are estimated densely at multiple window sizes and discretized. On each of the discretized measurements the fractal dimension is computed to obtain the so-called multifractal spectrum, which is invariant to geometric transformations and illumination changes. Finally to achieve robustness to scale changes, a multi-scale representation of the multifractal spectrum is developed using a framelet system, that is, a redundant tight wavelet frame system.
• Experiments on classification demonstrate that the descriptor outperforms existing methods on the UIUC as well as the UMD high-resolution dataset.
![Page 23: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/23.jpg)
主要内容• 纹理描述 - 强的描述能力
– 局部判别性较高的特征– 全局的统计特征
![Page 24: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/24.jpg)
总体思路
ComputeMulti-levelOrientationHistogram
ConstructMFS
ConstructTexture Description
![Page 25: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/25.jpg)
总体思路
ComputeMulti-levelOrientationHistogram
ConstructMFS
ConstructTexture Description
![Page 26: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/26.jpg)
Compute Multi-level Orientation Histogram
Figure 2. Orientation histogram when using the neighborhood of size 5 × 5.
![Page 27: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/27.jpg)
Construct MFS
Figure 4. One rotated element and one mirror-reflected elementfrom the basic elements shown in the right column.
Figure 3. Basic elements of 29 orientation histogram templates
![Page 28: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/28.jpg)
Construct texture description
![Page 29: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/29.jpg)
基础知识• 小波分解 - 》卷积
![Page 30: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/30.jpg)
基础知识• 卷积 - 》矩阵乘法
mnm
![Page 31: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/31.jpg)
![Page 32: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/32.jpg)
![Page 33: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/33.jpg)
Construct texture description
![Page 34: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/34.jpg)
实 验
![Page 35: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/35.jpg)
实 验
![Page 36: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/36.jpg)
![Page 37: 陈 静 2009.09.04 JDL 视觉建模与识别组. 文献列表 标题: Combining powerful local and global statistics for texture description 作者: Y. Xu, S.B. Huang, H. Ji, C](https://reader034.vdocuments.mx/reader034/viewer/2022052203/56649e2a5503460f94b176c9/html5/thumbnails/37.jpg)
致谢
向被我打扰过的洪晓鹏师兄、李安南师兄、翟德明师姐和阚美娜师姐表示感谢
感谢各位在这听我“瞎扯”