computer vision processing scale invariant feature …2016/08/16 · scale invariant feature...
TRANSCRIPT
![Page 1: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/1.jpg)
Scale InvariantFeature Transform
Computer Vision Processing
![Page 2: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/2.jpg)
Content- Introduction to SIFT
- Detection of Scale-Space Extrema
- Accurate Keypoint Localization
- Orientation Assignment
- The Local Image Descriptor
- Application to object Recognition
![Page 3: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/3.jpg)
Introduction to SIFT
David Lowe invent SIFT at 1999
- Point Matching
- Scale Invariant
- Luminance Invariant
- Orientation Invariant
- Affine Transformation Invariant
![Page 4: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/4.jpg)
Detection of Scale-Space Extrema
- Difference of Gaussian
𝑫 𝒙, 𝒚, 𝝈 = 𝑮 𝒙, 𝒚, 𝒌𝝈 − 𝑮 𝒙, 𝒚, 𝝈 ∗ 𝑰 𝒙, 𝒚
= 𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
Where 𝑳 𝒙, 𝒚, 𝝈 = 𝑮 𝒙, 𝒚, 𝝈 ∗ 𝑰 𝒙, 𝒚
- Laplacian of Gaussian : 𝛁𝟐𝑳(𝒙, 𝒚, 𝝈) = 𝑳𝒙𝒙 + 𝑳𝒚𝒚
- Heat diffusion equation : 𝝏𝑳
𝝏𝝈= 𝝈𝛁𝟐𝑳
![Page 5: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/5.jpg)
Detection of Scale-Space Extrema
𝝈𝛁𝟐𝑳 =𝝏𝑳
𝝏𝝈=𝚫𝑳
𝚫𝝈= lim𝒌→𝟏
𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
𝒌𝝈 − 𝝈
≈𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
(𝒌 − 𝟏)𝝈
𝒌 − 𝟏 𝝈 ∙ 𝝈𝛁𝟐𝑳 = 𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
𝒌 − 𝟏 𝝈𝟐𝛁𝟐𝑳 = 𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
𝒌 − 𝟏 𝝈𝟐𝑳𝒐𝑮 = 𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
𝒌 − 𝟏 𝑳𝒐𝑮𝒏𝒐𝒎𝒂𝒍𝒊𝒛𝒆𝒅 = 𝑫𝒐𝑮
![Page 6: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/6.jpg)
Detection of Scale-Space Extrema
→ 𝑰
→ 𝒈
→ 𝒈 ∗ 𝑰
→ 𝑮 ∗ 𝑰 = 𝑳
![Page 7: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/7.jpg)
Detection of Scale-Space Extrema
![Page 8: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/8.jpg)
Detection of Scale-Space Extrema
Gaussian Pyramid
![Page 9: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/9.jpg)
Detection of Scale-Space Extrema
Gaussian Pyramid𝑫 𝒙, 𝒚, 𝝈 = 𝑳 𝒙, 𝒚, 𝒌𝝈 − 𝑳(𝒙, 𝒚, 𝝈)
![Page 10: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/10.jpg)
Detection of Scale-Space Extrema
Gaussian Pyramid
![Page 11: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/11.jpg)
Detection of Scale-Space Extrema
Gaussian Pyramid
- Cascade property of Gaussian Kernerl
𝒈 𝒙, 𝒚, 𝒔𝟏 + 𝒔𝟐 = 𝒈 𝒙, 𝒚, 𝒔𝟏 ∗ 𝒈 𝒙, 𝒚, 𝒔𝟐𝒈 𝒙, 𝒚, 𝒔𝟏 + 𝒔𝟐 ∗ 𝑰 𝒙, 𝒚 = 𝒈 𝒙, 𝒚, 𝒔𝟏 ∗ 𝒈 𝒙, 𝒚, 𝒔𝟐 ∗ 𝑰(𝒙, 𝒚)𝑳 𝒙, 𝒚, 𝒔𝟏 + 𝒔𝟐 = 𝒈 𝒙, 𝒚, 𝒔𝟏 ∗ 𝑳 𝒙, 𝒚, 𝒔𝟐𝑳 𝒙, 𝒚, 𝒔𝟏 − 𝒔𝟏 + 𝒔𝟐 = 𝒈 𝒙, 𝒚, 𝒔𝟐 − 𝒔𝟏 ∗ 𝑳 𝒙, 𝒚, 𝒔𝟏𝑳 𝒙, 𝒚, 𝒔𝟐 = 𝒈 𝒙, 𝒚, 𝒔𝟐 − 𝒔𝟏 ∗ 𝑳 𝒙, 𝒚, 𝒔𝟏
𝑳 𝒙, 𝒚, 𝒔𝟐 = 𝒈 𝒙, 𝒚, 𝒔𝟐 − 𝒔𝟏 ∗ 𝑳 𝒙, 𝒚, 𝒔𝟏
𝒔𝒊𝒈𝒎𝒂𝟏 = 𝝈, 𝒔𝒊𝒈𝒎𝒂𝟐 = 𝟐𝝈
𝑺𝒊𝒈𝒎𝒂𝟐 = 𝟐𝝈𝟐− 𝝈 𝟐
𝑺𝒊𝒈𝒎𝒂 = 𝟐𝝈𝟐− 𝝈 𝟐 = 𝝈
![Page 12: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/12.jpg)
Detection of Scale-Space Extrema
Extrema Detection(candidate)
- Compare to 28 pixel.
- Detect the local maxima and minima.
![Page 13: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/13.jpg)
Accurate Keypoint Localization
𝑿 = (𝒙, 𝒚, 𝝈)
Difference of Gaussian function D(X)
𝑫 𝑿 = 𝑫+𝝏𝑫
𝝏𝑿
𝑻
𝑿 +𝟏
𝟐𝑿𝑻𝝏𝟐𝑫
𝝏𝑿𝟐𝑿
Taylor series
Extremum point
𝑫′ 𝑿 = 𝟎 +𝝏𝑫
𝝏𝑿
𝑻
+𝝏𝟐𝑫
𝝏𝑿𝟐𝑿
𝝏𝟐𝑫
𝝏𝑿𝟐𝑿 = −
𝝏𝑫
𝝏𝑿
𝑿 = −𝝏𝟐𝑫
𝝏𝑿𝟐
−𝟏𝝏𝑫
𝝏𝑿
𝑫 𝑿 = 𝑫 +𝟏
𝟐
𝝏𝑫
𝝏𝑿
𝑻
𝑿
𝑫 𝑿 < 𝟎. 𝟑 discard
![Page 14: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/14.jpg)
Accurate Keypoint Localization
Harris Edge Detection
Base Idea
E(𝒖, 𝒗) =
(𝒙,𝒚)∈𝑾
𝑰 𝒙 + 𝒖, 𝒚 + 𝒗 − 𝑰(𝒙, 𝒚) 𝟐
𝑰 𝒙 + 𝒖, 𝒚 + 𝒗 Taylor series
𝑰 𝒙 + 𝒖, 𝒚 + 𝒗 = 𝑰 𝒙, 𝒚 +𝝏𝑰
𝝏𝒙𝒖 +𝝏𝑰
𝝏𝒚𝒗 + 𝒉𝒊𝒈𝒉𝒕 𝒐𝒓𝒅𝒆𝒓
≈ 𝑰 𝒙, 𝒚 +𝝏𝑰
𝝏𝒙𝒖 +𝝏𝑰
𝝏𝒚𝒗
= 𝑰 𝒙, 𝒚 + 𝑰𝒙 𝑰𝒚𝒖𝒗
![Page 15: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/15.jpg)
Accurate Keypoint Localization
Harris Edge Detection
E(𝒖, 𝒗) =
(𝒙,𝒚)∈𝑾
𝑰 𝒙 + 𝒖, 𝒚 + 𝒗 − 𝑰(𝒙, 𝒚) 𝟐
𝑬(𝒖, 𝒗) ≈
(𝒙,𝒚)∈𝑾
𝑰 𝒙, 𝒚 + 𝑰𝒙 𝑰𝒚𝒖𝒗− 𝑰(𝒙, 𝒚)
𝟐
= 𝒖 𝒗
(𝒙,𝒚)∈𝑾
𝑰𝒙𝟐
(𝒙,𝒚)∈𝑾
𝑰𝒙𝑰𝒚
(𝒙,𝒚)∈𝑾
𝑰𝒚𝑰𝒙
(𝒙,𝒚)∈𝑾
𝑰𝒚𝟐
𝒖𝒗
= 𝒖 𝒗 𝑯𝒖𝒗
![Page 16: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/16.jpg)
Accurate Keypoint Localization
Harris Edge Detection
𝑯′𝒔 𝑬𝒊𝒈𝒆𝒏𝒗𝒂𝒍𝒖𝒆 𝒊𝒔 𝝀𝟏, 𝝀𝟐 (𝝀𝟏 > 𝝀𝟐)
𝑫𝒆𝒕 𝑯 = 𝝀𝟏𝝀𝟐
𝑻𝒓𝒂𝒄𝒆 𝑯 = 𝝀𝟏 + 𝝀𝟐
𝑻𝒓 𝑯 𝟐
𝑫𝒆𝒕 𝑯=𝝀𝟏 + 𝝀𝟐
𝟐
𝝀𝟏𝝀𝟐=𝒓 + 𝟏 𝟐
𝒓𝒂𝒕, (𝝀𝟏 = 𝒓𝝀𝟐)
∴𝑻𝒓 𝑯 𝟐
𝑫𝒆𝒕 𝑯<𝒓 + 𝟏 𝟐
𝒓𝒂𝒕, (𝒓 = 𝟏𝟎)
Why remove the Edge?
![Page 17: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/17.jpg)
Accurate Keypoint Localization
![Page 18: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/18.jpg)
Orientation Assignment
𝒎 𝒙, 𝒚 = 𝑳 𝒙 + 𝟏, 𝒚 − 𝑳 𝒙 − 𝟏, 𝒚𝟐+ 𝑳 𝒙, 𝒚 + 𝟏 − 𝑳 𝒙, 𝒚 − 𝟏
𝟐
𝜽 𝒙, 𝒚 = 𝒕𝒂𝒏−𝟏𝑳 𝒙 + 𝟏, 𝒚 − 𝑳 𝒙 − 𝟏, 𝒚
𝑳 𝒙, 𝒚 + 𝟏 − 𝑳 𝒙, 𝒚 − 𝟏
In addition, each sample added to the histogram is weighted by its gradient magnitude and by a Gaussian-weighted circular window with a that is 1.5 scale of the keypoint
For 16x16 Pixel
![Page 19: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/19.jpg)
Orientation Assignment
Make Histogram graph
The orientation histogram has 36 bins covering the 360 degree range of orientation.The highest peak in the histogram is detected, and then any other local peak that is within 80% of the highest peak is used to also create a keypoint with that orientation.
![Page 20: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/20.jpg)
The local image Descriptor
Keypoint Descriptor
We know the Magnitude, orientation of Keypoint.But this is not special feature.This case, 𝜎 of Gaussian weighted function is half of Descriptor window size. In addition orientation is subtract the orientation of previous session. Finally create the histogram for the 4x4 pixel.
![Page 21: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/21.jpg)
Application to Object Recognition
Keypoint Matching
Euclidean distance
𝐴 = (𝑎1, 𝑎2, 𝑎3, … , 𝑎𝑛)
𝐵 = (𝑏1, 𝑏2, 𝑏3, … , 𝑏𝑛)
𝐷 =
𝑖=1
𝑛
𝑎𝑖 − 𝑏𝑖2
However, many features from an image will not have any correct match in the DB. Thus a global threshold on distance to the closest feature does not perform well. A more effective measure is obtained by comparing the distance of the closest neighbor to that of the second-closest neighbor.
Successful keypoint matching is very small Euclidean distance.
![Page 22: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/22.jpg)
Application to Object Recognition
Hough Transform or RANdom Sample Consensus(RANSAC)
𝑦𝑖 = 𝑎𝑥𝑖 + 𝑏 𝑏 = −𝑥𝑖𝑎 + 𝑦𝑖
![Page 23: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/23.jpg)
Application to Object Recognition
Hough Transform or RANdom Sample Consensus(RANSAC)
![Page 24: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/24.jpg)
Application to Object Recognition
Affine transform
![Page 25: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/25.jpg)
Application to Object Recognition
Affine transform
𝑢𝑣=𝑚1 𝑚2𝑚3 𝑚4
𝑥𝑦 +𝑡𝑥𝑡𝑦
Unknown variable
𝑚1,𝑚2,𝑚3,𝑚4, 𝑡𝑥, 𝑡𝑦
One matching has two equations.So requires three matching.
1 1 1
1 1 1
2 2 2
2 2 2
3 3 3
3 3 3
10 0 1 0
20 0 0 1
30 0 1 0
40 0 0 1
0 0 1 0
0 0 0 1
x
y
mx y u
mx y v
mx y u
mx y v
tx y u
tx y v
𝐴𝑥 = 𝑏𝑥 = 𝐴−1𝑏
![Page 26: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/26.jpg)
Result IMG
![Page 27: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/27.jpg)
Result IMG
![Page 28: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/28.jpg)
Q & A
![Page 29: Computer Vision Processing Scale Invariant Feature …2016/08/16 · Scale Invariant Feature Transform Computer Vision Processing Content - Introduction to SIFT - Detection of Scale-Space](https://reader034.vdocuments.mx/reader034/viewer/2022042601/5fa85ac04a283c77bf7739c8/html5/thumbnails/29.jpg)