shape matching and object recognition using low distortion ...€¦ · presenter: kalyan chandra...
TRANSCRIPT
![Page 1: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/1.jpg)
Shape Matching and Object Recognition using Low Distortion
Correspondences Authors: Alexander C. Berg, Tamara L.
Berg, Jitendra Malik
Presenter: Kalyan Chandra Chintalapati
![Page 2: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/2.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 3: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/3.jpg)
Introduction
• Recognition is a most important and interesting problem of Computer Vision.
• Any recognition problem is fundamentally a problem of deformable shape matching.
• According to the literature survey, the research on the deformable shape matching is on since 1970’s.
![Page 4: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/4.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 5: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/5.jpg)
Problem Statement Proposed Algorithm
Query Image
Corresponding Exemplar
Exemplars of Categories
![Page 6: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/6.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 7: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/7.jpg)
Basic Outline of the Approach
• The deformable matching takes an unknown input image and compares it to a model by solving the correspondence problem.
• It uses the correspondences to perform an aligning transformation.
• The overall similarity can be estimated by considering Aligning Transformation and Residual difference between both images.
![Page 8: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/8.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 9: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/9.jpg)
The difficult part!!
• Solving Correspondence problem itself.
• Interesting challenges are: – Position of Correspondence points
– Intra Category Variation
– Occlusion and Clutter
– 3-D pose changes
*We will try to see how the algorithm tackles all these challenges during the course of presentation
![Page 10: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/10.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 11: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/11.jpg)
Correspondence Points
• For every image, the algorithm considers 50-100 pixel locations on the edge detector output.
• These points are chosen at random.
![Page 12: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/12.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 13: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/13.jpg)
Constraints - Cues
• Similar local descriptors
• Minimizing the geometric distortion:
• Smoothness of the transformation:
– Regularized thin plate spline
Query
Template ri’j’j'
rij
rij'
![Page 14: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/14.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 15: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/15.jpg)
Geometric Blur
• Geometric Blur plays a key role in deciding the Geometric Distortion.
• The two ways a calculating the Geometric Blur are
– Signal(sparse and indicate the presence of an edge)
– Blur(should indicate nature and kind of distortion expected.)
![Page 16: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/16.jpg)
…continued
• S=S*Gd (G is the Gaussian Kernel filter)//signal calculation
• Bx0(x)=Sa|x|+b(x0-x) //blur calculation
• The feature descriptor is a concatenation of subsampled geometric blurs computed at each point of a correspondence point.
• These values are compared using sqrt(SSD) metric(L2 distance)
• *Experimentally, it only takes <1 sec for 1 image to calculate geometric blur value.
![Page 17: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/17.jpg)
Correspondence Result
![Page 18: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/18.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 19: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/19.jpg)
Cost function for similarity
• Quality of correspondence=Similarity of the correspondence points(match) + Change in the spatial arrangement(distortion between a pair of points in each image)
• Outliers: We assign a qnull for all the outliers in P when P is being mapped to Q.
![Page 20: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/20.jpg)
Measuring Distortion (Similarity in Configuration)
i i'
j j'
i i'
j j'Query Template
Rij Si'j'
Measure distortion in vectors between
pairs of feature points
- R and S same length for rotations
- R and S same direction for scalings
Da(sig)=0 Translation and pure scale
Dl(sig)=0Translation and rotation
![Page 21: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/21.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 22: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/22.jpg)
Correspondence Algorithm
• First term for the distortion(R,S in the above example) and second term is match value.
• Integer Quadratic problem reduction of above equation.
![Page 23: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/23.jpg)
..continued
• IQP is NP hard
• But here the instance the algorithm solves are generated to be easy.
• As an optimization step– The algorithm starts by initializing the bounds by gradient descent.
• Over all it assumes a run time of O(n2) time for each correspondence(n the total number of feature descriptors chosen).
• *Math is given less emphasis with comparison to idea explanation
![Page 24: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/24.jpg)
Quadratic Assignment (Using IQP)
![Page 25: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/25.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 26: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/26.jpg)
Results
• 2 object recognition problems are answered here.
– Multi Class Recognition(48%-- previous best 16%)
– Face Recognition(With only 20 exemplar faces, ROC curve is obtained with better generalizations and slightly worse false detection rate than existing best Face Recognition algorithm)
![Page 27: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/27.jpg)
Correspondence Examples (Shape Matching)
![Page 28: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/28.jpg)
Correspondence Examples (Shape Matching)
![Page 29: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/29.jpg)
Correspondence Examples (Shape Matching)
![Page 30: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/30.jpg)
Correspondence Examples (Shape Matching)
![Page 31: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/31.jpg)
Correspondence Examples (Shape Matching)
![Page 32: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/32.jpg)
Correspondence Examples (Shape Matching)
![Page 33: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/33.jpg)
Correspondence Examples (Shape Matching)
![Page 34: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/34.jpg)
Correspondence Examples (Shape Matching)
![Page 35: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/35.jpg)
Correspondence Examples (Shape Matching)
![Page 36: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/36.jpg)
Correspondence Examples (Shape Matching)
![Page 37: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/37.jpg)
Correspondence Examples (Shape Matching)
![Page 38: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/38.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 39: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/39.jpg)
Idea Applicability
• Applicability to break the distorted image of characters.
• Why and where is used??
– CAPTCHA analysis.
![Page 40: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/40.jpg)
Flow of the Presentation
Introduction Problem Statement
Basic Outline of the Approach The difficult part
Correspondence points Constraints-Cues Geometric Blur
Cost function for Similarity Correspondence Algorithm
Results Idea Applicability
Conclusion
![Page 41: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/41.jpg)
Conclusion
• Deformable shape matching.
• Solving Correspondence problem
• Geometric Blur concept
• IQP and fast reduction usage
• 300% improvement in Recognition rate(16% to 48%).
![Page 42: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/42.jpg)
One last interesting point
• Ever had a doubt regarding Rainbow colored representation of feature points on images and why they are marked like that??
.
.
.
.
There is a logic of zones of diagonals of colors to place the feature points.
![Page 43: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/43.jpg)
References
• Original paper url: http://acberg.com/papers/berg_correspondence_cvpr.pdf
• Presentation url: acberg.com/papers/berg_cvpr05.ppt
![Page 44: Shape Matching and Object Recognition using Low Distortion ...€¦ · Presenter: Kalyan Chandra Chintalapati . Flow of the Presentation Introduction Problem Statement Basic Outline](https://reader033.vdocuments.mx/reader033/viewer/2022053017/5f1b4e7a1ceec457be300938/html5/thumbnails/44.jpg)
Thank you for the patience.
Any Questions that I could
Answer??