computer vision detecting the existence, pose and position of known objects within an image michael...
TRANSCRIPT
![Page 1: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/1.jpg)
Computer Vision Detecting the existence, pose and position of known
objects within an image
Michael Horne, Philip Sterne (Supervisor)
![Page 2: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/2.jpg)
Background
• Computer vision is a diverse field– Many schools of thought
• Many different ways of achieving– Monocular– Stereoscopic (quite popular)
![Page 3: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/3.jpg)
Problem Statment
• Recognize objects within a single image– Cutting down the total information in
the images– Extracting and building up useful subset– Using that information to recognize
objects• Model Based system
– System already knows of objects
![Page 4: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/4.jpg)
System structure
• Image capture and segmentation– Pre-process image– Extract useful information
• Matching– Find links between image and objects
known to the system• Pose Solution
– Use the matches to estimate the position and orientation of the object
![Page 5: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/5.jpg)
Image Segmentation
• Feature extraction– Images have lots of information– What features are of interest?– Edges are definitive attributes of
objects– Corners can easily be matched against
![Page 6: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/6.jpg)
Image Segmentation
• Gaussian filter is applied to reduce noise– Noise adds complexity, but no useful
information.• Canny Edge detector is applied to
extract edges
![Page 7: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/7.jpg)
Image Segmentation
• Arbitrary shaped edges are converted into straight line segments.
![Page 8: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/8.jpg)
Similar Geometries
• Need image data to be represented in a similar way to the object model
• Objects are stored as wireframes
• Image data converted to a wireframe like form.
![Page 9: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/9.jpg)
Matching
• To estimate a pose we find correspondences with corners
• Classification problem– Which object to match?– Or what data to match to which objects?
(Multiple object case)• Complex problem• Random Sample Consensus
approach
![Page 10: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/10.jpg)
Matching
• Take only the minimum amount of image data needed to estimate a solution
• But do it at random• Then test the validity of the
estimation
![Page 11: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/11.jpg)
Pose Estimation
• Now using the correspondences• Algorithm based on POSIT
– A quick method– In tune with the RANSAC approach
• POSIT– Scaled orthographic projection model
![Page 12: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/12.jpg)
Pose Estimation
• POSIT – Minimum correspondences is four– Will solve regardless correctness
• Validation is necessary
![Page 13: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/13.jpg)
Pose Estimation
• Checking need to be rapid• Various levels of verification
– Object must not be skewed in making the four points of the object to the image points.
– Distance to estimated position must be minimal
• Estimations that pass undergo further checking
![Page 14: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/14.jpg)
Pose Estimation
• Next step is to project object over image– Number of matches is expanded.– All forward facing vertices are checked
for a corresponding image point• Also the geometries are verified
![Page 15: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/15.jpg)
Pose Estimation
• Goodness ratio = corner ratio * edge ratio– General means of judging the fitted
model
– The higher the ratio, the better the fit• Model fitting with the best ratio is
chosen
![Page 16: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/16.jpg)
Final
• End result
![Page 17: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/17.jpg)
Multiple objects
• After each model is fitted the points used are marked as used– Independence of objects
• The matching process restarts
![Page 18: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/18.jpg)
Results
• Tests were setup for a set of four different objects
• Varying degrees of symmetry
• Images of various poses were captured
• Varying difficulty, degenerate poses to definitive
• 85% success in estimating pose of single objects
![Page 19: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/19.jpg)
Other examples
1 2 3
![Page 20: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/20.jpg)
Multiple Objects
• Multiple objects solved
![Page 21: Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)](https://reader031.vdocuments.mx/reader031/viewer/2022032723/56649cf85503460f949c8ff8/html5/thumbnails/21.jpg)