development of a computer platform for object 3d reconstruction using computer vision techniques
DESCRIPTION
DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES. Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz. Contents. Introduction to Computer Vision; Computer Platform presentation; Experimental results; Conclusions; Future work. - PowerPoint PPT PresentationTRANSCRIPT
DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D
RECONSTRUCTION USING COMPUTER VISION TECHNIQUES
DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D
RECONSTRUCTION USING COMPUTER VISION TECHNIQUES
Teresa C. S. Azevedo
João Manuel R. S. Tavares
Mário A. P. Vaz
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 2
Contents
I. Introduction to Computer Vision;
II. Computer Platform presentation;
III. Experimental results;
IV. Conclusions;
V. Future work.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 3
Computer Vision
Introduction
Platform
Conclusions
Future Work
Results
Computer Vision is continuously trying to develop theories and methods for automatic extraction of useful information from images, as similar as possible to the complex human visual system.
Some applications: Medicine - 3D reconstruction / modelling, surgery planning;
Identification and navigation systems;
Virtual reality;
…
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 4
Goals and Methodology
Introduction
Platform
Conclusions
Future Work
Results
Contactless techniques to recover the 3D geometry of an object are usually divided in two classes:
Our goal was to obtain 3D models of objects using an active vision technique called Structure From Motion (SFM).
• active techniques - require some kind of energy projection or the camera’s (or object’s) movement to obtain 3D information about the shape;
• passive techniques - only use ambient light and so, usually, the extraction of 3D information becomes more difficult.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 5
Computer Platform
Introduction
Platform
Conclusions
Future Work
Results
Integration of functions for 3D reconstruction, available from five software programs and one computational library, all open source:
• OpenCV;
• Peter’s Matlab Functions;
• Torr’s Matlab Toolkit;
• KLT;
• Projective Rectification without Epipolar Geometry;
• Depth Discontinuities by Pixel-to-Pixel Stereo.
Modular structure; User’s graphical interface; Computer language: C++;
Operational system: Microsoft Windows.
Ported to C using MATLAB Compiler toolbox
Developing tool: Microsoft Visual Studio, using MFC libraries (Microsoft Foundation Classes);
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 6
Computer Platform
Introduction
Platform
Conclusions
Future Work
Results
The functions integrated enclose several Computer Vision techniques:
• feature points detection;
• feature points matching between two images;
• epipolar geometry determination;
• rectification;
• dense matching.
For each technique, the user can easily choose the algorithm to use, as well as conveniently define its parameters.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 7
Feature Points detection
Introduction
Platform
Conclusions
Future Work
Results
available algorithms for feature points
detection
OpenCV KLT
Reflect the relevant discrepancies between their intensity values and those of their neighbours; Usually represent vertices of objects, and their detection allows posterior matching between the images of the sequences.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 8
Feature Points matching
Introduction
Platform
Conclusions
Future Work
Results
Image 2D points association between sequential images, which are the projection of the same 3D object point;
A short set of matching points is enough to determine the epipolar geometry between two images (the fundamental matrix).
1st image feature points coordinates
matching points coordinates on
2nd image
fundamental matrix
available algorithms for feature points
matching
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 9
Feature Points matching
Introduction
Platform
Conclusions
Future Work
Results
Some results:
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 10
Epipolar Geometry determination
Introduction
Platform
Conclusions
Future Work
Results
Corresponds to the geometrical structure between two stereo images and its expressed mathematically by the fundamental matrix;
Also allows the elimination of some previous wrong matches (outliers), as well as make easier the determination of new matching points (dense matching).
algorithms for epipolar lines determination
algorithms for epipolar geometry
determination
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 11
Epipolar Geometry determination
Introduction
Platform
Conclusions
Future Work
Results
Some results:
Epipolar line Inlier
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 12
Rectification
Introduction
Platform
Conclusions
Future Work
Results Method that changes two stereo images, in order to make them coplanar; Performing this step makes dense matching easier to obtain;
available algorithm for rectification
The quality of the results is proportional to the quality of the epipolar geometry determination.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 13
Dense matching
Introduction
Platform
Conclusions
Future Work
Results
Disparity map - codifies the distance between the object and the camera(s): closer points will have maximal disparity and farther points will get minimum disparity;
A disparity map gives some perception of discontinuity in terms of depth;
One of the algorithms also returns a discontinuity map – defines the pixels who border the changing between at least two levels of disparity.
available algorithms for
dense matching
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 14
Dense matching
Introduction
Platform
Conclusions
Future Work
Results
Some results:Original images
Disparity map Discontinuity map
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 15
Conclusions
Introduction
Platform
Conclusions
Future Work
Results
The functions, already integrated in the computer platform, give good results when applied to objects with strong characteristics;
From the experimental results, it is possible to conclude that low quality results are strongly correlated with few (strong) feature points detection and wrong matching;
This weakness is higher as the object shape variation is smooth.
Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz 16
Future work
Introduction
Platform
Conclusions
Future Work
Results
The next steps of this work will focus on improving the results obtained when the objects have smooth and continuous surfaces:
Finally, the computer platform will be used in 3D reconstruction and characterization of 3D external human shapes.
• inclusion of space carving techniques for object reconstruction;
• the feature points to use in the 3D space object definition will be detected with the use of a reduced number of markers added on the object;
• inclusion of a camera calibration technique, as well as pose and motion estimation algorithms;
DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D
RECONSTRUCTION USING COMPUTER VISION TECHNIQUES
DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D
RECONSTRUCTION USING COMPUTER VISION TECHNIQUES
Teresa C. S. Azevedo
João Manuel R. S. Tavares
Mário A. P. Vaz
AcknowledgmentsThis work was partially done in the scope of the project “Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEA-SRI/55386/2004, financially supported by FCT - Fundação para a Ciência e a Tecnologia in Portugal.