an introduction to visual perception – image formation › ~furnari › slides ›...
TRANSCRIPT
![Page 1: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/1.jpg)
AUGMENTED REALITY
Antonino Furnari
http://dmi.unict.it/~furnari
IPLab - Image Processing Laboratory
Dipartimento di Matematica e Informatica
Università degli Studi di Catania
http://iplab.dmi.unict.it
Computer Vision A.Y. 2014-2015
![Page 2: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/2.jpg)
AUGMENTED REALITY
“a live copy, view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video,
graphics or GPS data”
“it is related to a more general concept called mediated reality, in which a view of reality is modified (possibly
even diminished rather than augmented) by a computer”
“as a result, the technology functions by enhancing one’s current perception of reality.”
wikipedia
![Page 3: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/3.jpg)
APPLICATIONS
![Page 4: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/4.jpg)
VISION BASED AUGMENTED REALITY
Computer Vision allows to create augmented realityapplications by superimposing 2D or 3D contents on thescene;
In order to do so we need to:
1) detect and track the area where to show the content;
2) estimate its 3D position in the real world;
3) render the 2D/3D content according to the estimatedposition and the inferred geometry of the scene;
Two main technologies:
fiduciary markers;
markerless (i.e., object detection).
![Page 5: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/5.jpg)
SOME HISTORY
The term “augmented reality” appears since the1940s;
The first augmented head mounted display is inventedby Ivan Sutherland in 1968;
First systems using mobile devices, internet andgeolocalization appear in the 90s;
Advances in the 2000s;
Augmented Reality diffusion in the 2010s.
![Page 6: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/6.jpg)
HARDWARE
Some technologies which make AR interesting:
Handheld:
Mobile phones;
Tablets;
Wearable devices:
Google glass;
Microsoft Holo Lens;
Orcam (video - http://www.orcam.com/);
Epson Moverio.
![Page 7: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/7.jpg)
FIDUCIARY MARKERS AUGMENTED REALITY: ARTOOLKIT
ARToolkit is an Open Source toolkit for marker-based augmented reality;
It is quite old (last update in 2007) but still a goodstarting point for understanding the AR concepts (open& well documented);
It offers functions for detecting and tracking single ormultiple markers while relaying on OpenGL/glut for2D/3D rendering;
http://www.hitl.washington.edu/artoolkit.
![Page 8: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/8.jpg)
ARTOOLKIT DEMO
![Page 9: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/9.jpg)
ARTOOLKIT
For additional informations about the nexttopics, the reader is referred to the very wellwritten ARToolkit documentation and tutorials: http://www.hitl.washington.edu/artoolkit/documentation/
Some other useful information can be found inthe examples provided with the toolkit;
![Page 10: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/10.jpg)
BASIC PRINCIPLES
![Page 11: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/11.jpg)
FIDUCIARY MARKERS
The marker plays the role of an object whichgeometry is known;
In particular:
we chose markers which are easilydetectable (thick black borders);
we know the real world size of the marker;
we chose the inner symbol which is neitherhorizontally nor vertically symmetric in orderto estimate its rotation.
![Page 12: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/12.jpg)
MARKER DETECTION
original image thresholded image connected components
contours edges and corners fitted square
![Page 13: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/13.jpg)
MARKER DETECTION
A simple detection algorithm is used to find just candidates: anypattern with thick black borders;
The actual marker is found normalizing the candidates and comparing them with the searched pattern using template matching;
The candidate giving the highest confidence is selected.
![Page 14: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/14.jpg)
MARKER MATCHING
...
...
...
searched pattern
normalized candidates
found candidates
![Page 15: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/15.jpg)
ESTIMATION OF THE 3D POSITION AND ORIENTATION
Now that we have an objectwhich geometry, size, positionand orientation are known, wecan estimate its 3D position withrespect to the camera;
It can be done computing theextrinsic parameters as seen forcamera calibration;
Intrinsic parameters which aregood for most cameras arepart of the toolkit. Specificparameters can be obtainedcalibrating the camera.
![Page 16: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/16.jpg)
ARTOOLKIT COORDINATE SYSTEMS
![Page 17: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/17.jpg)
EXVIEW DEMO
![Page 18: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/18.jpg)
ARTOOLKIT – CAMERA CALIBRATION
Intrinsic parameters which are enough general to work with most of the cameras are available in the toolkit;
However, in order to improve the detection and tracking performances, a utility for camera calibration is provided in order to calibrate your own camera.
![Page 19: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/19.jpg)
MARKERLESS AUGMENTED REALITY?
Tracking a number of feature points (e.g., SIFT) inorder to detect a marker object (e.g., a photo)and to estimate its position and orientation.
![Page 20: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/20.jpg)
AUGMENTED REALITY TOOLKITS
![Page 21: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/21.jpg)
DEMO TIME
![Page 22: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/22.jpg)
QUESTION TIME
![Page 23: An Introduction to Visual Perception – Image Formation › ~furnari › slides › augmented_reality... · Computer Vision A.Y. 2014-2015. AUGMENTED REALITY “a live copy, view](https://reader036.vdocuments.mx/reader036/viewer/2022081611/5f02ed6a7e708231d406b340/html5/thumbnails/23.jpg)
CONTACTS
For any doubts feel free to contact me:
Room 30;
Slides availabe at:
My personal page:
http://www.dmi.unict.it/~furnari
Studium course page