5/13/2015cam talk g.kamberova computer vision introduction gerda kamberova department of computer...
TRANSCRIPT
04/18/23 CAM Talk G.Kamberova
Computer VisionIntroduction
Gerda Kamberova
Department of Computer Science
Hofstra University
04/18/23 CAM Talk G.Kamberova
What is Computer Vision?
• Trucco and Verri– computing properties of the 3D world from one or more digital
images
• Stockman and Shapiro– To make useful decisions about real physical objects and scenes
based on sensed images
• Ballard and Brown– The construction of explicit, meaningful description of physical
objects from images
• Forsyth and Ponce– ...extracting descriptions of the world from pictures or sequences of
pictures
04/18/23 CAM Talk G.Kamberova
From 2D images to 3D model
04/18/23 CAM Talk G.Kamberova
General Rule
If you can’t understand (i.e. model)the forward process, you will have
a hard time solving the inverse!
04/18/23 CAM Talk G.Kamberova
What Information is in Images?
04/18/23 CAM Talk G.Kamberova
What information is in the image?
04/18/23 CAM Talk G.Kamberova
Typical Sensor: A CCD Camera• Basic process:
– photons hit a detector
– the detector becomes charged
– the charge is read out as brightness
– high sensitivity
– Noise issues
04/18/23 CAM Talk G.Kamberova
Pixel
Binary1 bit
Grey1 byte
Color3 bytes
Image Representation
Each pixel is a measure of the brightness (intensity of light)that falls on an area of an sensor (typically a CCD chip)
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Radiometry
What are the physical and geometric processes thatgovern (digital) imaging?
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Imaging Geometry
What are the physical and geometric processes thatgovern (digital) imaging?
04/18/23 CAM Talk G.Kamberova
Some Related Terms
• Image Processing: the study of the properties of operators that produce images from other images
• Machine Vision: a somewhat outdated term which now tends to refer to industrial vision applications where (usually) a single camera is used to solve a structured inspection task
• Pattern Recognition: typically refers to the recognition of structures in 2D images (usually without reference to any underlying 3D information).
• Photogrammetry: the science of measurement though non-contact sensing, e.g. terrain maps from satellite images. Usually is more focused on accuracy issues than interpretation.
04/18/23 CAM Talk G.Kamberova
Computer Vision vs. Graphics
• Computer Graphics– Produce “plausible” images– You choose the models, conditions, imaging parameters, etc.
• Computer Vision– Given real images with noise, sampling artifacts …– Estimate physically quantities– Ill-posed ---- what is the minimum world knowledge we need?
Is Vision the “Inverse” of Graphics?
04/18/23 CAM Talk G.Kamberova
From 2D images to 3D model and back
04/18/23 CAM Talk G.Kamberova
Image Filter Result
Problems of Computer Vision: Feature Extraction
What are the “informative” areas of an image and how do we detect them?
04/18/23 CAM Talk G.Kamberova
Filter kernels that are larger see effects at coarserscales
Problems of Computer Vision: Feature Extraction
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Feature Extraction – edge detection
04/18/23 CAM Talk G.Kamberova
Computer Vision vs. Image Processing
• Image Processing– Mostly concerned with image-to-image transformations
• Filtering• Enhancement• Compression
• Computer Vision– Concerned with how images reflect the 3D world
• Filtering for feature extraction• Enhancement for recognition/detection• Compression that preserves geometric information in
images
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Segmentation and Grouping
What portionsof an image pertainto one another andto relevant physicalphenomena?
04/18/23 CAM Talk G.Kamberova
Computer Vision vs. Human Vision
What is the rightsegmentation?To us it seemsobvious …“common sense”
04/18/23 CAM Talk G.Kamberova
Grouping
04/18/23 CAM Talk G.Kamberova
Illusions
04/18/23 CAM Talk G.Kamberova
Illusions
04/18/23 CAM Talk G.Kamberova
Stereo(psis)
• Stereo • Recovers information about 3D structure• 2 images of the same scene (different viewpoints)
04/18/23 CAM Talk G.Kamberova
04/18/23 CAM Talk G.Kamberova
Autostereogram
04/18/23 CAM Talk G.Kamberova
Computer Vision Problems: Stereo
• Correspondence (matching):• From two (or more) images,
determine the geometry ofhe scene by matching corresponding areas ofthe images
•Reconstruction: from 2D matched elements to 3D pointshe.
RIGHT IMAGE PLANE
LEFT IMAGE PLANE
04/18/23 CAM Talk G.Kamberova
THE ORGANIZATION OF AN IMAGE SEQUENCE
Frames
04/18/23 CAM Talk G.Kamberova
THE MOTION FIELD
The “instantaneous” velocity of points in an image
The focus of expansion
1. Direction of motion 2. Time to collision
04/18/23 CAM Talk G.Kamberova
MOVING CAMERAS ARE LIKE STEREO: Structure from Motion
Locations ofpoints on the object(the “structure”)
The change in spatial locationbetween the two cameras (the “motion”)
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Recognition
Given a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.
04/18/23 CAM Talk G.Kamberova
Problems of Computer Vision: Recognition
Given a database ofobjects and an imagedetermine what, if anyof the objects are present in the image.
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision: Biometrics
• Face recognition
• Iris scanning
• Fingerprint recognition
• Activity recognition
• 3D biometric
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision: Medical Imaging
04/18/23 CAM Talk G.Kamberova
Examples: Virtual colonoscopy
Patient: data, imaging, rather non-invasive
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision: HCI
calculator
GUI Face trackerGesture recognition
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision: Image Databases
(Courtesy D. Forsyth & J. Ponce)
From a searchfor horse pixin 100 horseimages and1086 non-horseimages
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision:
3D model aquisition(Jitendra Malik, Berkeley)
04/18/23 CAM Talk G.Kamberova
Applications of Computer Vision: Motion Control
04/18/23 CAM Talk G.Kamberova
CMU: Vitrualized Reality
04/18/23 CAM Talk G.Kamberova
Tele-Immersion: UPenn
is the technology that enables remotely located users to share the same real or virtual environments in real time.
04/18/23 CAM Talk G.Kamberova
Teleimmersion
• sensing of the real world
• interaction with the real world
• visualization of real and synthetic data
• networking
• Why 3D?
• Collaborative design
• Telemedicine
• Training in 3D
• Visualization of complex data
• Entertainment
04/18/23 CAM Talk G.Kamberova
Tele-immersion for the Common Citizen
04/18/23 CAM Talk G.Kamberova
Teleimmersion: Collaboration and Working in Virtual 3D Worlds(UPenn, UNC)
04/18/23 CAM Talk G.Kamberova
Computer Vision Problems: 3D reconstruction
A Reconstructed Depth Map
04/18/23 CAM Talk G.Kamberova
Polynocular Stereo
• From images to unorganized 3D point cloud
04/18/23 CAM Talk G.Kamberova
Surface Reconstruction from unorganized 3D point cloud
Orient Locally Reconstruction by polyhedral approximation
Compute 3D shape ifrom the polyhedral surface: mean and Gauss curvatures
04/18/23 CAM Talk G.Kamberova
3D Shape recovery: applications in registration and matching of
surfaces/objects
• G^2 Kamberov: Ongoing program to associate geometric descriptors and invariants directly to an unorganized oriented cloud of points
• Without polyhedral approximation, or model fitting
04/18/23 CAM Talk G.Kamberova
Another cloud
04/18/23 CAM Talk G.Kamberova
Example: Principal directions
04/18/23 CAM Talk G.Kamberova
3D shape: the mean curvature surface
04/18/23 CAM Talk G.Kamberova
3D shape from medical data
04/18/23 CAM Talk G.Kamberova
Medical Data 2