1ellen l. walker 3d vision why? the world is 3d not all useful information is readily available in...
Post on 17-Jan-2018
220 Views
Preview:
DESCRIPTION
TRANSCRIPT
1 Ellen L. Walker
3D Vision
Why? The world is 3D
Not all useful information is readily available in 2D
Why so hard? “Inverse problem”: one image = many scenes
Complex relationship between objects & pixels
Noise, occlusion, etc.
What can we do? Use "hints" from our knowledge of the world
Add more information to the problem!
2 Ellen L. Walker
Labeling Image Edges
Many edge labels carry 3D information Occluding blade (>)
Occluding surface to right along arrow
Convex crease (+) Edge is closer than both surfaces
Concave crease (–) Edge is further than both surfaces
Limb (>>) Edge is "horizon" of curving-away surface
Others are about reflectance or illumination changes Mark (M)
Change due to paint or material boundary
Illumination Boundary (S) Shadow edge
3 Ellen L. Walker
Intrinsic Image Pixel Contents
Depth (range)
Orientation (surface normal)
Illumination
Albedo (reflectance)
Given this information, the picture (intensity values) can be completely reconstructed
4 Ellen L. Walker
3D Cues in Single 2D Images
Occlusion Occluding objects are closer to the camera The crossbar of a T-junction belongs to the occluding
object
Perspective scaling and foreshortening If two copies of the same object appear in a picture, the
smaller one is further away. Scaling is parallel to the image plane Foreshortening is perpendicular to the image plane
Texture Gradient Since texture is composed of repeated patterns, changes
in size and density of texture convey depth cues
5 Ellen L. Walker
"Shape-From" Methods
Use a cue (e.g. texture, shading) from a small region of an image
Cues generally give partial surface orientation information E.g. degree of tilt
Related cues can give "boundary conditions" to start from
Solve for continuous surfaces that satisfy both the general and boundary constraints
6 Ellen L. Walker
Example: Shape From Shading
Figure 12.2
7 Ellen L. Walker
Gradient Space Represents Surface Normal
p
q
8 Ellen L. Walker
Reflectance Geometry
Three directions are important: normal to the surface, surface to light source, and surface to camera
surface patch
light source
normal vector
Camera
9 Ellen L. Walker
Types of Reflection (Review)
Specular reflection (mirror) Color depends on light source color Limited scattering: angle of incidence = angle of reflection Camera sees light if it’s pointed in the right direction Nearly all light is reflected
Lambertian reflection (matte) Color depends on material properties of object Light evenly scattered throughout half-space Camera sees light if surface is visible Amount of light reflected is proportional to angle between
surface and light source
10 Ellen L. Walker
Shape From Shading
Lambertian surface, known light source direction Reflective component (highlights) can be subtracted out in
preprocessing
Relative brightness of surface patches constrain their directions Darker patches are more tilted away A given brightness value represents a circle in gradient
space
Boundary pixels indicate surface at 90 degrees from normal (if smooth surface)
Solve an optimization problem: brightness term and smoothness term
11 Ellen L. Walker
Shape From Shading (cont’d)
Brightness term Intensity is a function of reflectance, and reflectance is a
function of surface normals (p,q) and light source direction (vx, vy, vz)
Smoothness term Try to minimize integral of partial derivatives of p and q in
x and y direction€
I(x,y)= R(p(x,y),q(x,y))
R(p,q) = max(0,ρpvx +qvy + vz1+ p2 +q2
)
€
ES = px2∫ + py
2 +qx2 +qy
2dxdy
12 Ellen L. Walker
Photometric Stereo
Extension to multiple "images" Lambertian surface, several light sources
Each image has one light source, constrains surfaces
Solve an overdetermined linear (matrix) system - like camera calibration with extra points
Implementation Surround your object with a frame containing inward-
pointing lights
Take an image with each light in turn
Use images and known light directions to solve the equations.
13 Ellen L. Walker
More Variations
SHINY Photometric stereo system for highly reflective materials
Used to accurately characterize welds
Accurate color determination (plastic objects) Separate highlights from matte portion
Determine illumination color from highlight
Determine object color
Create “matte object” for photometric stereo or shape from shading
14 Ellen L. Walker
Shape From Texture
Figure 12.3
15 Ellen L. Walker
Shape from Texture
Transformation of original texture related to surface normal
Solve for affine transformation between original texel and viewed texel
Transformation depends on surface normal & distance
(Assume camera is far enough to avoid worst perspective distortion)
If the original texel is known, transformations can be computed directly
If the original texel is unknown, assume the largest visible texel is directly facing the camera
Use smoothness or shape constraints to eliminate alternatives
16 Ellen L. Walker
Shape from Focus
Vary the focal length of the camera (i.e. zoom lens)
Objects at different distances will become clear at different focal lengths
Can use comparisons between pairs of images to get relative distances
17 Ellen L. Walker
Choice Depends on Object
Solid, matte object (or matte separated from specular) Shape from shading, photometric stereo
Highly reflective object Shape from specular reflection
Regular textured object Shape from texture, stereo, focus
Irregular textured object Stereo, focus
top related