photometric stereo, shape from shading sfs chapter 12.1.1 ... · photometric stereo, shape from...
TRANSCRIPT
![Page 1: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/1.jpg)
Photometric Stereo, Shape from Shading SfS
Chapter 12.1.1. Szelisky
Guido Gerig
CS 6320, Spring 2012
Credits: M. Pollefey UNC CS256, Ohad Ben-Shahar CS BGU, Wolff JUN
(http://www.cs.jhu.edu/~wolff/course600.461/week9.3/index.htm)
![Page 2: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/2.jpg)
Photometric Stereo
Depth from Shading?
First step: Surface
Normals from Shading
Second step:
Re-integration of
surface from Normals
![Page 3: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/3.jpg)
Examples http://www.youtube.com/watch?v=sfCQ7f7PMbc&feature=related
http://www.youtube.com/watch?v=KiTA6ftyQuY
Simulated voyage over the surface of Neptune's large moon Triton
http://www.youtube.com/watch?v=nwzVrC2GQXE
![Page 4: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/4.jpg)
Credit: Ohad Ben-Shahar CS BGU
![Page 5: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/5.jpg)
Shape from Shading
![Page 6: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/6.jpg)
Photometric Stereo
• Assume:
– a local shading model
– a set of point sources that are infinitely distant
– a set of pictures of an object, obtained in exactly the same camera/object configuration but using different sources
– A Lambertian object (or the specular component has been identified and removed)
![Page 7: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/7.jpg)
Setting for Photometric Stereo
Multiple images with different lighting (vs binocular/geometric stereo)
![Page 8: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/8.jpg)
Goal: 3D from One View and multiple Source positions
Input images Usable Data
Mask
![Page 9: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/9.jpg)
Scene Results
Needle Diagram:
Surface Normals
Albedo
Re-lit:
![Page 10: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/10.jpg)
Projection model for surface recovery - usually called a Monge patch
![Page 11: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/11.jpg)
Lambertian Reflectance Map
LAMBERTIAN MODEL
E = r <n,ns> =r COS q
Y
X Z
q (ps,qs,-1)
(p,q,-1)
2222 11
1
LL
LL
qpqp
qqppCOS
q
![Page 12: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/12.jpg)
Wolff, November 4, 1998 12
REFLECTANCE MAP IS A VIEWER-CENTERED
REPRESENTATION OF REFLECTANCE
Depth
Surface
Orientation
Y
X
Z
IMAGE PLANE
z=f(x,y)
x y
dx dy
y (-f , -f , 1)
(f , f , -1)
(0,1,f ) x
(0,1,f ) x
(1,0,f )
(1,0,f ) y
= x y
![Page 13: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/13.jpg)
13
REFLECTANCE MAP IS A VIEWER-CENTERED
REPRESENTATION OF REFLECTANCE
(-f x , -f y , 1) = (-p, -q, 1)
p, q comprise a gradient or gradient space representation for
local surface orientation.
Reflectance map expresses the reflectance of a material directly
in terms of viewer-centered representation of local surface
orientation.
Wolff, November 4, 1998
![Page 14: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/14.jpg)
Reflectance Map (ps=0, qs=0)
![Page 15: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/15.jpg)
Reflectance Map
Credit: Ohad Ben-Shahar CS BGU
![Page 16: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/16.jpg)
Reflectance Map
Credit: Ohad Ben-Shahar CS BGU
![Page 17: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/17.jpg)
Reflectance Map
![Page 18: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/18.jpg)
Reflectance Map (General)
y
zq
x
zp
isophote
ss qp ,
![Page 19: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/19.jpg)
Reflectance Map
Given Intensity I in image, there are multiple (p,q) combinations (= surface orientations).
Use multiple images with
different light source directions.
y
zq
x
zp
Isophote I
ss qp ,
![Page 20: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/20.jpg)
Multiple Images = Multiple Maps
Can isolate p, q as contour intersection
(p,q) I1
I2
I2 I1
![Page 21: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/21.jpg)
Example: Two Views
Still not unique for certain intensity pairs.
![Page 22: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/22.jpg)
Constant Albedo
Solve linear equation system
to calculate 𝑁 .
![Page 23: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/23.jpg)
Varying Albedo
Solution Forsyth & Ponce:
For each point source, we know the source vector (by assumption). We assume we know the scaling constant of the linear camera (k). Fold the normal (N) and the reflectance (𝝆(x,y)) into one vector g, and the scaling constant and source vector into another Vj.
• Out of shadow:
• In shadow: 𝐼 𝑥, 𝑦 = 0
![Page 24: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/24.jpg)
Multiple Images: Linear Least Squares Approach
• Combine albedo and normal
• Separate lighting parameters
• More than 3 images => overdetermined system
• How to calculate albedo 𝜌 and 𝑁 ?
𝑔 𝑥, 𝑦 = 𝜌 𝑥, 𝑦 𝑁 (x,y)
→ 𝑁 =𝑔
𝑔 , 𝜌 𝑥, 𝑦 = 𝑔
: 𝑔 (x,y)=𝑉−1i(x,y)
![Page 25: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/25.jpg)
Example LLS Input
Problem: Some regions in some images are in
the shadow (no image intensity).
![Page 26: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/26.jpg)
Dealing with Shadows (Missing Info)
For each point source,
we know the source
vector (by assumption).
We assume we know the
scaling constant of the
linear camera. Fold the
normal and the
reflectance into one
vector g, and the scaling
constant and source
vector into another Vj
Out of shadow:
In shadow:
I j(x,y) 0
I j (x, y) kB(x, y)
kr(x, y) N(x, y)S j g(x,y)Vj
No partial shadow
![Page 27: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/27.jpg)
Matrix Trick for Complete Shadows
• Matrix from Image Vector:
• Multiply LHS and RHS with diag matrix
I12(x,y)
I22(x,y)
..
In2(x,y)
I1(x,y) 0 .. 0
0 I2(x,y) .. ..
.. .. .. 0
0 .. 0 In(x,y)
V1T
V2T
..
VnT
g(x,y)
Known Known Known
Unknown
Relevant elements of the left vector and the matrix
are zero at points that are in shadow.
![Page 28: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/28.jpg)
Obtaining Normal and Albedo
• Given sufficient sources, we can solve the previous equation (most likely need a least squares solution) for g(x, y).
• Recall that N(x, y) is the unit normal.
• This means that rx,y) is the magnitude of g(x, y).
• This yields a check
– If the magnitude of g(x, y) is greater than 1, there’s a problem.
• And N(x, y) = g(x, y) / rx,y).
![Page 29: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/29.jpg)
Example LLS Input
![Page 30: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/30.jpg)
Example LLS Result
• Reflectance / albedo:
![Page 31: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/31.jpg)
Recap
• Obtain normal / orientation, no depth
IMAGE PLANE
Depth?
Surface
Orientation
![Page 32: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/32.jpg)
Goal
Shape as surface with depth and normal
![Page 33: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/33.jpg)
Recovering a surface from normals - 1
Recall the surface is
written as
This means the normal
has the form:
If we write the known
vector g as
Then we obtain values
for the partial derivatives
of the surface:
(x,y, f (x, y))
g(x,y)
g1(x, y)
g2 (x, y)
g3(x, y)
fx (x,y) g1(x, y) g3(x, y)
fy(x, y) g2(x,y) g3(x,y)
N(x,y)1
fx2
fy2
1
fx
fy
1
![Page 34: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/34.jpg)
Recovering a surface from normals - 2
Recall that mixed second partials are equal --- this gives us an integrability check. We must have:
We can now recover the
surface height at any
point by integration
along some path, e.g.
g1(x, y) g3(x, y) y
g2(x, y) g3(x, y) x
f (x, y) fx (s, y)ds0
x
fy (x,t)dt0
y
c
![Page 35: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/35.jpg)
Height Map from Integration
How to integrate?
![Page 36: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/36.jpg)
Possible Solutions
• Engineering approach: Path integration
(Forsyth & Ponce)
• In general: Calculus of Variation
Approaches
• Horn: Characteristic Strip Method
• Kimmel, Siddiqi, Kimia, Bruckstein: Level
set method
• Many others ….
![Page 37: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/37.jpg)
Shape by Integation (Forsyth&Ponce)
• The partial derivative gives the change in surface height
with a small step in either the x or the y direction
• We can get the surface by summing these changes in
height along some path.
![Page 38: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/38.jpg)
Simple Algorithm
Forsyth & Ponce
Problem: Noise and
numerical (in)accuracy are
added up and result in
distorted surface.
Solution: Choose several
different integration paths,
and build average height
map.
![Page 39: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/39.jpg)
Mathematical Property: Integrability
• Smooth, C2 continuous surface:
𝑍(𝑥, 𝑦)𝑥𝑦=𝑍(𝑥, 𝑦)𝑦𝑥
𝜕𝑝
𝜕𝑦=𝜕𝑞
𝜕𝑥
check if (𝜕𝑝
𝜕𝑦−
𝜕𝑞
𝜕𝑥)2 is small
![Page 40: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/40.jpg)
Wolff, November 4, 1998
SHAPE FROM SHADING
(Calculus of Variations Approach)
• First Attempt: Minimize error in agreement
with Image Irradiance Equation over the
region of interest:
( ( , ) ( , ))I x y R p q dxdyobject
2
![Page 41: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/41.jpg)
SHAPE FROM SHADING
(Calculus of Variations Approach)
• Better Attempt: Regularize the Minimization of
error in agreement with Image Irradiance Equation
over the region of interest:
p p q q I x y R p q dxdyx y x y
object
2 2 2 2 2 ( ( , ) ( , ))
Wolff, November 4, 1998
![Page 42: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/42.jpg)
Horn: Characteristic Strip Method
Horn,
Chapter11,
pp. 250-255 Small step in x,y → change in depth:
New values of p,q at this new point (x,y):
(r, s, t: second partial derivatives of z(x,y) w.r.t. x and y)
Hessian: curv. of surface
x
pr
x
q
y
ps
y
qt
![Page 43: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/43.jpg)
Horn: Characteristic Strip Method
Horn,
Chapter11,
pp. 250-255 Irradiance Equation, Reflectance Map:
Derivatives (chain rule):
Relationship between gradient
in the image and gradient in the
reflectance map
![Page 44: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/44.jpg)
Horn: Characteristic Strip Method
Horn,
Chapter11,
pp. 250-255 2 Equations for 3 unknowns (r,s,t): We
can’t continue in artibrary direction.
→ Trick: Specially chosen direction
Step in image E(x,y) parallel
to gradient in R
![Page 45: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/45.jpg)
Horn: Characteristic Strip Method
Horn,
Chapter11,
pp. 250-255 2 Equations for 3 unknowns (r,s,t): We
can’t continue in artibrary direction.
→ Trick: Specially chosen direction
Step in image E(x,y) parallel
to gradient in R
Solving for new values for p,q:
Change in (p,q) can
be computed via
gradient of image
![Page 46: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/46.jpg)
Horn: Characteristic Strip Method
![Page 47: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/47.jpg)
Horn: Characteristic Strip Method
Solution of differential equations: Curve on surface
![Page 48: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/48.jpg)
Horn: Characteristic Strip Method
![Page 49: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/49.jpg)
Horn: Characteristic Strip Method
Horn,
Chapter11,
pp. 250-255
![Page 51: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/51.jpg)
Kimmel, Siddiqi, Kimia, Bruckstein
![Page 52: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/52.jpg)
Kimmel, Siddiqi, Kimia, Bruckstein
![Page 53: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/53.jpg)
Kimmel, Siddiqi, Kimia, Bruckstein
![Page 54: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/54.jpg)
Application Area: Geography
![Page 56: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/56.jpg)
Mars Rover Heads to a New Crater NYT Sept 22, 2008
![Page 57: Photometric Stereo, Shape from Shading SfS Chapter 12.1.1 ... · Photometric Stereo, Shape from Shading SfS Chapter 12.1.1. Szelisky Guido Gerig CS 6320, Spring 2012 ... • Kimmel,](https://reader031.vdocuments.mx/reader031/viewer/2022021811/5c83bdc809d3f290718bf97f/html5/thumbnails/57.jpg)
Limitations
• Controlled lighting environment – Specular highlights?
– Partial shadows?
– Complex interrreflections?
• Fixed camera – Moving camera?
– Multiple cameras?
=> Another approach: binocular / geometric stereo