photometric image formation cse 559: computer vision guest lecturer: austin abrams images/demo from...
TRANSCRIPT
![Page 1: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/1.jpg)
Photometric Image Formation
CSE 559: Computer VisionGuest Lecturer: Austin Abrams
Images/Demo from Steve Seitz, Wikipedia
![Page 2: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/2.jpg)
How are images made?
• One half: geometric vision– “how the pixel projected onto the image”
• Today: photometric vision (aka radiometric)– “how the pixel got its color”
![Page 3: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/3.jpg)
Vision and Graphics
Properties of a sceneImage
Computer Graphics
Vision
![Page 4: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/4.jpg)
Image Formation Approach
• Come up with a model for how the scene was created
• Given images, find the most likely properties that fit that model
![Page 5: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/5.jpg)
Diffuse Surfaces
Brightness of a pixel depends on:• object color• lighting direction• surface normal
But NOT view direction!
![Page 6: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/6.jpg)
Lambertian Cosine Law
• The intensity of an observed diffuse object is proportional to the cosine of the angle between the normal and lighting direction
= ρ L N
I = ρ cos θ
= ρ |L||N| cos θL Nθ
![Page 7: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/7.jpg)
=
L N = L N
![Page 8: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/8.jpg)
= x
I = ρ L N
![Page 9: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/9.jpg)
Recovering Albedo and Normals
• Can you decompose a single image into its albedo and normal images?
![Page 10: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/10.jpg)
=
x
x
x
![Page 11: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/11.jpg)
Photometric Stereo
• Given multiple images taken with varying illumination, recover albedo and normals.– take pictures in dark room with varying
illumination.– estimate lighting directions L.– recover albedo and normals.
![Page 12: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/12.jpg)
Side note 1: How to get the lighting direction?
• Put a shiny sphere in the scene• Sphere’s geometry (normals) are known• Find specular highlight
![Page 13: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/13.jpg)
Side-note 2: Why “Stereo”?
Surface normals provide constraints on depth differences
![Page 14: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/14.jpg)
Photometric Stereo
• If L is known, and albedo is grayscale this is a linear problem.
I = ρ(L N) = ρ (Lx Nx + Ly Ny + Lz Nz ) = Lx Nxρ + Ly Nyρ + Lz Nzρ = Lx a + Ly b + Lz c
![Page 15: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/15.jpg)
Lx1 Ly1 Lz1
Lx2 Ly2 Lz2
Lx3 Ly3 Lz3
…
Lxn Lyn Lzn
I1I2I3…In
abc
=
I = ρ(L N) = Lx a + Ly b + Lz c
Then:ρ = sqrt(a2 + b2 + c2)N = (a,b,c) / ρ
For each pixel:
![Page 16: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/16.jpg)
Demo
![Page 17: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/17.jpg)
When does this model fail?
I ≠ ρ (L N)
![Page 18: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/18.jpg)
Attached shadows
I = ρ max(L N, 0)
L N > 0
L N = 0
L N < 0
![Page 19: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/19.jpg)
Cast Shadows, Ambient Light
I = ρ (S L N + a) S = 0 or 1
![Page 20: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/20.jpg)
Radiometric Camera Calibration
• Pixel intensities are usually not proportional to the energy that hit the CCD
RAW image Published image
![Page 21: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/21.jpg)
Radiometric Camera Calibration
f
RAW
Published
![Page 22: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/22.jpg)
Radiometric Camera Calibration
Observed = f(RAW)
(Grossberg and Nayar)
f -1 (Observed) = RAW
![Page 23: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/23.jpg)
Radiometric Camera Calibration
• How do you model f -1?
f -1(x) = xγ
f -1(x) = c0 + c1x + c2x2 + c3x3 + …
f -1(x) = f0(x) + f1(x) c1 + f2(x)c2 + …
mean camera curve basis camera curves
![Page 24: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/24.jpg)
Radiometric Camera Calibration
I = f (ρ (S L N + a))
Adding exposure:
I = f (e ρ (S L N + a))
![Page 25: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/25.jpg)
Heliometric StereoGiven lots of images from a stable webcam,
use lighting from the sun to recover:
I = f (e ρ (S L N + a))
![Page 26: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/26.jpg)
Heliometric Stereo
![Page 27: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/27.jpg)
Heliometric Stereo
![Page 28: Photometric Image Formation CSE 559: Computer Vision Guest Lecturer: Austin Abrams Images/Demo from Steve Seitz, Wikipedia](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551a98b4550346b52d8b61bd/html5/thumbnails/28.jpg)
Heliometric Stereo