introduction to computational photography behzad sajadimajumder/photo/intro.pdf · what is...

38
Introduction to Computational Photography Behzad Sajadi Borrowed from Frédo Durand’s Lectures at MIT

Upload: lebao

Post on 25-Aug-2018

223 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Introduction to Computational Photography

Behzad Sajadi

Borrowed from Frédo Durand’s Lectures at MIT

Page 2: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Today's plan• Introduction of Computational Photography• Introduction to Digital Imaging

Page 3: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

What is computational photography• Convergence of image processing, computer vision,

computer graphics and photography• Digital photography

– Simply replaces traditional sensors and recording by digital technology

– Involves only simple image processing• Computational photography

– More elaborate image manipulation and computation– New types of media (panorama, 3D, etc.)– Camera design that takes computation into account

Page 4: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Examples• Tone mapping• Defocus Matting• Motion magnification• Multi-Modal Imaging

Page 5: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Tone mapping

Before After

• Suitable for HDR images

Page 6: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Motion magnification

http://people.csail.mit.edu/celiu/motionmag/motionmag.html

Original Magnified

Page 7: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Defocus Matting• What can be achieved

• Design: use 3 streams with different focus

http://people.csail.mit.edu/wojciech/DefocusVideo/index.html

Page 8: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Multi-Modal Cameras• What can be achieved

• How it works?

CMY

Camera

RGB

Camera

CMY

Camera

RGB

Camera

Dark Condition Lighted Condition

Ground Truth 6-Primary Camera RGB Camera

http://www.ics.uci.edu/~bsajadi/SwitchableCam.html

CMY Camera

Page 9: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Today's plan• Introduction of Computational Photography• Introduction to Digital Imaging

Page 10: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Overview• Lens and viewpoint determine perspective• Aperture and shutter speed determine exposure• Aperture and other effects determine depth of field• Film or sensor record image

Page 11: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Reference• http://courses.csail.mit.edu/6.869/lectnotes/lect1• http://en.wikipedia.org/wiki/Lens_(optics)

• The slides use illustrations from these books

Page 12: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

More references

Page 13: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Plan• Pinhole optics• Lenses• Exposure

Page 14: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

7-year old’s question

• Why is there no image on a white piece of paper?

Page 15: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

It receives light from all directions

From Photography, London et al.

Page 16: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Pinhole

From Photography, London et al.

Page 17: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Focal length

pinhole

Film/sensor scene

f

s

Page 18: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Focal length: pinhole optics• What happens when the focal length is doubled?

– Projected object size– Amount of light gathered

pinhole

Film/sensor scene

f d

s

2f

is doubledis divided by 4

Page 19: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Questions?

Page 20: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Pinhole size?

From Photography, London et al.

Page 21: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Diffraction limit• Optimal size for visible light:

sqrt(f)/28 (in millimiters) where f is focal length

From Wandell

Page 22: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Problem with pinhole?• Not enough light!• Diffraction limits sharpness

Page 23: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Solution: refraction!

From Photography, London et al.

Page 24: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Lenses • gather

more light!

• But need to be focused

From Photography, London et al.

Page 25: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

• Simplification of geometrical optics for well-behaved lenses

• All parallel rays converge to one point on a plane located at the focal length f

• All rays going through the center are not deviated– Hence same perspective as pinhole

Thin lens optics

f

Page 26: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

How to trace rays• Start by rays through the center

Page 27: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

How to trace rays• Start by rays through the center• Choose focal length, trace parallels

f

Page 28: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

How to trace rays• Start by rays through the center• Choose focal length, trace parallels• You get the focus plane for a given scene plane

– All rays coming from points on a plane parallel to the lens are focused on another plane parallel to the lens

f

Page 29: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Focusing• To focus closer than infinity

– Move the sensor/film further than the focal length

f

Page 30: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Thin lens formula

fDD’

Page 31: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Thin lens formula

fDD’

Similar triangles everywhere!

Page 32: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Thin lens formula

fDD’

Similar triangles everywhere!

y’y

y’/y = D’/D

Page 33: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Thin lens formula

fDD’

Similar triangles everywhere!

y’y

y’/y = D’/Dy’/y = (D’-f)/D

Page 34: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Thin lens formula

fDD’

1D’ D

1 1f+ =

Page 35: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Minimum focusing distance• By symmetry, an object at the focal length requires

the film to be at infinity.

film

Rays from infinity

Rays from object at f

Page 36: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Extensions tubes• Allow us to put sensor/film farther

focus closer

Page 37: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Field of view & focusing• What happens to the field of view when one focuses closer?

– It's reduced

film focused at infinity

film focused close

Page 38: Introduction to Computational Photography Behzad Sajadimajumder/PHOTO/Intro.pdf · What is computational photography • Convergence of image processing, computer vision, computer

Questions?• http://www.pinhole.cz/en/pinholecameras/dirkon_01.

html