1

Diffusion Coded Photography

for Extended Depth of Field

SIGGRAPH 2010Oliver Cossairt, Changyin Zhou, Shree Nayar

Columbia University

Supported by ONR and NSF

2

Conventional Camera (F/1.8)

3

Conventional Camera (F/18)

4

Focused ImagePSF

Image Noise Captured Image

Camera Blur ModelS

pati

al d

om

ain

Focused Image MTF Image Noise Captured ImageFreq

uen

cy d

om

ain

(Modulated Transfer Function)

5

Deblurring Problems

MTFCaptured image

Low SNR

Low MTF values

Problem 1:

Focused image

Lens

Sensor

P

Object

Q

Problem 2:

Variation with depth

6

Extending Depth of Field (EDOF): Previous Work

Focal Sweep Cameras[Hausler ’72]

[Nagahara et al. ’08]Lens Focal PlaneSensor

Other Related Work[Levin et al. ’07][Veeraraghavan et al. ’07] [Levin et al. ’09]

Wavefront Coding Cameras

[Dowski and Cathey ’95] [Chi and George ’01] [Garcia-Guerrero et al. ‘07] LensSensor

Cubic Phase Plate

7

Focal sweep vs. wavefront coding

Focal Sweep

WavefrontCoding

ConventionalCamera

near farfocus

depth

Note: only a single PSF will be used to deblur the whole image.

8

Deblurring Error vs. DepthD

eblu

rrin

g E

rror

noise

DepthWavefront

CodingFocal Sweep

2

0FFE dd

Focal sweep vs. wavefront coding

9

Achieve the performance of focal sweep without any moving parts?

10

Optical Diffusers

Diffuser

[http://www.luminitco.com]

Diffuser sheets

w

SensorDiffuser

Light ray

x

x

Scatter function

w

SEM image

11

Diffuser Kernels

xu A/2

-A/2

u

x

Light FieldWithout a diffuser:

Lens

A

Sensor

12

Diffuser Kernels

A/2

-A/2

u

x

Light FieldWithout a diffuser:

With a diffuser:

A/2

-A/2

u

x

xu

Lens Sensor

xu

Lens Sensor

A

A

13

Diffuser Kernels

A/2

-A/2

u

x

Light FieldWithout a diffuser:

With a diffuser:

u

x

A/2

-A/2

xu

Lens Sensor

xu

Lens Sensor

A

A

14

Diffusion Kernels

A/2

-A/2

u

x

Without diffuser

u

x

With diffuser

u

x

Diffuser kernel

Lig

ht

field

15

A/2

-A/2

u

x

u

x

u

x

Lig

ht

field

Diffusion Kernels

x

project

x

project

x

project

PS

F

Diffused PSFScatter functionCamera PSF

Without diffuser With diffuserDiffuser kernel

16

Radially Symmetric Diffuser

LensSensor

PSFWithout a diffuser:

17

Without a diffuser:

LensSensor

With a diffuser:

LensSensor

PSF

Radially Symmetric Diffuser

18

LensSensor

With a diffuser:

LensSensor

PSFWithout a diffuser:

Radially Symmetric Diffuser

19

LensSensor

LensSensor

PSF

With a diffuser:

Without a diffuser:

Radially Symmetric Diffuser

20Lens

Sensor

With a diffuser:

Radially Symmetric Diffuser

LensSensor

PSFWithout a diffuser:

21

LensSensor

LensSensor

PSF

With a diffuser:

Without a diffuser:

Radially Symmetric Diffuser

22

LensSensor

LensSensor

PSF

With a diffuser:

Without a diffuser:

Radially Symmetric Diffuser

23

Radially Symmetric Diffuser

PSF:

Scatter function

CameraPSF

PSF(1D slice)

MTF(1D slice)

Normalized frequency Normalized frequency Normalized frequency Normalized frequency

K7

K9

K11

K13

k7

k9

k11

k13

-50px 50px -50px 50px -50px 50px -50px 50px

depth

24

Diffusion Coding PerformanceDeblurring Error vs. Depth

Depth

Wavefront CodingFocus Sweep

noise

Diffusion Coding (light field)Diffusion Coding (wave optics)

Similar performance to focal sweep without moving parts

Deb

lurr

ing

Err

or

25

Diffuser ImplementationDiffuser scatter function

)(rf

r (mm)

[www.rpcphotonics.com]

Diffuser height map Fabricated diffuser

110

Thic

kn

ess

(u

m)

3

863

2

1

)(rt

r (mm)

Diffuser surface profile

RPC Photonics

26

Garcia-Guerrero

Comparison with Prior Work

Deblurring Error vs. Depth

Depth

Diffusion Coding

Diffusion coding significantly outperforms prior work.

Deblu

rrin

g E

rror

x 16

1

27

Diffusion Coding Experiments

Fabricated DiffuserCannon 50mm EF lensCannon 450D Sensor

Experimental Setup

Measured PSFs

depth

Without diffuser

With diffuser

28

Examples

29

Stuffed Toys

f-number = 1.8, exposure time = 16ms

Conventional Camera

30

Stuffed Toys

f-number = 18 , exposure time = 16ms

Conventional Camera

31

Stuffed Toys

Diffusion Coding Camera: Captured

f-number = 1.8, exposure time = 16ms

32

Stuffed Toys

f-number = 1.8, exposure time = 16ms

Diffusion Coding Camera: Deblurred

33

Statues

f-number = 1.8; exposure time = 10ms

Captured Deblurred

34

People and Flowers

f-number = 1.8, exposure time = 16ms

Conventional Camera

35

People and Flowers

f-number = 1.8, exposure time = 16ms

Diffusion Coding Camera: Captured

36

People and Flowers

f-number = 1.8, exposure time = 16ms

Diffusion Coding Camera: Deblurred

37

LimitationsConventional Camera Diffusion Coding

• Loss of image texture

• Loss of contrast

• Occlusion errors

38

Conclusions

Diffusion Coding Theory

Diffusion Coding Implementation)(rf

Diffusion Coding Examples

Radially Symmetric Diffusers

39

Diffusion Coded Photography

for Extended Depth of Field

SIGGRAPH 2010Oliver Cossairt, Changyin Zhou, Shree Nayar

Columbia University

Supported by ONR and NSF