out-of-focus point spread functions - aggregate.orgaggregate.org/dit/ei20140204.pdf · out-of-focus...
TRANSCRIPT
![Page 1: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/1.jpg)
Out-of-focuspoint spread functions
Prof. Hank Dietz
EI, February 4, 2014
University of KentuckyElectrical & Computer Engineering
![Page 2: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/2.jpg)
Canon Hack Development Kit (CHDK)
Enables running arbitrary C code in a Canon PowerShot with full access to camera
![Page 3: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/3.jpg)
Spring 2009, EE499
∙ Jennifer Danhauer, Joe Lanford, RossLevine project to capture a depthmap insidea Canon PowerShot using depth-from-focus
∙ CHDK scripting so a single press captures a sequence with different focus distances ∙ CHDK processing modified with custom C code to measure blur & combine images ∙ Blur measurement was fairly state-of-the-art
![Page 4: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/4.jpg)
![Page 5: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/5.jpg)
How Good Is The Depthmap?
∙ Accurate depths at edges ∙ No depth in featureless fields ∙ Wrong depths near edges! ∙ Wrong by a lot ∙ Wrong both directions ∙ Seems to “echo” edges...
![Page 6: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/6.jpg)
What Went Wrong?
∙ Most image processing algorithms modelout-of-focus (OOF) as Gaussian blur:
![Page 7: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/7.jpg)
Out-Of-Focus Isn't Blurry!
∙ OOF point spread function (PSF) typicallyhas a sharp edge!
![Page 8: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/8.jpg)
Why The Sharp Edge?
The lens aperture occuldes rays:
![Page 9: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/9.jpg)
PSFs, In Focus Or Not
∙ PSF describes the response of an imagingsystem to a point source (impulse response);it is the spatial domain representation ofthe modulation transfer function (MTF)
∙ Image is the sum of the non-occulded partsof the PSFs of all points of light in the scene
∙ What do OOF PSFs of real lenses look like?
![Page 10: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/10.jpg)
Measuring OOF PSF
∙ Work in stable, dark, unobstructed, area∙ Place point light source at 10m
(often can use a white LED penlight)∙ Manual focus to 1m, 2m, or 3m∙ Expose to show detail inside OOF PSF
I've collected & measured over 125 lenses
![Page 11: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/11.jpg)
Which Lenses? Fixed+ZoomF (mm) F/1 F/1.4 F/2 F/2.8 F/4 F/5.6 F/11
20 1 5+5 +3 1
24 1 4 1+1
28 5 1+5
40 4 2 2+5
60 2 6 16 1 1+4 +3
135 2 2 9+1 9+7 3+9
250 3+7 +2 2
500 1 1+2 5
![Page 12: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/12.jpg)
Why Bother?
∙ Justifies LBA to spouse ;-)∙ OOF PSF is easy to measure∙ OOF PSF is not the same for all lenses:
– Diagnose inherent & acquired lens defects– Forensic applications– Recovery of depth & stereo capture– Predict & shape bokeh
![Page 13: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/13.jpg)
Diagnostic Use
∙ Ever buy a used lens?∙ Two classes of lens defects:
– Inherent from design or manufacture– Acquired from use, storage, and age
![Page 14: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/14.jpg)
Vignetting
![Page 15: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/15.jpg)
Decentering
![Page 16: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/16.jpg)
Axial Chromatic Aberration
![Page 17: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/17.jpg)
Undercorrected / Overcorrected Spherical Aberration
![Page 18: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/18.jpg)
Dust & Dirt
![Page 19: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/19.jpg)
Oily Fingerprint
![Page 20: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/20.jpg)
Fungus Infection
![Page 21: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/21.jpg)
Nicked Element
![Page 22: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/22.jpg)
Element Separation
![Page 23: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/23.jpg)
Forensic Use
∙ Identify faked images∙ Make more credible fakes
(e.g., for computer-generated movie effects)∙ Identify the lens used:
– Determine most likely type of lens– Distinguish between similar lenses– Acquired defects are “lens fingerprints”
![Page 24: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/24.jpg)
Compact Camera Lens
![Page 25: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/25.jpg)
Ultrawide Zoom (corner)
![Page 26: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/26.jpg)
Conventional Telephoto
![Page 27: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/27.jpg)
Mirror Lens
![Page 28: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/28.jpg)
Distinguishing Similar Lenses
∙ Four very similar lenses, @ f/1.4 and f/5.6∙ Vignetting, blade count/shape/rotation...
![Page 29: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/29.jpg)
Recovery Of Depth
∙ Recognize OOF PSF pattern – not “blur”∙ The signed diameter of the OOF PSF is:
Const1 – (Const
2 / ObjectDistance)
∙ How can we distinguish before/after focus?– Stuff in front can occlude stuff behind– OOF PSF is “turned inside out”
![Page 30: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/30.jpg)
Before Or After Focus Point?
![Page 31: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/31.jpg)
Axial CA After / Before Focus
![Page 32: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/32.jpg)
Axial CA in a photo
![Page 33: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/33.jpg)
Extreme Undercorrected SA,After / Before Focus
![Page 34: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/34.jpg)
Extreme Undercorrected SAin a photo
![Page 35: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/35.jpg)
Depth And Multiple Views
∙ Multiple viewpoints in OOF PSF can becomputationally extracted
∙ Phase detect autofocus (PDAF) suggestsdirection & amount to change focus in onereading – so can OOF PSF recognition
∙ Light field can be extracted(similar to coded aperture or plenoptic)
∙ Stereo pairs can be extracted
![Page 36: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/36.jpg)
Bokeh
∙ The properties of OOF regions of images∙ Not about quantity or size of OOF things,
hence often considered a “magical” property∙ Good bokeh look smooth, bad don't∙ Worst is nisen bokeh double line artifacts
![Page 37: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/37.jpg)
Both Good & Bad Bokeh Here
![Page 38: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/38.jpg)
OOF PSFs Make The Bokeh
∙ Bright center ⇒ good bokeh∙ Bright outer ring ⇒ nisen bokeh∙ Vignette + field curvature ⇒ “swirly” bokeh∙ Axial CA ⇒ “bokeh CA”∙ Can shape (apodize) OOF PSF to improve
![Page 39: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/39.jpg)
Conclusion
∙ Out-of-focus really isn't blurry:measuring OOF PSFs is easy & worthwhile
∙ The OOF PSF tells you a lot about a lens...and about the scene (e.g., depth & stereo)
∙ Understanding & manipulating OOF PSF canenable things you couldn't do otherwise
∙ Lots of algorithms to develop...∙ I guess Hank isn't going to show us all 125+
of his lenses....
![Page 40: Out-of-focus point spread functions - Aggregate.Orgaggregate.org/DIT/ei20140204.pdf · Out-of-focus point spread functions Prof. Hank Dietz EI, February 4, ... of the PSFs of all](https://reader031.vdocuments.mx/reader031/viewer/2022020204/5b15cbdb7f8b9a332f8e2154/html5/thumbnails/40.jpg)
Want To Know More?
Watch our research WWW site: