psfs and bokeh - aggregate.orgaggregate.org/dit/cp20110928.pdf · 9/28/2011 · psfs and bokeh...
TRANSCRIPT
![Page 1: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/1.jpg)
PSFs and Bokeh
Hank Dietz
Computational Photography, Sept. 28, 2011
University of KentuckyElectrical & Computer Engineering
![Page 2: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/2.jpg)
Computational Photography
● About using cameras to capture data for computational processing, rather than making a pretty image on the sensor...
● Multispectral image processing● New camera/sensor models● Intelligent computer control of capture● Detection/manipulation of image properties
![Page 3: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/3.jpg)
Spring 2009, EE499
● Jennifer Danhauer, Joe Lanford, Ross Levine● Project to capture a depthmap inside a Canon PowerShot using depthfromfocus
● CHDK scripting used so single press captures a sequence with different focus distances
● CHDK processing modified with custom C code to measure blur & combine images
● Blur measurement was fairly stateoftheart
![Page 4: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/4.jpg)
Measuring Focus Blur
● This is how contrastdetect autofocus works● Various algorithms in the literature for determining local contrast between pixels... Sobel worked best
● Actually done on raw sensor data using just the green pixel values (they have less noise)
● Limited memory in camera... some cleverness to avoid keeping N images in memory
![Page 5: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/5.jpg)
![Page 6: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/6.jpg)
How Good Is The Depthmap?
● Accurate depths at edges● No depth in featureless fields● Wrong depths near edges!
● Wrong by a lot● Wrong both directions● Seems to “echo” edges
![Page 7: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/7.jpg)
What Went Wrong?
● What does an outoffocus (OOF) point light source point spread function (PSF) – look like?
![Page 8: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/8.jpg)
What Went Wrong?
● Most image processing algorithms treat OOF point light sources as Gaussian blur:
![Page 9: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/9.jpg)
They Look Like This
● Note the sharp edge!
![Page 10: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/10.jpg)
Point Spread Function(PSF)
● Describes the response of an imaging system to a point source (impulse response)
● The spatial domain representation of the Modulation Transfer Function (MTF)
● An image is essentially the sum of the PSFs of all points of light in the scene
● PSF size grows in proportion to how OOF
![Page 11: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/11.jpg)
The Wrong PSF Model!
● But an OOF point light source imaged by a real lens (Takumar 135mm f/2.5) looks like:
![Page 12: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/12.jpg)
Each Lens PSF Is Unique
● For a Minolta 50mm f/1.7 with barely visible fungus it looks like this:
![Page 13: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/13.jpg)
Optical Formula Affects PSF
● For my Sony 1870mm zoom at 18mm f/3.5 the PSF looks like this:
![Page 14: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/14.jpg)
A Simple Retrofocus Lens PSF
● For my Vivitar 28mm f/2.5 it looks like this:
![Page 15: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/15.jpg)
A Classic Mirror Lens PSF
● For my Bower 500mm f/6.3 mirror lens the PSF looks like this:
![Page 16: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/16.jpg)
PSF with Spherical Aberration
Over
Under
Near Far
![Page 17: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/17.jpg)
Axial Chromatic Aberrations
![Page 18: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/18.jpg)
Cat's Eye / Swirl Vignetting
![Page 19: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/19.jpg)
Symmetric Near & Far
● Before and after focus are inverses (no, they are not ambiguous!)
● Funny things happen around focus● Different colors in focus at different depths● Partial “inversion” of pattern
![Page 20: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/20.jpg)
Computational PhotographyUsing PSF Structures
● After characterizing 100+ real lenses...
● Depthfromfocus/defocus, refocus/allinfocus● Diagnose lens defects like contamination and fabrication flaws (e.g., decentering)
● Forensically identify the type of lens● Forensically identify the specific lens● PSF substitution● Structured apertures & apodization
![Page 21: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/21.jpg)
Bokeh
● Japanesederived word for the general properties of OOF regions
● Good bokeh come from Gaussian blur PSFs● Nisen bokeh – double line artifacts
![Page 22: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/22.jpg)
Minolta's STF(Smooth Trans Focus)
● The Sony/Minolta 135mm f/2.8 t/4.5 STF incorporates an apodizing element
![Page 23: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/23.jpg)
Minolta'sMaxixum 7 STF Mode
● Fakes apodization using multiple exposures!● Here's my version using CHDK:
![Page 24: PSFs and Bokeh - Aggregate.Orgaggregate.org/DIT/cp20110928.pdf · 9/28/2011 · PSFs and Bokeh Hank Dietz Computational Photography, Sept. 28, 2011 University of Kentucky Electrical](https://reader034.vdocuments.mx/reader034/viewer/2022042807/5f78eae7088f3b5d1520066c/html5/thumbnails/24.jpg)
PSF Substitution
● Commonly attempted for image refocus● Can improve image Bokeh by replacing native PSF with a Gaussian blur (or other) PSF
● Can directly synthesize “3D” stereo pairs and enhance their apparent depth – more on that next lecture!