post-processing pipeline - computer graphics laboratory - stanford
TRANSCRIPT
![Page 1: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/1.jpg)
Post-processing pipelineCS 448A, Winter 2010
Marc LevoyComputer Science DepartmentStanford University
![Page 2: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/2.jpg)
© 2010 Marc Levoy
Camera pixel pipeline
✦ every camera uses different algorithms
✦ the processing order may vary
✦ most of it is proprietary2
sensor
processing:demosaicing,
tone mapping &white balancing,
denoising &sharpening,compression
analog to digitalconversion
(ADC)storage
![Page 3: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/3.jpg)
© 2010 Marc Levoy
Sensing color
✦ silicon detects all visible frequencies well
✦ can’t differentiate wavelengthsafter photon knocks an electron loose• all electrons look alike
✦ must select desired frequenciesbefore light reaches photodetector• block using a filter, or separate using a prism or grating
✦ 3 spectral responses is enough• a few consumer cameras record 4
✦ silicon is also sensitive to near infrared (NIR)• most sensors have an IR filter to block it• to make a NIR camera, remove this filter
3
![Page 4: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/4.jpg)
© 2010 Marc Levoy
Color sensing technologies
✦ field-sequential
✦ 3-sensor
✦ vertically stacked
✦ color filter arrays
4
![Page 5: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/5.jpg)
© 2010 Marc Levoy
Sergey Prokudin-Gorsky
• shot sequentially through R, G, B filters• simultaneous projection provided good saturation,
but available printing technology did not• digital restoration lets us see them in full glory...
5
![Page 6: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/6.jpg)
Sergey Prokudin-Gorsky, Alim Khan, emir of Bukhara (1911)
![Page 7: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/7.jpg)
Sergey Prokudin-Gorsky,Pinkhus Karlinskii, Supervisor of the Chernigov Floodgate (1919)
![Page 8: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/8.jpg)
© 2010 Marc Levoy
3-CCD cameras
✦ high-quality video cameras
✦ prism & dichroic mirrors split the image into 3 colors, each routed to a separate CCD sensor
✦ no light loss, as compared to filters
✦ expensive, and complicates lens design8
(Theuwissen)
![Page 9: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/9.jpg)
© 2010 Marc Levoy
Foveon stacked sensor
✦ longer wavelengths penetrate deeper into silicon,so arrange a set of vertically stacked detectors• top gets mostly blue, middle gets green, bottom gets red• no control over spectral responses, so requires processing
✦ fewer color artifacts than color filter arrays• but possibly worse noise performance
9
![Page 10: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/10.jpg)
© 2010 Marc Levoy
Color filter arrays
✦ Why more green pixels than red or blue?• because humans are most
sensitive in the middle ofthe visible spectrum
• sensitivity given by the humanluminous efficiency curve
10
G R
B G
Bayer pattern Sony RGB+Ebetter color
Kodak RGB+Cless noise
(Stone)
![Page 11: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/11.jpg)
© 2010 Marc Levoy
Example of Bayer mosaic image
11
Small fan atStanford women’s soccer game
(Canon 1D III)
![Page 12: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/12.jpg)
© 2010 Marc Levoy
Example of Bayer mosaic image
12
![Page 13: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/13.jpg)
© 2010 Marc Levoy
Before demosaicking (dcraw -d)
13
![Page 14: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/14.jpg)
© 2010 Marc Levoy
Demosaicing
✦ linear interpolation• average of the 4 nearest neighbors
✦ smoother kernels are possible• e.g. bicubic interpolation (what Photoshop uses by default)• but need more neighbors (16 instead of 4)
14
![Page 15: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/15.jpg)
© 2010 Marc Levoy
Demosaicing errors
✦ color fringesor moiré
✦ the cause of color moiré• fine black and white detail in
scene is mis-interpreted byinterpolation algorithmas color information
15
simplified1D detector
fine diagonalB&W stripes
![Page 16: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/16.jpg)
© 2010 Marc Levoy
Common solution:low-pass filter chrominance signal
✦ color artifacts are places where chrominance changes abruptly but only transiently
16
![Page 17: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/17.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
17
originalimage
(Wandell)
Y
Cb
Cr
![Page 18: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/18.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
18
(Wandell)
red-greenchannel blurred
Y
Cb
Cr
![Page 19: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/19.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
19
originalimage
(Wandell)
Y
Cb
Cr
![Page 20: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/20.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
20
(Wandell)
blue-yellowchannel blurred
Y
Cb
Cr
![Page 21: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/21.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
21
originalimage
(Wandell)
Y
Cb
Cr
![Page 22: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/22.jpg)
© 2010 Marc Levoy
Apparent spatial sharpness dependsmainly on luminance, not chrominance
22
(Wandell)
luminancechannel blurred
Y
Cb
Cr
![Page 23: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/23.jpg)
© 2010 Marc Levoy
Common solution:low-pass filter chrominance signal
✦ color artifacts are places where chrominance changes abruptly but only transiently
✦ use a median filter on chrominance to remove outlier transient chrominance changes [Freeman 1988]
• replace the chrominance of each pixel by themedian value in a neighborhood
• this is a non-linear filter
23
5-pixel neighborhood
in: out:
in: out:
spike noise is removed
monotonic edges remain unchanged
![Page 24: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/24.jpg)
© 2010 Marc Levoy
Common solution:low-pass filter chrominance signal
✦ color artifacts are places where chrominance changes abruptly but only transiently
✦ use a median filter on chrominance to remove outlier transient chrominance changes [Freeman 1988]
• replace the chrominance of each pixel by themedian value in a neighborhood
• this is a non-linear filter
✦ summary of algorithm• 1. apply naive interpolation• 2. convert to YCbCr• 3. median filter Cr & Cb• 4. reconstruct R, G, B from
sensor value and filtered Cr & Cb24
(wikipedia)
Y
Cb
Cr
![Page 25: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/25.jpg)
© 2010 Marc Levoy
Comparison
✦ take-home lesson: 2/3 of your data is made up!
✦ there are better and worse ways to do this25
linear interpolation median-filtered interpolation
![Page 26: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/26.jpg)
© 2010 Marc Levoy
Camera pixel pipeline
26
sensor
processing:demosaicing,
tone mapping &white balancing,
denoising &sharpening,compression
analog to digitalconversion
(ADC)storage
![Page 27: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/27.jpg)
© 2010 Marc Levoy
Summary of chromaticity diagrams (1 of 2)
✦ choose three primaries R,G,B, pure wavelengths or not
✦ adjust R=1,G=1,B=1 to obtain a desired reference white
✦ this yields an RGB cube
✦ one may factor the brightness out of anypoint in the cube by drawing a line to theorigin and intersecting this line with thetriangle made by corners Red, Green, Blue
✦ all points on this triangle, which areaddressable by two coordinates, have thesame brightness but differing chromaticity
27r
g
(Flash demo)http://graphics.stanford.edu/courses/
cs178/applets/threedgamut.html
r =R
R + G + B
g =G
R + G + B
![Page 28: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/28.jpg)
© 2010 Marc Levoy
Summary of chromaticity diagrams (2 of 2)
✦ this triangle is called the rgb chromaticity diagram for the chosen RGB primaries• mixtures of colors lie along straight lines• neutral (black to white) lies at (⅓, ⅓)• r>0, g>0 does not enclose spectral locus
✦ the same construction can be performed using any set of 3 vectors as primaries, even physically impossible ones
✦ the CIE has defined a set of primaries XYZ, and the associated xyz chromaticity diagram• x>0, y>0 does enclose spectral locus• one can connect red and green on the
locus with a line of extra-spectral purples• x,y is a standardized way to denote colors
28
r
g rgbchromaticity
diagram
(Hunt)
CIE xyzchromaticity
diagram
x
y
![Page 29: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/29.jpg)
© 2010 Marc Levoy
Application of chromaticity diagrams #1:color temperature and white balancing
✦ the apparent colors emitted by a black-body radiator heated to different temperatures fall on a curve in the chromaticity diagram
✦ for non-blackbody sources, the nearest point on the curve is called the correlated color temperature29
correlated color temperatures 3200°K incandescent light 4000°K cool white fluorescent 5000°K equal energy white (D50, E) 6000°K midday sun, photo flash 6500°K overcast, television (D65) 7500°K northern blue sky
(wikipedia)
![Page 30: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/30.jpg)
© 2010 Marc Levoy
White balancing in digital photography
✦ 1. choose an object in the photograph you think is neutral(somewhere between black and white) in the real world
✦ 2. compute scale factors (SR,SG,SB) that map the object’s (R,G,B) to neutral (R=G=B), i.e. SR = ⅓ (R+G+B) / R, etc.
✦ 3. apply the same scaling to all pixels in the sensed image
✦ the eventual appearance of R=G=B, hence of your chosen object, depends on the color space of the camera• the color space of most digital cameras is sRGB• the reference white for sRGB is D65 (6500°K)
✦ thus, white balancing on an sRGB camera forcesyour chosen object to appear 6500°K (blueish white)
✦ if you trust your object to be neutral, this procedure is equivalent to finding the color temperature of the illumination
30
![Page 31: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/31.jpg)
© 2010 Marc Levoy
✦ Auto White Balance (AWB)• gray world: assume the average color of a scene is gray, so
force the average color to be gray - often inappropriate
Finding the color temperature of the illumination
31
average (R, G, B) = (100%, 81%, 73%) ➔ (100%, 100% 100%)
(Marc Levoy)
![Page 32: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/32.jpg)
© 2010 Marc Levoy
✦ Auto White Balance (AWB)• gray world: assume the average color of a scene is gray, so
force the average color to be gray - often inappropriate• assume the brightest pixel (after demosaicing) is a specular
highlight and therefore should be white- fails if pixel is saturated- fails if object is metallic - gold has gold-colored highlights
• find a neutral-colored object in the scene- but how??
Finding the color temperature of the illumination
32 (Nikon patent)
![Page 33: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/33.jpg)
© 2010 Marc Levoy
✦ Auto White Balance (AWB)
✦ manually specify the illumination’s color temperature• each color temperature corresponds to a unique (x,y)• for a given camera, one can measure the (R,G,B) values
recorded when a neutral object is illuminated by this (x,y)• compute scale factors (SR,SG,SB) that map this (R,G,B) to
neutral (R=G=B); apply this scaling to all pixels as before
Finding the color temperature of the illumination
33
tungsten: 3,200Kfluorescent: 4,000K
daylight: 5,200Kcloudy or hazy:
6,000Kflash: 6,000Kshaded places: 7,000K
![Page 34: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/34.jpg)
© 2010 Marc Levoy
Incorrectly chosen white balance
✦ scene was photographed in sunlight, then re-balanced as if it had been photographed under something warmer, like tungsten• re-balancer assumed illumination was very reddish, so it boosted blues• same thing would have happened if originally shot with tungsten WB
34
(Eddy Talvala)
![Page 35: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/35.jpg)
© 2010 Marc Levoy
Contrast correction (a.k.a. tone mapping)
✦ manual editing• store image in RAW mode, then fiddle with histogram in
Photoshop, dcraw, Canon Digital Photo Professional, etc.
35 (cambridgeincoulour.com)
![Page 36: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/36.jpg)
© 2010 Marc Levoy
Contrast correction (a.k.a. tone mapping)
✦ manual editing• store image in RAW mode, then fiddle with histogram in
Photoshop, dcraw, Canon Digital Photo Professional, etc.
✦ gamma transform• output = inputγ (for 0 ≤ Ii ≤ 1)• simple but crude
36γ = 0.5 γ = 2.0original
![Page 37: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/37.jpg)
© 2010 Marc Levoy
Contrast correction (a.k.a. tone mapping)
✦ manual editing• store image in RAW mode, then fiddle with histogram in
Photoshop, dcraw, Canon Digital Photo Professional, etc.
✦ gamma transform• output = inputγ (for 0 ≤ Ii ≤ 1)• simple but crude
✦ global versus local transformations
37
![Page 38: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/38.jpg)
© 2010 Marc Levoy
Traditional dodging and burning
38
dodging(leaves print lighter)
burning(makes print darker)
Ansel Adams in his darkroom
(Rudman)
(Adams)
![Page 39: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/39.jpg)
Ansel Adams, Clearing Winter Storm, 1942
straightprint
![Page 40: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/40.jpg)
Ansel Adams, Clearing Winter Storm, 1942
tonedprint
![Page 41: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/41.jpg)
© 2010 Marc Levoy
Camera pixel pipeline
41
sensor
processing:demosaicing,
tone mapping &white balancing,
denoising &sharpening,compression
analog to digitalconversion
(ADC)storage
![Page 42: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/42.jpg)
© 2010 Marc Levoy
Canon 5D II at dusk
42
• ISO 6400• f/4.0• 1/13 sec
![Page 43: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/43.jpg)
© 2010 Marc Levoy
Denoising
✦ goal is to remove sensor noise• blurring works, but also destroys edges• I don’t know what Canon does,
but here’s something that works...43
RAW (ISO 6400) Gaussian blur, radius = 1.3 Canon denoising
![Page 44: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/44.jpg)
© 2010 Marc Levoy
Bilateral filtering [Tomasi ICCV 1998]
✦ assume the image is piecewise constant with noise added
✦ therefore, nearby pixels are probably a different noisy measurement of the same data
✦ blurring doesn’t work
✦ we should do a weighted blur where the weight is the probability a pixel is from the same piece of the scene
44
⊗
=
![Page 45: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/45.jpg)
© 2010 Marc Levoy
Bilateral filtering✦ if the pixels are similar in intensity,
the probability they are from the same piece of the scene is high
✦ so perform a convolution where the weight assigned to nearby pixels falls off• with increasing (x,y) distance from the
pixel being blurred• with increasing intensity difference
from the pixel being blurred
✦ i.e. blur in the domain and range dimensions!
45
⊗
=
![Page 46: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/46.jpg)
© 2010 Marc Levoy
Denoising
✦ bilateral filtering removes sensor noise without blurring edges
✦ can easily be extended to RGB
46
RAW (ISO 6400)
Gaussian blur, radius = 1.3 Canon denoising
bilateral filtering
![Page 47: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/47.jpg)
© 2010 Marc Levoy
Denoising
✦ can be applied more (or less) strongly to chrominance than luminance
✦ can be combined with demosaicing
✦ active area of research...47
RAW (ISO 6400)
Gaussian blur, radius = 1.3 Canon denoising
bilateral filtering
![Page 48: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/48.jpg)
© 2010 Marc Levoy
Sharpening
48
original
(Marc Levoy)
![Page 49: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/49.jpg)
© 2010 Marc Levoy
Sharpening
49
Filter/Other/Custom in Photoshop CS4
(Marc Levoy)
![Page 50: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/50.jpg)
© 2010 Marc Levoy
Sharpening
50
Filter/Other/Custom in Photoshop CS4
(Marc Levoy)
![Page 51: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/51.jpg)
© 2010 Marc Levoy
Sharpening
51
original
(Marc Levoy)
![Page 52: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/52.jpg)
© 2010 Marc Levoy
Sharpening
52
1st layer is original,2nd layer is sharpened,blend w. 30% opacity
(Marc Levoy)
![Page 53: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/53.jpg)
© 2010 Marc Levoy
Sharpening
53
Filter/Other/Custom in Photoshop CS4
![Page 54: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/54.jpg)
© 2010 Marc Levoy
Unsharp masking
✦ blend between original and high-contrast version, controlled by a mask that represents scene edges
✦ dropping (thresholding) the darkest mask pixelsavoids sharpening noise, and makes the filter non-linear
5453
(cambridgecolor.com)
(cambridgecolor.com)
![Page 55: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/55.jpg)
© 2010 Marc Levoy
Sharpening
55
Filter/Other/Custom in Photoshop CS4
![Page 56: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/56.jpg)
© 2010 Marc Levoy
Sharpening
56
Filter/Sharpen/Unsharp Mask in CS4
![Page 57: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/57.jpg)
© 2010 Marc Levoy
Sharpening
57
original
![Page 58: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/58.jpg)
© 2010 Marc Levoy
Camera pixel pipeline
58
sensor
processing:demosaicing,
tone mapping &white balancing,
denoising &sharpening,compression
analog to digitalconversion
(ADC)storage
![Page 59: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/59.jpg)
© 2010 Marc Levoy
JPEG files
✦ Joint Photographic Experts Group• organized 1986, standard adopted 1994
✦ defines how an image is to be compressed into a stream of bytes (codec) and the file format for storing that stream• file format is JFIF, but people use .JPG or .JPEG extensions
✦ good for compressing images of natural scenes;not so good for compressing drawings or graphics
✦ lossy, so loses quality each time you open → edit → save• especially if you crop or shift pixels (hence block boundaries) • for lossless compression, use PNG or TIFF
59
![Page 60: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/60.jpg)
© 2010 Marc Levoy
EXIF data
✦ Exchangeable Image File Format• created by Japan Electronic Industries Development Assoc.
✦ used by nearly every digital camera manufactured today• actually a file format• JPEG or TIFF file + metadata about the camera and shot• .JPG or .JPEG extension is used
60
![Page 61: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/61.jpg)
© 2010 Marc Levoy
EXIF data
61
File/File Info inPhotoshop CS4
shot with Canon 5D Mark II
(Marc Levoy)
![Page 62: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/62.jpg)
© 2010 Marc Levoy
EXIF data
62
exiftool
shot with Canon 5D Mark II
![Page 63: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/63.jpg)
© 2010 Marc Levoy
EXIF data
63
Mac Preview
shot with iPhone 3G
![Page 64: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/64.jpg)
© 2010 Marc Levoy
RAW files
✦ minimally processed images, not even demosaiced
✦ uncompressed or losslessly compressed
✦ includes metadata, possibly encrypted
✦ file format varies by manufacturer
✦ example extensions: .CR2, .NEF, .RW2
✦ processed and converted to a JPEG file using• proprietary software (e.g. Canon Digital Photo Professional)• Photoshop or Lightroom (if you’re lucky)• freeware programs like dcraw• but their processing algorithms are all different!
64
![Page 65: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/65.jpg)
© 2010 Marc Levoy
RAW file processor
65
Lens aberration correction panel in
Canon Digital Photo Professional
![Page 66: Post-processing pipeline - Computer Graphics Laboratory - Stanford](https://reader030.vdocuments.mx/reader030/viewer/2022032816/620611708c2f7b17300454b4/html5/thumbnails/66.jpg)
© 2010 Marc Levoy
Slide credits✦ Fredo Durand
✦ Stone, M., A Field Guide to Digital Color, A.K. Peters, 2003.
✦ Wandell, B., Foundations of Vision, Sinauer, 1995.
✦ Rudman, T., Photographer’s Master Printing Course, Focal Press, 1998.
✦ Adams, A., The Print, Little, Brown and Co., 1980.
66