review: image formation - computer sciencelazebnik/spring11/lec04_color.pdf · review: image...
TRANSCRIPT
![Page 1: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/1.jpg)
Review: Image formation• What determines the brightness of an image
pixel?Light sourceLight sourceproperties
Surface shape and orientation
Sensor characteristics
E pos reExposure
Surface reflectanceSurface reflectancepropertiesOptics
Slide by L. Fei-Fei
![Page 2: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/2.jpg)
Color
Phillip Otto Runge (1777-1810)
![Page 3: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/3.jpg)
What is color?• Color is the result of
interaction between physical light in thephysical light in the environment and our visual systemC• Color is a psychological property of our visual experiences when we look pat objects and lights, not a physical property of those objects or lightsthose objects or lights (S. Palmer, Vision Science: Photons to Ph l )Phenomenology)
![Page 4: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/4.jpg)
Electromagnetic spectrum
Human Luminance Sensitivity Function
![Page 5: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/5.jpg)
The Physics of Light
Any source of light can be completely describedphysically by its spectrum: the amount of energy emittedphysically by its spectrum: the amount of energy emitted (per time unit) at each wavelength 400 - 700 nm.
# PhotonsRelativespectral(per ms.)spectral
power
400 500 600 700
© Stephen E. Palmer, 2002
Wavelength (nm.)
![Page 6: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/6.jpg)
The Physics of Light
Some examples of the spectra of light sourcesB. Gallium Phosphide Crystal
oton
s
A. Ruby Laser
oton
s
pow
er
pow
er
400 500 600 700
# P
ho
400 500 600 700
# P
ho
Rel
. p
Rel
. p
D. Normal Daylight
Wavelength (nm.)Wavelength (nm.)
C. Tungsten Lightbulb
Pho
tons
Pho
tons
el. p
ower
Rel
. pow
er#
P
400 500 600 700
# P
400 500 600 700
© Stephen E. Palmer, 2002
Re R
![Page 7: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/7.jpg)
The Physics of Light
Some examples of the reflectance spectra of surfaces
cted Red Yellow Blue Purple
t Ref
lec
% L
ight
Wavelength (nm)400 700 400 700 400 700 400 700
© Stephen E. Palmer, 2002
![Page 8: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/8.jpg)
Interaction of light and surfaces
R fl t d l i th lt• Reflected color is the result of interaction of light source spectrum with surface preflectance
![Page 9: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/9.jpg)
Interaction of light and surfaces• What is the observed color of any surface
under monochromatic light?
Olafur Eliasson, Room for one color
![Page 10: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/10.jpg)
The Eye
The human eye is a camera!I i l d l ith di l l• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris• Lens - changes shape by using ciliary muscles (to focus on objects
t diff t di t )at different distances)• Retina - photoreceptor cells
Slide by Steve Seitz
![Page 11: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/11.jpg)
Rods and cones
pigmentcone
rod
p gmolecules
Rods are responsible for intensity, cones for color perceptionRods and cones are non-uniformly distributed on the retinaRods and cones are non uniformly distributed on the retina
• Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones (and no rods)
Slide by Steve Seitz
![Page 12: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/12.jpg)
Rod / Cone sensitivity
Why can’t we read in the dark?Slide by A. Efros
![Page 13: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/13.jpg)
Physiology of Color Vision
Three kinds of cones:N
CE
(%) 100
440
S
530 560 nm.
M L
AB
SO
RB
AN
50
RE
LATI
VE
A
400 450 500 550 600 650
WAVELENGTH (nm.)
© Stephen E. Palmer, 2002
• Ratio of L to M to S cones: approx. 10:5:1• Almost no S cones in the center of the fovea
![Page 14: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/14.jpg)
Color perceptionM L
Power
S
Rods and cones act as filters on the spectrum
Wavelength
Rods and cones act as filters on the spectrum• To get the output of a filter, multiply its response curve by the
spectrum, integrate over all wavelengthsEach cone yields one number– Each cone yields one number
• Q: How can we represent an entire spectrum with 3 numbers?• A: We can’t! Most of the information is lostA: We can t! Most of the information is lost.
– As a result, two different spectra may appear indistinguishable» such spectra are known as metamers
Slide by Steve Seitz
![Page 15: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/15.jpg)
Metamers
![Page 16: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/16.jpg)
Spectra of some real-world surfaces
metamers
![Page 17: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/17.jpg)
Standardizing color experience• We would like to understand which spectra
produce the same color sensation in people d i il i i ditiunder similar viewing conditions
• Color matching experiments
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
![Page 18: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/18.jpg)
Color matching experiment 1
Source: W. Freeman
![Page 19: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/19.jpg)
Color matching experiment 1
p1 p2 p3Source: W. Freeman
![Page 20: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/20.jpg)
Color matching experiment 1
p1 p2 p3Source: W. Freeman
![Page 21: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/21.jpg)
Color matching experiment 1
The primary color amounts neededamounts needed for a match
p1 p2 p3Source: W. Freeman
![Page 22: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/22.jpg)
Color matching experiment 2
Source: W. Freeman
![Page 23: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/23.jpg)
Color matching experiment 2
p1 p2 p3Source: W. Freeman
![Page 24: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/24.jpg)
Color matching experiment 2
p1 p2 p3Source: W. Freeman
![Page 25: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/25.jpg)
Color matching experiment 2
We say a The primary color amounts neededy
“negative” amount of p2was needed to
amounts needed for a match:
was needed to make the match, because we p1 p2 p3added it to the test color’s side.
1 2 3
p1 p2 p3p1 p2 p3Source: W. Freeman
![Page 26: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/26.jpg)
Trichromacy• In color matching experiments, most people
can match any given light with three primariesP i i t b i d d t• Primaries must be independent
• For the same light and same primaries, most people select the same weightspeople select the same weights• Exception: color blindness
• Trichromatic color theoryc o at c co o t eo y• Three numbers seem to be sufficient for encoding color• Dates back to 18th century (Thomas Young)
![Page 27: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/27.jpg)
Grassman’s Laws• Color matching appears to be linear• If two test lights can be matched with the same
set of weights, then they match each other: • Suppose A = u1 P1 + u2 P2 + u3 P3 and B = u1 P1 + u2 P2 + u3 P3.
Then A = B.Then A B.
• If we mix two test lights, then mixing the matches will match the result:• Suppose A = u1 P1 + u2 P2 + u3 P3 and B = v1 P1 + v2 P2 + v3 P3.
Then A + B = (u1+v1) P1 + (u2+v2) P2 + (u3+v3) P3.
• If we scale the test light then the matches get• If we scale the test light, then the matches get scaled by the same amount:• Suppose A = u1 P1 + u2 P2 + u3 P3. pp 1 1 2 2 3 3
Then kA = (ku1) P1 + (ku2) P2 + (ku3) P3.
![Page 28: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/28.jpg)
Linear color spaces• Defined by a choice of three primaries • The coordinates of a color are given by the
weights of the primaries used to match it
mixing two lights producescolors that lie along a straight
li i l
mixing three lights produces colors that lie within the triangle
th d fi i lline in color space they define in color space
![Page 29: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/29.jpg)
How to compute the weights of the primaries to match any spectral signalto match any spectral signal
Given: a choice of Find: weights of the
?three primaries and a target color signal
primaries needed to match the color signal
?p1 p2 p3p1 p2 p3
• Matching functions: the amount of each primary needed to match a monochromatic light source at each wavelength
![Page 30: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/30.jpg)
RGB space• Primaries are monochromatic lights (for monitors,
they correspond to the three types of phosphors)S bt ti t hi i d f• Subtractive matching required for some wavelengths
RGB matching functionsRGB primaries
![Page 31: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/31.jpg)
How to compute the weights of the primaries to match any spectral signalto match any spectral signal
• Let c(λ) be one of the matching functions, and let t(λ) be h f h i l Th h i h f hthe spectrum of the signal. Then the weight of the
corresponding primary needed to match t is
∫∫=λ
λλλ dtcw )()(
Matching functions, c(λ)
Signal to be matched, t(λ)
λ
![Page 32: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/32.jpg)
Comparison of RGB matching functions with best 3x3 transformation of cone responses3x3 transformation of cone responses
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
![Page 33: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/33.jpg)
Linear color spaces: CIE XYZ• Primaries are imaginary, but matching
functions are everywhere positive• The Y parameter corresponds to brightness or• The Y parameter corresponds to brightness or
luminance of a color• 2D visualization: draw (x,y), where2D visualization: draw (x,y), where
x = X/(X+Y+Z), y = Y/(X+Y+Z) Matching functions
http://en.wikipedia.org/wiki/CIE_1931_color_space
![Page 34: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/34.jpg)
Uniform color spaces• Unfortunately, differences in x,y coordinates do not
reflect perceptual color differences• CIE u’v’ is a projective transform of x y to make the• CIE u v is a projective transform of x,y to make the
ellipses more uniform
McAdam ellipses: Just noticeable differences in color
![Page 35: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/35.jpg)
Nonlinear color spaces: HSV
• Perceptually meaningful dimensions: Hue, Saturation, Value (Intensity)
G• RGB cube on its vertex
![Page 36: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/36.jpg)
Color perception• Color/lightness constancy
• The ability of the human visual system to perceive the intrinsic reflectance properties of the surfaces despiteintrinsic reflectance properties of the surfaces despite changes in illumination conditions
• Instantaneous effects• Simultaneous contrast• Mach bands
• Gradual effects• Gradual effects• Light/dark adaptation• Chromatic adaptation• Afterimages
J. S. Sargent, The Daughters of Edward D. Boit, 1882
![Page 37: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/37.jpg)
Lightness constancy
http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html
![Page 38: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/38.jpg)
Lightness constancy
• Possible explanations• Simultaneous contrast• Reflectance edges vs. illumination edges
http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html
![Page 39: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/39.jpg)
Simultaneous contrast/Mach bands
Source: D. Forsyth
![Page 40: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/40.jpg)
Chromatic adaptation• The visual system changes its sensitivity depending
on the luminances prevailing in the visual field• The exact mechanism is poorly understoodThe exact mechanism is poorly understood
• Adapting to different brightness levels• Changing the size of the iris opening (i.e., the aperture)
h th t f li ht th t t thchanges the amount of light that can enter the eye • Think of walking into a building from full sunshine
• Adapting to different color temperaturep g p• The receptive cells on the retina change their sensitivity • For example: if there is an increased amount of red light, the
cells receptive to red decrease their sensitivity until thecells receptive to red decrease their sensitivity until the scene looks white again
• We actually adapt better in brighter scenes: This is why candlelit scenes still look yellowcandlelit scenes still look yellow
http://www.schorsch.com/kbase/glossary/adaptation.html
![Page 41: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/41.jpg)
White balance• When looking at a picture on screen or print, we adapt to
the illuminant of the room, not to that of the scene in the picturepicture
• When the white balance is not correct, the picture will have an unnatural color “cast”
incorrect white balance correct white balance
http://www.cambridgeincolour.com/tutorials/white-balance.htm
![Page 42: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/42.jpg)
White balance• Film cameras:
• Different types of film or different filters for different illumination conditionsillumination conditions
• Digital cameras: • Automatic white balance• White balance settings corresponding to
several common illuminants• Custom white balance using a reference
object
http://www.cambridgeincolour.com/tutorials/white-balance.htm
![Page 43: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/43.jpg)
White balanceV K i d i• Von Kries adaptation
• Multiply each channel by a gain factor
![Page 44: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/44.jpg)
White balanceV K i d i• Von Kries adaptation
• Multiply each channel by a gain factor
• Best way: gray card• Take a picture of a neutral object (white or gray)• Deduce the weight of each channel
– If the object is recoded as rw, gw, bwuse weights 1/rw, 1/gw, 1/bw
![Page 45: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/45.jpg)
White balance• Without gray cards: we need to “guess” which
pixels correspond to white objects• Gray world assumption• Gray world assumption
• The image average rave, gave, bave is gray• Use weights 1/rave, 1/gave, 1/bave
• Brightest pixel assumption• Highlights usually have the color of the light source • Use weights inversely proportional to the values of theUse weights inversely proportional to the values of the
brightest pixels
• Gamut mappingGam t con e h ll of all pi el colors in an image• Gamut: convex hull of all pixel colors in an image
• Find the transformation that matches the gamut of the image to the gamut of a “typical” image under white light
U i t ti ti l i t h i• Use image statistics, learning techniques
![Page 46: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/46.jpg)
White balance by recognition• Key idea: For each of the
semantic classes present in the image compute thein the image, compute the illuminant that transforms the pixels assigned to that l th t thclass so that the average
color of that class matches the averagematches the average color of the same class in a database of “typical” iimages
J. Van de Weijer, C. Schmid and J. Verbeek, Using High-Level Visual Information for Color Constancy, ICCV 2007.
![Page 47: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/47.jpg)
Mixed illumination• When there are several types of illuminants in the
scene, different reference points will yield different ltresults
http://www.cambridgeincolour.com/tutorials/white-balance.htm
Reference: moon Reference: stone
![Page 48: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/48.jpg)
Spatially varying white balance
Input Alpha map Output
E. Hsu, T. Mertens, S. Paris, S. Avidan, and F. Durand, “Light Mixture Estimation for Spatially Varying White Balance,” SIGGRAPH 2008
![Page 49: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/49.jpg)
Uses of color in computer visionColor histograms for image matching
Swain and Ballard, Color Indexing, IJCV 1991.
![Page 50: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/50.jpg)
Uses of color in computer visionColor histograms for image matching
http://labs.ideeinc.com/multicolr
![Page 51: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/51.jpg)
Uses of color in computer visionImage segmentation and retrieval
C C S B l i H G d Ji M lik Bl b ldC. Carson, S. Belongie, H. Greenspan, and Ji. Malik, Blobworld: Image segmentation using Expectation-Maximization and its application to image querying, ICVIS 1999.
![Page 52: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/52.jpg)
Uses of color in computer visionSkin detection
M. Jones and J. Rehg, Statistical Color Models with Application to Skin Detection, IJCV 2002.
![Page 53: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/53.jpg)
Uses of color in computer visionRobot soccer
M. Sridharan and P. Stone, Towards Eliminating Manual Color Calibration at RoboCup. RoboCup-2005: RobotColor Calibration at RoboCup. RoboCup 2005: Robot Soccer World Cup IX, Springer Verlag, 2006
Source: K. Grauman
![Page 54: Review: Image formation - Computer Sciencelazebnik/spring11/lec04_color.pdf · Review: Image formation • What determines the brightness of an image pixel? Light sourceLight source](https://reader035.vdocuments.mx/reader035/viewer/2022071106/5fe073e66d39a8467305bb0f/html5/thumbnails/54.jpg)
Uses of color in computer visionBuilding appearance models for tracking
D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI 2007.