1 color. 2 what is color try describing the color “red” to a blind person color is merely a...
TRANSCRIPT
1
Color
2
What is Color
• Try describing the color “red” to a blind person• Color is merely a concept, something we “see”
within our minds– It’s interpretation involves both physics and biology
• Clearly, it plays a critical role in everyday life• Thus, building a mathematical description of color
is necessary
3
Color is Complex
• “Standard” mathematical models began in the early 20th century and have evolved
• Confusion arises in that the early standards were not discarded as the evolution took place
• Today, “old” and “new” standards live side by side
• Thus, when discussing color the first thing the participants must agree upon is the standard in which they are basing their discussion
4
The Standards
• Based on a tristimulus system of additive primaries
• Tristimulus – three primary colors
• Additive – all other colors can be created by adding different proportions of the primaries
5
Preliminaries
6
Tristimulus, Additive Primaries
• Red, Green, and Blue primaries were agreed upon based on a normal human visual system
• A normal visual system consists of the eyes and sections of the brain, all operating properly
7
The Eye
8
The Retina
• The retina contains two types of light sensors– Rods that are highly sensitive to light and provide us
with “night vision”• Located primarily in the outer (non-foveal) region of the retina
– Cones that are highly sensitive to color and provide us with “color vision”
• Located primarily in the central (foveal) region of the retina
9
The Retina
10
Cones
• There are 3 types of cones contained within the retina– Red-sensitive (long)– Green-sensitive (medium)– Blue-sensitive (short)
11
Cone Sensitivity(probable)
12
The Visual System
• Once the eye has sensed the color it is up to the brain to interpret it
• This is where things get very complex and relatively little is known about the actual inner-workings
• One [interesting] thing we do know– Red and green receptor genes are carried on the X chromosome,
and these are the ones that typically go wrong– Men have one X and one Y so the probability of color blindness
(or deficiency) is good– Women need two bad X chromosomes to have a deficiency, which
is less likely
13
So What?
• With what we know (or think we know) about the visual system, we now try to develop useful models to support the more mundane tasks of everyday life
14
The Standards
and other preliminaries
15
Standard Observers
• To set a standard a group of people were shown color patches of a given size and asked “what colors they saw”
• Results were averaged and thus the standards were created
16
Standard Observers
• 1931 (2°) and 1964 (10°) standard observers
17
Color Temperature
• Black body illuminatorApply heat
Look through here
Temperature is the only variable determining the color you see
)1/(exp(
11)(
5
khcE
18
Illumination
• The color of the ambient light affects the colors we perceive– Incandescent – redish tinge due to the melting
temperature (black body) of the element– Fluorescent – bluish tinge due to high-speed electrons
striking gas causes the release of ultraviolet radiation – Arc lamps – various colors created by arcing in gaseous
metals (sodium and mercury)
• Most cameras correct for the effects of ambient lighting through a process called white balance
19
Gamut
• Basically, this is the set of colors that can be captured or displayed on a given device– Not all colors are possible on all devices
20
The Color Spaces
21
XYZ Color Space (everything starts from here)
• Combine– Known illuminant
– Colors on known (non-reflective) material
– Standard observer
– The result is a tristimulus space for describing colors
22
xyY Color Space(the first offspring)
• There’s no good way to visualize the XYZ color space
• The xyY space is a normalized version of XYZ– x and y correspond to normalized X and Y respectively
– The luminance (black/white level) is lost in the normalization process so Y (in xyY) is also computed from XYZ
– z is not needed since the normalization process constrains x + y + z = 1
23
xyY Color Space(well, one of them anyway)
Saturated Colors
MonitorGamut
24
xyY Color Space
• Pro– We can visualize the proximity of one color to
another
• Con– The space is non-uniform so we cannot use it to
compare colors
25
Other Useful Color Spaces
• What do we know?– XYZ and xyY are not very intuitive– All color spaces are tristimulus– All are useful (convenient) in certain situations– All are useless (inconvenient) in certain
situations
• So, we invent new color spaces to suit our needs
26
RGB Color Space
• RGB is a linear color space– We can think of a given
color as a 3-vector– Pure red, green, and
blue are the basis vectors for the color space
– Useful for cameras, monitors, and related manipulations
Gray (black to white) axis
Black
White
27
RGB Color Space
Back Surfaces Front Surfaces
28
RGB Color Bars
29
RGB Operations
• Color mixing is performed by vector addition and subtraction operations– Adding/subtracting colors is the same as
adding/subtracting vectors
red green+ yellow=
30
RGB Operations
• Increasing or decreasing luminance is performed by scalar multiplication– Same as scalar multiplication of vectors
yellow 2* brighter yellow=
31
RGB Operations
• One difference…• Operations must be clamped…
– …at 0 to make sure components don’t go negative
– …at some pre-specified maximum to ensure display compatibility
• Scaling down from a value greater than the allowed maximum can be performed but care must be taken– Bright colors may end up less bright than other colors in the
scene
– The answer is to scale ALL colors in the scene which can be expensive
32
Lenna
RGB
Red
Green
Blue
33
RGB Color Space
• Pro– Very intuitive and easy to manipulate when generating
colors
• Con– Very unintuitive when it comes to comparing colors
– Consider the Euclidian distance between red and green and between green and blue
• So we invent another one…
34
Luminance-Chrominance Color Spaces (there are many)
• Luminance channel– Corresponds to the black and white signal of a color
television
• Two chrominance channels– Red and blue
– Correspond to the color signal that “rides” on top of the black and white signal of a color television
• Various forms– YUV, YIQ, YCbCr, YPbPr…
35
Luminance-Chrominance Color Spaces (there are many)
• Luminance is a square wave• Chrominance is a sine wave (modulation) on top
of the square wave
36
Luminance-Chrominance Color Spaces (there are many)
• Simple conversion from RGB and YPbPr
• And from YPbPr to RGB
PPY rb
B
G
R
081.418.500.
500.331.168.
114.587.299.
BGR
Y
PP
r
b
000.0772.1000.1
714.0344.0000.1
402.1000.0000.1
37
Luminance-Chrominance Color Spaces (there are many)
• Note that there are various different matrices for these conversions– Based on different needs– Be careful about the one you select
• Chrominance channels are +/- so to display you must translate and scale
38
Luminance-Chrominance Color Spaces
RGB
Luminance
ChrominanceBlue
ChrominanceRed
39
Luminance-Chrominance Color Spaces
• Pro– Separate high frequency components from low
frequency components– Easy to compute
• Con– Not very intuitive– Require signed, floating point (or scaled)
representation
40
Compression(uses for luminance/chrominance)
• The goal of image compression is to reduce the amount of data while retaining visual quality– “visually lossless compression”– Throw away as much data as possible without
degrading the picture– JPEG, MPEG, …
• Again, it all relates back to the human visual system
41
JPEG/MPEG
• The edge/structure detail is contained in the luminance channel– This is referred to as “high frequency” data
• The color information is in the chrominance channels which are lacking edges/structure detail– This is referred to as “low frequency” data
42
Subsample Cb and Cr
43
Subsampling
• By subsampling we achieve a 2:1 compression without doing any “work”– This is the default mode for MPEG– The default mode for JPEG is to subsample in 1
dimension only so it’s 3:2 compression without doing any “work”
• The decompressed image still looks good because of the low frequency nature of the chrominance channels
JPEG2K
• Completely revamped
• No longer works with 8x8 blocks and discreet cosine transforms
• Gets its low and high frequency “channels” from wavelet transformations across the entire image
44
45
Cyan-Magenta-Yellow-blacK
• Used in printing• Colored pigments (inks) remove color from
incident light that is reflected off of the paper
• CMYK is a subtractive set of primaries– K (Black) is not actually necessary but is added
for practical printing applications
• CMYK is a linear color space
46
Cyan-Magenta-Yellow-blacK
• Subtractive nature of CMYK is consistent with RGB– Cyan = White – Red– Magenta = White – Green– Yellow = White – Blue
• RGB space manipulations(W – R) + (W – G) = 2W – R – G
= (W) – R – G
= (R + G + B) – R – G
= B– 2W → W since an inked page can’t reflect more light than an
uninked page
47
Cyan-Magenta-Yellow-blacK
• It’s not used in computer displays because the conversion from CMYK to RGB is not a simple subtraction from white
• Knowing what white “is” is the problem
• There are many, many different white standards
48
Cyan-Magenta-Yellow-blacK
Cyan Magenta
Yellow BlackRGB
49
Cyan-Magenta-Yellow-blacK
• Pro– Good for printing (as long as you include the K
ink)
• Con– Difficult to convert from RGB to CMYK as it
is not a simple subtraction from white like much of the world would lead you to believe
50
Hue/Saturation/Lightness
• Also Hue/Saturation/Value or Hue/Saturation/Intensity
• Suitable to processing images for “human consumption” (viewing)– Easy to make colors more “vibrant” (and other
features that we can name but can’t really describe)
51
Hue/Saturation/Lightness
• Hue is the pure color content– Corresponds to the edges of the RGB cube
• Saturation is the intensity of color – The faces of the RGB cube are fully-saturated
• Lightness is, as the name implies, the brightness of the color– Ranges from black to white
52
Hue/Saturation/Lightness
• Mapping the RGB cube to a hex-cone
53
Hue/Saturation/Lightness
RGB
Hue
Saturation
Lightness
54
Hue/Saturation/Lightness
• Pro– Used by artists and color designers– Captures the “human” qualities of color
• Con– Very difficult to describe– Not very intuitive
55
L*a*b* Color Space
• While convenient for various reasons, the previous color spaces are not great for comparing colors– Most attempts treat the colors as a 3-vector and try
to do some modified Euclidian distance measure and some sort of clustering algorithm
– The color spaces are non-uniform• La*b* is a uniform color space
– A small perturbation in a color component is equally perceptible across the entire range
56
L*a*b* Color Space
• Unfortunately…– Not very intuitive– Not easy to convert to/from RGB
• Requires knowledge of a reference white• Requires computation of cube-roots
• It’s strength lies in comparing colors– It converts the non-uniform XYZ space to a
uniform space– Colors can be compared [accurately] using the
Euclidian distance formula
57
L*a*b* Color Space
RGB
L*
a*
b*
58
L*a*b* Color Space
• Pro– Uniform space– Colors can be compared [accurately] using the
Euclidian distance formula
• Con– Not very intuitive– Not easy to convert from/to RGB
• Requires knowledge of a reference white
• Requires computation of cube-roots
59
Other Related Topics
And what good talk on color would dare to leave out these topics…
60
The Misused and AbusedGretagMacBethTM ColorChecker®
61
The JOBO Card
62
Gamma
• RGB values from a camera (for instance) are linear
• RGB values viewed on a monitor are non-linear
• Gamma correction is a non-linear pre-adjustment of the linear RGB values to match (or meet the expectations of) the non-linear human visual system when viewing a non-linear monitor
• Implemented as a look-up table
0.0 1.0
1.0
0.0
R’G’B’
RGB
63
Gamma Correction
Linear RGB from camera
Uncorrected Linear RGB on monitor
Corrected Linear RGB from camera
Corrected Linear RGB on monitor
64
Alpha
• In computer graphics, we often speak of 32-bit RGB
• The additional 8-bits is not another color basis, but rather a value called Alpha
• Alpha defines how colors combine with one another in an operation called Alpha Blending
65
Alpha Blending
• In 3D computer graphics objects naturally obscure other objects
• Depending on the make-up of the object in front– You may not see the object in back, the object
in front is opaque– You may only see the object in back, the object
in front is translucent– You may see some combination of both objects
66
Alpha Blending
• The specification of an objects opacity is done through alpha
• The basic formula is one of linear interpolation
• The alpha value of the object in back is ignored• In the event that we have multiple objects stacked,
then the z-buffer rendering performs this calculation in order, back to front
CACACAC backfrontfrontfrontbackfrontfrontAlphaBlend )1(),,(
67
Color Space Quantization
• There are times (used to be times?) when our hardware does not (did not?) support 224 (24-bit) colors
• The alternative is (was?) typically an 256 (8-bit) color palette system
• The question then arises as to which 256 colors we should choose
68
Color Space Quantization
• The popularity algorithm prescribes that we select the 256 most frequently used colors in the scene we are displaying
• Create a histogram of all 224 possible colors
• Keep only the top 256
69
Popularity AlgorithmC
OU
NT
COLOR INDEX
Color Frequency Histogram
• Create color frequency histogram• Sort histogram by count• Keep the 256 colors with the largest counts• Convert all other scene colors to the closest kept color
70
Popularity Algorithm
• This algorithm works fine for a small amount of original scene colors (relative to the target number of colors)
• When the number of different colors in the original scene is much greater than the target number, the algorithm breaks down– Especially where small scene objects are
concerned
71
Median-Cut Algorithm
• Rather than just histogram and keep the most popular colors, the median-cut algorithm attempts to find colors that represent equal numbers of colors in the original scene
• It does so by histograming the scene colors into the color cube (rather than a linear histogram)
• Then the cube is recursively split into smaller cubes, attempting to keep the number of pixels in each cube the same
• The procedure ends when n (the target number of colors) cubes are created
• The centroids of the cubes are the retained colors• All other pixel colors in each cube are set to the cube centroid
color
72
Dithering
• But what happens when the number of available colors is only 2? (monochrome display device)
• Popularity and median-cut algorithms won’t produce suitable results in this scenario
73
Thresholding
• If we merely select a threshold and set pixel values below it to black and above it to white we lose a lot of information
74
Gray Level Image
75
Thresholding
76
Thresholding
• The human visual system is so good that we can still see the picture (in our minds) even though the data (taken in by the eyes) is minimal
77
Dithering
• By replacing individual pixels with a pattern of binary values, the human visual system can be fooled into seeing shades
• The problem with pure thresholding is that all of the error ends up in the pixel being processed
• With dithering, we attempt do distribute the error to surrounding pixels
78
Floyd-Steinberg Dithering
For each pixeldisplay the closest available colorcompute error = actualColor – displayedColorspread error over (weighted addition) neighboring actual pixels to the right and below
7 * error16
1 * error16
5 * error16
3 * error16
Currentpixel
79
Floyd-Steinberg Dithering
80
Dithering
• Other dithering techniques involve replacing pixels with patterns meant to approximate the amount of “ink” (intensity) on the page
• The downside of these approaches are that the display size is typically larger than the actual image (the Floyd-Steinberg method does not suffer from this)
81
Thresholding
• Again, the human visual system is so good that we can still see the picture (in our minds) even though the data (taken in by the eyes) is minimal
82
Summary
• Color is complex
• The human visual system is complex and very good at processing light
• Together they comprise a system that we aren’t even close to understanding but utilize very effectively
83
[a few selected]References
• How the Retina Works – Helga Kolb American Scientist, Volume 91
• Calculation From the Original Experimental Data of the CIE 1931 RGB Standard Observer Spectral Chromaticity Coordinates and Color Matching Functions – D.A. Broadbent University de Sherbrooke
• Eye, Brain, and Vision – David H. Hubel Scientific American Library 1988
• RGB Coordinates of the Macbeth ColorChecker – Danny Pascale www.babelcolor.com
• The RGB Code: The Mysteries of Color Revealed – Danny Pascale www.babelcolor.com
84
Reading for next week
• Chapter 18 – Model-Based Vision