rethinking color for next- generation tv craig todd, dolby [email protected]@dolby.com masayuki...
TRANSCRIPT
Rethinking Color for Next-Generation TV
Craig Todd, Dolby [email protected] Sugawara, NHK, [email protected]
Seo Young Choi, Samsung, [email protected]
Parameters in draft UHDTV signal format
Parameter Value
Aspect Ratio 16x9
Pixel Raster 3840x2160 (4k), 7680x4320 (8k)
Pixel aspect ratio 1:1 (square pixels)
Scan Progressive
Frame Rate 120, 60, 60/1.001, 50, 30, 30/1.001, 25, 24, 24/1.001
Precision 10 bits 12 bits
Code for Reference Black 64 256
Code for Reference White 940 3760
Value for Reference Black (cd/m2) unspecified
Value for Reference White (cd/m2) unspecified
Non-linear coding Linear+Gamma 2.2 (same as Rec.709)
Annex 4 to Document ITU-R WP6C/564-E, 17 October 2011
PerceptionLimit
UHDTV Color Primaries
Chromaticity coordinates (CIE ,1931) x y Wavelength
Red primary (R) 0.708 0.292 630 nm
Green primary (G) 0.170 0.797 532 nm
Blue primary (B) 0.131 0.046 467 nm
Reference white (D65) 0.3127 0.3290
Picture information can be linearly indicated by the tristimulus values of RGB in the range of 0-1.The colorimetric values of the picture information can be determined based on the reference RGB primaries and the reference white.The color primaries are on the spectral locus.
Color gamut comparison in the CIE u′ v′ chromaticity diagram
Motivation for color difference encoding
• Convert R, G, B to Y , R′ ′-Y′, B′-Y′• Human perception is less sensitive to resolution in color
difference signals vs luma signals– High frequency cut-off at around 60 cpd in luma,
but at around 10-11 cpd in chrominance (R -Y , B -Y ) signals′ ′ ′ ′( cpd - cycle per 1 degree of visual angle)
• We can sub-sample R -Y′ ′, B -Y′ ′ to save bandwidth– In storage and interfacing– In compression
• Sub-sample R -Y′ ′, B -Y′ ′ by a factor of two in each dimension
• Resulting number of samples is ½ that of original RGB
Why luminance and chrominance signals are used ?(A hypothesized diagrammatic representation to describe color perception mechanism)
(http://www.nature.com/nrn/journal/v4/n7/fig_tab/nrn1138_F1.html)
• Zone Theory (G. E. Muller, "Ueber die Farbenempfindungen," Bd. 1 and 2. Psychol. Physiol. Sinnesorg., 1930, Erganzungsbd. 17, 1-430, 1930)
LMS cones in retina absorb light reaching eye
One achromatic and two opponent color signals in nerve fibers
De-correlating cone absorptions Providing efficient neural representation of color Taking advantage of low spatial acuity of the opponent colors for image storage and transmission
Sending these three signals to the lateral geniculate nucleus in brain
Transferring to the visual cortex in brain for interpretation
Barten's luminance contrast sensitivity as a function of
spatial frequency for different angular display sizes
Mullen's chromatic contrast sensitivity as a function of
spatial frequency for red-green and yellow-blue channels
(P. G. J. Barten, "Evaluation of Subjective Image Quality with the Square-Root Integral Method", Journal of the Optical Society of America 7: 2024)
(K. T. Mullen, "The Contrast Sensitivity of Human Color Vision to Red-Green and Blue-Yellow Chromatic Gratings” Journal of Physiology (London) 359: 381)
The background for down-sampling chrominance signals
• High frequency cut-off at around 10-11 cpd for the chromatic contrast sensitivity function but at around 60 cpd for the luminance contrast sensitivity function (※ cpd - cycle per 1 visual angle)
Red-Green
Yellow-Blue
The conventional way to create lumaNon-constant luminance
R
G
B0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R’
G’
B’
0.2627R’+ 0.6780G’+ 0.0593B’
Y’
LinearNon-Linear
Y’ is not a pure luma signal; it contains some chroma information because it was made from non-linear RGB signals. This is called “non-constant luminance”, or NCL
The conventional way to create Y’, Cb’, Cr’ Non-constant luminance
R
G
B 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R’
G’
B’
0.2627R’+ 0.6780G’+ 0.0593B’
Y’
R’
B’
R’- Y’
B’- Y’
Y’
Cb’
Cr’
Linear Non-Linear
The Cb’ and Cr’ signals contain some luminance information. If they are sub-sampled, there will be some loss in luminance resolution
Constant Luminance
Y’ is now a pure luma signal; it contains no chroma information because it was made from linear RGB signals. This is called “constant luminance”, or “CL”. The color difference signals contain less luminance information and can be down sampled with much less effect on the luminance resolution. The Cb’, Cr’ signals can be created using slightly different schemes.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R
G
B
0.2627R+ 0.6780G+ 0.0593B
Y Y’
R’
B’
R’- Y’
B’- Y’
R
B
Y’Cb
’Cr
’
Linear
Benefits of Constant Luminance Encoding
• Luminance resolution is less affected by down-sampling the chroma signals– detail and edge information is more accurately
maintained
• Lower correlation between Cr’, Cb’ and Y’• Better compression efficiency
– Chroma information survives compression better
For high luminance colors, correlation is significantly high between Y’ and Cb’ in the NCL
How much correlated in Y’-Cb’, Y’-Cr’, and Cb’-Cr’?
For most luminance values, correlation is significantly high between Y’ and Cr’ in the NCL→ non-orthogonal!
For high luminance colors, correlation is high between Cb’ and Cr’ in the NCL
Using uniformly sampled 4096 colors from the UHDTV gamut
Y’-Cr’
Y’-Cb’
Cb’-Cr’
The impact of the maintenance of lightness on image sharpness
• Significant lightness discrepancies in the
right-side four dark pixels after 4:1:0 sub-
sampling
Blurred boundary (white box)
Smeared black stripe line (green box)
(a) (b) (c)
(a) (b) (c)
48 45 23 7
46 41 17 6
(a) (b) (c)
49 42 23 11
47 39 20 6
(a) (b) (c)
45 39 28 22
43 37 25 19
Original 4:2:0 4:1:0
•The lightness values are almost maintained
in the right-side four dark pixels after 4:1:0
sub-sampling
Original sharp boundary (white box)
Apparent black strip line (green box)
(a) (b) (c)
Original image 4:2:0 4:1:0
(a) (b) (c)
48 45 23 7
46 41 17 6
(a) (b) (c)
48 44 23 8
46 41 17 6
(a) (b) (c)
48 44 23 7
46 41 18 6
Original 4:2:0 4:1:0
Constant Luminance Signals
Non-Constant Luminance Signals
Relative compression efficiency using HEVC (V. 4)
Test sequence having high colorfulnessTest sequence having high colorfulness
PSNR-lightness (L*) PSNR-color difference (ΔE*)
Downside of Constant Luminance
• It is a non-linear color space, i.e. color mixing is not along straight lines
+ =
Cyan + Red = Grey ?
RGB, R’G’B’, NCL Y’Cr’Cb’
CL Y’Cr’Cb’
= not quite Grey ?
Cross-fade between two pictures (addition of pictures in different domains)
R, G, B
R’, G’, B’ orNCL Y’, Cb’, Cr’
Cross-fade between two pictures
R’G’B’ orNCL Y’, Cb’, Cr’
CL Y’, Cb’, Cr’
[Mixing two images in RGB domain]
[Mixing two images in nonlinear R’G’B’ domain]
[Mixing two images in NCL-Y’Cb’Cr’ domain]
[Mixing two images in CL-Y’Cb’Cr’ domain]
Advantages of non-constant luminance
• Same results for color mixing between R’G’B’ and Y’Cr’Cb’
• We have been doing NCL for 60 years• Tried and true• No-risk involved, no surprises lurking• Why change something that works?
Conclusion• Open issues re UHDTV signal specification:
Terminology - do we call both 4k and 8k “Ultra HDTV”? Or should we use different words for each?
EOTF – don’t make the same mistake as Rec.709Color encoding – any reason to change? What is a suitable
threshold of benefit that a new method should reach to justify a change?
• If the UHDTV signal specification matters to you, get involved in the discussions and help drive this to the right conclusion.