COLOR HIGH DYNAMIC RANGE (HDR) IMAGINGIN LUMINANCE-CHROMINANCE SPACE

Ossi Pirinen, Alessandro Foi, Atanas Gotchev

Institute of Signal ProcessingTampere University of Technology, Finland

firstname.lastname@tut.fi

ABSTRACT

In this paper, we address the issue of color in high dynamicrange (HDR) imaging. In contrast to state-of-the-art meth-ods, we propose to move the complete HDR imaging pro-cess from RGB to a luminance-chrominance color space.Our aim is to get a more computationally efficient techniqueand to avoid also any possible color distortions originatingfrom three color channels processed separately. To achievethis, we build a camera response function for the luminancechannel only and weight and compose the HDR luminanceaccordingly, while for the chrominance channels we applyweighting in relation with the saturation level. We demon-strate that our technique yields natural and pleasant to per-ceive tone-mapped images and is also more robust to noise.

1. INTRODUCTION

The visual representation of natural scenes has reached apoint where spatial resolution is no longer an issue and great-er realism is achieved by either adding the third dimensionor utilizing a more and more realistic gamut of light andcolor. The HDR imaging approaches the latter problem bycapturing multiple images of the same scene with differ-ent exposures and then composing them into a single imagespanning the whole dynamic range of the scene [1], [2]. Theimage composition step requires a preliminary calibrationof the camera response function [2]. A related problem thenis to adequately tone map the so obtained HDR image backto a low-dynamic range (LDR) display. Tone mapping tech-niques range in complexity from a simple gamma-curve tosophisticated histogram equalization methods and compli-cated lightness perception models [3], [4], [5].

State-of-the-art techniques for HDR imaging have beendeveloped generically for the RGB color space. Luminance-chrominance color space representations have been neglected.We consider a generic luminance-chrominance color space

This work was supported by the Academy of Finland, project No.213462 (Finnish Centre of Excellence program 2006-2011).

linearly related to the RGB space. Such a space is com-posed of an achromatic (gray) luminance component andtwo chrominance components, which are orthogonal to theachromatic one. Examples of such spaces are YUV/YCbCrand opponent color spaces [6]. We denote the luminanceand the two chrominance components by Y , U , and V , re-spectively. Let z = [zR , zG , zB ] be an image in the RGBspace and ζζζζζ = [ζY , ζU , ζV ] is the same image in the lumi-nance-chrominance space (we use Roman letters to denoteimages in RGB and the corresponding Greek letters to de-note images in luminance-chrominance). Transformation ofthe image from RGB to luminance-chrominance is definedin matrix form as ζζζζζ = zA, where the matrix A is nor-malized in such a way that if z(·) ∈ [0, 1]3 then ζζζζζ(·) ∈[0, 1]× [−0.5, 0.5]2. The hue and saturation can be definedas H = arctan ζU

ζVand S =

√(ζU )2 + (ζV )2, respectively.

HDR imaging techniques working in luminance-chro-minance seem more meaningful and preferable for a num-ber of reasons. First, decorrelated color space offers bet-ter compressibility. Considering image compression tech-niques, they all store images in some luminance-chromi-nance space. When one starts with already-compressed mul-tiple-exposure LDR images, it is more efficient to composethe HDR image directly in the same color space. The result-ing HDR image is then better suited for compression and, ifto be displayed, it can be mapped to sRGB during the tonemapping stage. Second, any HDR technique operating inRGB space requires post-composition white balancing sincethe three color channels undergo parallel transformations.While the white balancing would yield perceptually con-vincing colors, they might not be the true ones. For the sakeof hue preservation and better compression, it is beneficialto opt for a luminance-chrominance space, even if the inputdata is in RGB, e.g. uncompressed TIFF. Third, the lumi-nance channel, being a weighted average of the R, G, and Bchannels, enjoys a better signal-to-noise ratio (SNR), whichis crucial if the HDR imaging process takes place in noisyconditions.

In this contribution, we address the problem of HDRimaging in a luminance-chrominance space, propose effi-

cient algorithms for HDR image composition and tone map-ping and emphasize the benefits of such an approach. Thepaper is organized as follows. HDR image composition,including camera response function definition and process-ing the luminance and chrominance channels, is presentedin Section 2. An adapted tone mapping method for lumi-nance-chrominance HDR images is presented in Section 3.Examples demonstrating the viability of our approach aregiven in Section 4 and finally some conclusions are drawnin Section 5.

2. HDR IMAGE COMPOSITION

Consider a set of images ζζζζζi = [ζYi , ζU

i , ζVi ] , i = 1, . . . , N

in the luminance-chrominance space, captured with differ-ent exposure times ∆ti and with LDR, assuming ζζζζζ(x) ∈[0, 1] × [−0.5, 0.5]2, where x = [x1, x2] is a pixel coor-dinate. The goal is to obtain a single HDR image ζ̃ζζζζ =[ζ̃Y , ζ̃U , ζ̃V

]in the same color space. In our setting, the lu-

minance and chrominance channels are treated separately.While for the luminance channel, a pre-calibrated cameraresponse function is used, for the chrominance channels asaturation-driven weighting is applied.

2.1. Luminance component composition

The camera response function for the luminance channel isestimated from a set of images of a scene captured withdifferent exposure times. A sufficient number of suitablepixels, i.e. pixels having monotonically increasing valuesbetween under- and over-exposure are then chosen. Usingthese pixels, the camera response function is fitted employ-ing an SVD solver, in a fashion similar to that of [2], [7].In our experiments, we have found that 100 pixels is a suffi-cient amount for most cases. The camera response functionis estimated (calibrated) only once and then is used for thelinearization of the input values in all HDR compositions ofthe same device.

Similarly to the technique in [2], the HDR luminancecomponent is obtained by a pixelwise weighted average ofinput luminances. As a weighting function we use a Gaus-sian function wY with a mean of 0.5 and a standard devi-ation of 0.2 thus ensuring a smaller impact of the under-or over-exposed pixels. The logarithmic HDR luminance isobtained as

ln ζ̃Y (x) =∑N

i=1 wY (ζYi (x))(g(ζY

i (x))− ln∆ti)∑Ni=1 wY (ζY

i (x)).

In the above equation g is the camera response function.Because of its nature, the HDR luminance is obtained inlogarithmic scale. After employing the natural exponential,the resulting values are positive, normally spanning [10−4

104] thus being truly high dynamic range.

2.2. Chrominance components composition

For the chrominance components we define no camera re-sponse. Instead, we weight the chrominances in relation tothe level of color saturation. The higher the color saturation,the more the pixel contains valuable chromatic information,and thus the higher the weight. This is motivated by the factthat when a pixel is over- or under-exposed it is always lesssaturated than it would be at the correct exposure. Morespecifically, wUV (S) = Sα, where α > 1. In our exper-iments, we have found that α = 1.5 is a good choice. Toguarantee the color preservation, we use the same weightsfor both chromatic components and compose any chromaticcomponent C ∈ {U, V } as

ζ̃C (x) =∑N

i=1 wUV (Si(x))ζCi (x)∑N

i=1 wUV (Si(x)),

where Si denotes the saturation of ζζζζζi. We remark that be-ing a convex combination of the input chrominances, therange of ζ̃C(x) is again in [−0.5, 0.5]. However, becauseof averaging, the possible number of distinct pixel values isremarkably higher than in the original source sequence.

3. TONE MAPPING FORLUMINANCE-CHROMINANCE HDR IMAGES

Tone mapping is an HDR imaging technique used to ap-proximate the visibility of the tones of an HDR image on anLDR media, such as LCD and CRT displays or print-outs[8]. Essentially, tone mapping compresses the contrast ofa scene to fit into the displayable range of a media whilepreserving details and color.

A number of tone mapping methods working in RGBspace exist. These techniques can easily be adapted for theluminance range reduction, however the term ’tone map-ping’ would be questionable in this context, as tone is usu-ally used in connection with color. We denote such a lumi-nance range reduction operator as T and define its output,the reduced-range luminance image, as ζ̊ζζζζ(x) ∈ [0, 1]. As forthe chromatic channels, we suggest a simple, yet effectiveapproach.

The sRGB gamut does not allow for the rendering ofvery dark or very bright vivid and saturated colors whichexist in real scenes and which are captured in HDR images.Therefore there is need for a chromatic tone mapping. Inour approach, in order to get faithful colors that fit into thesRGB gamut we keep the hue intact while sacrificing sat-uration. Introducing a scaling factor δ for the two chromi-nances will not then change the hue, but will scale downthe saturation. The scheme we use to guarantee legal sRGBvalues is embedded in the color space transformation itselfand described as follows.

Let B = A−1 be the luminance-chrominance to RGBtransformation matrix and define the gray (achromatic) im-age and its chromatic complement image in RGB space by

z̊gray(x) =[z̊R

gray(x) z̊G

gray(x) z̊B

gray(x)

]=

T(ζ̃Y

)(x)

00

T

B,

zchrom(x) =[zR

chrom(x) zG

chrom(x) zB

chrom(x)

]=

0ζ̃U(x)ζ̃V(x)

T

B.

We remark that z̊gray(x) is truly a gray image because in RGBto luminance-chrominance transforms b1,1 = b1,2 = b1,3.We look for a map δ ≥ 0 such that

z̊ (x) = z̊gray(x) + δ (x) zchrom(x) ∈ [0, 1]3 . (1)

We can define it by δ (x) = min {1, δR (x) , δG (x) , δB (x)}where

δR (x) =

z̊Rgray(x)

−zRchrom(x) if zR

chrom(x) < 0

1−z̊Rgray(x)

zRchrom(x) if zR

chrom(x) > 0

1 if zR

chrom(x) = 0

and δG and δB are defined analogously. Thus, δ is thelargest one (not larger than 1) which allows the condition(1) to hold. It is easy to realize that the hue of z̊ (x) is notinfluenced by δ, whereas the saturation is scaled proportion-ally to it. Roughly speaking, the low dynamic range imagez̊ (x) has colors which have same hue as those in the HDRimage ζ̃ζζζζ and which are desaturated a little as it is needed tofit within the sRGB gamut.

It is now obvious that the tone mapped LDR image canbe defined in luminance-chrominance space as

ζ̊ζζζζ(x) =[T

(ζ̃Y

)(x) δ (x) ζ̃U(x) δ (x) ζ̃V(x)

].

The tone mapped luminance-chrominance image ζ̊ζζζζ can becompressed and stored directly with an arbitrary method(for example, DCT-based compression, as in JPEG), andfor display transformed into RGB using the matrix B. Wedemonstrate that this approach yields lively, realistic colors.

4. RESULTS

We compare the proposed HDR imaging approach in lu-minance-chrominance space against established techniquesworking in the RGB space. In particular, Debevec’s HDRShop v.1.0.3 software [9] has been used for the HDR com-position in RGB space. Figure 1 shows the source LDRsequence {zi}i=1,2,3 used in the experiments. There are a

number of tone mapping techniques, most of them requiringcareful tuning of a set of input parameters. We have optedfor the photographic tone reproduction operator [8] as itis a global tone mapping technique with fully automatic pa-rameter estimation. The tone mapped result of processing inRGB space is shown in Figure 2. In our approach, we firsttransform the sequence {zi}i=1,2,3 to the opponent colorspace. The operator T we use for reducing the range of ζ̃Y

is an adaptation of the method developed in [3]. Our resultis shown in Figure 3. Beside some obvious differences inbrightness, chromatic distortions which can arise from theRGB processing can be seen when the tone-mapped imagesare compared with the source sequence in Figure 1. Over-all, the colors obtained by processing in luminance-chromi-nance space are much more faithful, as the hue has beenpreserved.

A further notable advantage of our approach appearswhen the source sequence is degraded by noise. In our sim-ulations we added zero-mean Gaussian noise with standard-deviations σ = 5

255 , 15255 , and 25

255 to the original source se-quence from Figure 1. The HDR images obtained from theoriginal sequence were used as reference in order to mea-sure the errors in the HDR images composed from the noisysequences. In order to obtain meaningful and comparableresults, we calculate the SNR only on the luminance com-ponent of the HDR images. This is based on the fact thatthe image content is differently distributed among the threechannels of different color-space representations and the lu-minance is the most informative one. For the RGB HDR im-ages this component is simply the average of the R, G, andB channels. The results are summarized in Table 1. As canbe seen from the table, the inherent noise-attenuation prop-erties of the luminance-chrominance representations alonegive strong support to our approach.

5. CONCLUSIONS

In this contribution we have presented an efficient methodfor composing HDR images in luminance-chrominance colorspaces along with a method for tone mapping images ac-quired in this fashion. Our approach yields more realisticcolors with better noise suppression qualities compared tothe traditional RGB methods, and does so in a computation-ally efficient manner. It is our belief, that the advantages of

σ SNRRGB

SNRlum−chrom

5/255 25.8 26.315/255 13.8 18.225/255 10.9 14.6

Table 1. SNR (dB) comparison between processing in aluminance-chrominance space vs. RGB.

Fig. 1. Three-image source sequence used to compose the HDR images.

Fig. 2. HDR image composed and tone mapped by process-ing in RGB color space.

Fig. 3. HDR image composed and tone mapped by our ap-proach in luminance-chrominance space.

the proposed approach shall motivate the migration of theprocess from RGB to luminance-chrominance space.

6. REFERENCES

[1] S. Mann and R. Picard, “Being ’undigital’ with digital cam-eras: Extending dynamic range by combining differently ex-posed pictures,” in Proc. IS&T 46th Conf., 1995, pp. 422–428.

[2] P. Debevec and J. Malik, “Recovering high dynamic rangeradiance maps from photographs,” in Proc. 24th Conf. Comp.Graphics and Interactive Techniques, 1997, pp. 369–378.

[3] G.W. Larson, H. Rushmeier, and C. Piatko, “A visibilitymatching tone reproduction operator for high dynamic rangescenes,” IEEE Trans. Visual and Computer Graphics, vol. 3,no. 4, pp. 291–306, 1997.

[4] K. Devlin, A. Chalmers, A. Wilkie, and W. Purgathofer, “Tonereproduction and physically based spectral rendering,” inProc. EUROGRAPHICS’02, 2002, vol. 22.

[5] G. Krawczyk, K. Myszkowski, and H. Seidel, “Lightness per-ception in tone reproduction for high dynamic range images,”in Proc. EUROGRAPHICS’05, 2005, vol. 24.

[6] K. Plataniotis and A. Venetsanopoulos, Color image process-ing and applications, Springer-Verlag New York, Inc., NewYork, NY, 2000.

[7] S. O’Malley, “A simple, effective system for automated cap-ture of high dynamic range images,” in Proc. IEEE Int. Conf.Computer Vision Systems, ICVS’06, 2006.

[8] E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, HighDynamic Range Imaging, Morgan Kauffman Puslishers, SanFrancisco, CA, 2006.

[9] “HDR shop: High dynamic range image processing and ma-nipulation,” available at http://www.hdrshop.com.