Color Correction for Multiple Light Sources Using Profile Correction

SystemJournal of Imaging Science and Technology

vol. 53, No. 5, 2009Vivek Maik, Joonki Paik, Dohee Cho and

Dongwhan Har

School of Electrical Engineering and Computer Science Kyungpook National Univ.

Presented by Dae Chul Kim

Flow chart

2 /32

Images capture under various illuminant include Qpcard

Obtainment of color correction parameters by Qpcard from three

different images

Modifying ICC profile to generation of UD-PCS

Color mixture map extraction forFusion of input source image

Fusion of color corrected images

L-channel reversal

Output image

Proposed─ Color correction system for images illuminated by multiple light

sources• Input image capture

– Two light sources and reference Qpcard

• Estimation of color correction matrix and tone reproductive curves• Modifying International Color Consortium profile tags to generate

UD-PCS• Color mixture map extraction for fusion of input source image

3 /32

Abstract

Color in digital world─ Combined presence of a light source, object and observer─ Help us to define, understand, and identify object

Color management system(CMS)─ Standardization of colors in devices

• Easy handling, sharing, and broadcasting

─ Role of CMS • It attaches a specific meaning of a color to image to make them

unambiguous• It allows each imaging device to produce the same color

4 /32

Introduction

Color management system in modern digital world

5 /32

Fig1. Color management system in the modern digital world

ICC-based color management─ Profile connection space(PCS)─ Profiles relate PCS color value with device color format─ Color management module(CMM)

• Calculations using PCS values and profile relationship with output device-dependent color values

6 /32

Proposed color correction algorithm─ Capture an input image─ Estimation of light source using Qpcard─ Obtainment of tone reproduction curve(TRC) and color

correction matrix─ Estimation of correction parameters

• Incorporated in to ICC profile to generate UD-PCS

─ Color correction with UD-PCS• Generation of color mixture map

– Produced single color corrected output

7 /32

Block diagram of proposed color correction method

8 /32Fig.2. Block diagram of the proposed algorithm.

Related works─ Color constancy problem

• Difference between human vision and computer vision area

─ Color constancy algorithms• Most color constancy preserving algorithm

– Restrictive assumptions including constant illumination

• Retinex algorithm– Partially addresses issue of varying illumination– Try to eliminate variation in illumination and computes surface

brightness for each of three channels independently– Error in classifying intensity change

» Serious mismatch in recovered results

• Other color constancy method– Bayesian approach– Neural network

• Gamut mapping method9 /32

─ Contributions of proposed method• Estimation of light mixture per pixel• Independently control light colors and intensities• Identification of light sources using color profiles• Not increase computational load• Not requirement technique information on image processing

10 /32

Image capture and estimation of color correction parameters─ Simplicity based tungsten(T) and fluorescent(F)

11 /32

Table I. Technical terminologies for different images and profiles.

─ Image capture• Generation of T,F, and P images using T and F light sources

– Estimation of difference in colors caused by each of light sources

12 /32

Fig.3. Block diagram of the proposed color constancy algorithm, which illustrates step-by-step functional flow from left to right.

• Captured images

13 /32

Fig. 4. UD-PCS process: a–c represents the T, F, and P imageswith profile targets placed in the direction of the light source.

(a)

(c)

(b)

─ Color correction parameters• Obtainment of tone reproductive curve and correction matrix

– Comparison between Qpcard from three different images(T,F and P) and reference Qpcard

• Block diagram of proposed color correction algorithm

14 /32Fig. 5. Schematic illustration of the TRC and color correction process for T, F, and P light sources

• Tone reproductive curves(TRC)– Use of six patches for each of Qpcard in T,F and P images

15 /32

Fig. 6. Qpcard patches. Patches 2, 12 and 22 represent the reference R, G and B colors, and patches 5–10 represent the reference gray levels for TRCs.

─ Function to fit R,G,B color value between Qpcard in T,F,P images and reference Qpcard

16 /35

1 1 2 2 3 3 4 4( ) ( ) ( ) ( ) ( ), 0 1g x a f x a f x a f x a f x x= + + + ≤ ≤

1 2 3 4( ) 1, ( ) , ( ) sin (), ( ) ex p ()f x f x x f x x f x x= = = =

(1)

(2)

where is an undetermined coefficient andis the normalized color of the patch

nax

– Obtainment of color correction matrix

17 /32

(3)

(4)

XYZ Crr XYZM M M ′=

1 2

1 2

1 2

N

XYZ N

N

X X XM Y Y Y

Z Z Z

=

11 12 13

21 22 23

31 32 33

Crr

m m mM m m m

m m m

=

(5)

1 2

1 2

1 2

N

XYZ N

N

X X XM Y Y Y

Z Z Z

′ ′ ′ ′ ′ ′ ′=

′ ′ ′

(6)

where denote the colors of reference chart andare those of the color patches in chart image

, and i i iX Y Z, and i i iX Y Z′ ′ ′

– Simple rearrangement using singular value decomposition(SVD)

18 /32

1 1 1 121

2 2 2 222

23n N N N

Y X Y Zm

Y X Y Zmm

Y X Y Z

′ ′ ′ ′ ′ ′ = ′ ′ ′

1 1 1 111

2 2 2 212

13n N N N

X X Y Zm

X X Y Zmm

X X Y Z

′ ′ ′ ′ ′ ′ = ′ ′ ′

1 1 1 131

2 2 2 232

33n N N N

Z X Y Zm

Z X Y Zmm

Z X Y Z

′ ′ ′ ′ ′ ′ = ′ ′ ′

(9)

(8)

(7)

UD-PCS generation and color mixture model─ UD-PCS generation

• Use of standard for P light source images• Estimated color correction matrix values can be embedded in

ICC profile

19 /32

11 12 13

21 22 23

31 32 33

Crr

m m mM m m m

m m m

=

(10)

x x x

y y y

z z z

redMatrixColumn greenMatrixColumn blueMatrixColumnredMatrixColumn greenMatrixColumn blueMatrixColumnredMatrixColumn greenMatrixColumn blueMatrixColumn

=

(11)

─ Consideration of chromatic adaptation transformation• Illuminant of color device is not equal to PCS• ICC 1:2004-10 recommends linear Bradford model

20 /32

11 12 13

21 22 23

31 32 33

0.436052 0.385082 0.1430870.222492 0.716886 0.0606210.013929 0.097097 0.714185

CAF

t t tM t t t

t t t

= =

(12)

CAFCrr Crr CAFM M M= (13)

where represents the color correction matrix after chromatic adaptive transformationCAFCrrM

─ Tone reproductive control tags• TRC’s for each of the three color channels are preserved in

each of rTRCTag, gTRCTag and bTRCTag• One dimensional look-up table(LUT)

– To map input color pixels to output color pixels

21 /32

[ ] ( )R iredTRC i g x=

[ ] ( )G igreenTRC i g x=

[ ] ( )B iblueTRC i g x=

(14)

(15)

(16)

where i represents the number of color pixels

─ ICC profile tags

─ Schematic representation of above algorithm

22 /32Fig. 7. Generation of color corrected output image using UD-PCS.

Table II. ICC profile tags.

Color mixture model─ Obtainment of final color enhanced image─ Color difference between Qpcard colors in T source image

and reference chart

23 /32

Fig. 8. Color difference between Qpcard colors in T source image and reference chart. The color difference of white point location is used in computing the color mixture map which is used to fuse T, F, and P source images.

─ Color mixture model to fuse three color corrected images

24 /32

Fig. 9. Pixel assignment process. The pixels from each pixel-corrected image are selected in appropriate proportions and merged to form the final image.

─ Obtainment of color mixture map• Use of Color difference of white point location

25 /32

R TT P PdE W W= −

R FF P PdE W W= −

R PP P PdE W W= −

where represent the color difference for T,F and ,P image( , , ) and ( , , )T F PT F P P P PdE dE dE W W W E∆

(17)

(18)

(19)

• Color mixture proportion

• In ideal case

26 /32

/ ( ) / ( ) / ( )

corr corr corrCorrout

T T F P F T F P P T F P

T F PIdE dE dE dE dE dE dE dE dE dE dE dE

= + ++ + + + + + (20)

where represent the color corrected images for T,F and P light source andrepresent the final fused image Corr

outI( , , )corr corr corrT F P

0.33( )

TorForP

T F P

dEdE dE dE

=+ +

(21)

L-channel reversal─ Color mixture process among various channel

• Significant errors including block artifact and color mismatching

─ Reduction of color mixture artifact• Replacement original P image instead of L channel

27 /32

Fig. 10. L-channel reversal process. (a) P image, (b) Fused final image, (c) parts of the image with color mixture artifacts, (d) LAB channelsof the images a and b, and e the L-channel replaced image.

Experiment─ Test on studio and real scene images

28 /35

Experimental results

Figure 11. Experimental results: (a) T source image, (b) F source image, (c) P source image, (d)–(f) color mixture map of T, F, and P source images, and g–h final fused image before and after L-channel replacement.

─ Color scatter plot

29 /32

Fig 12. Color scatter plot. (a) P image, (b) P image after applying the proposed method. The colors are uniformly distributed over the colorspace.

─ Color scatter plot• Comparison in region of color gradients or transition

30 /32

Fig. 13. Color scatter plot at specific regions of the image. The “above” images in (a)-(c) represent the scatter plot of the original P image and “below” images show the difference after the color constancy algorithm have been applied.

─ Test images

31 /32Fig. 14. Experimental results on (a) indoor, (b) outdoor, and (c) synthetic images.

Performance comparison─ Testing difference set of light source

32 /32

Table III. Root-mean-square error comparison of the proposed algorithm with standard profiling software.

Proposed algorithm─ Color constancy approach using color profile correction method

• Not required an excessive computational load• Easier to apply to image• Effectively adapted to UD-PCS to build an ICC color profile

33 / 32

Conclusions