Purdue UniversityPage 1

Color Color Image Fidelity Assessor Image Fidelity Assessor *

Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University)Jan P. Allebach (Purdue University)

* Research supported by HP Company while Wencheng Wu was at Purdue

Purdue UniversityPage 2

OutlineOutline

• Introduction

• Spatial color descriptor: chromatic difference

• Structure of Color Image Fidelity Assessor (CIFA)

• Psychophysical experiment and its results

• Test examples

• Conclusion

Purdue UniversityPage 3

IntroductionIntroduction(Motivation)(Motivation)

• Image fidelity assessment is important in the development of

imaging systems and image processing algorithms

Create visually lossless reproduction

Allocate efforts on most visible area

• Subjective evaluation is expensive and slow.

Purdue UniversityPage 4

IntroductionIntroduction(Prior work)(Prior work)

• Simple but not working

Root-Mean-Square Error

• Consider structure of HVS and perceptual process

Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA)

Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab)

Purdue UniversityPage 5

IntroductionIntroduction(CVDM vs. CIFA)(CVDM vs. CIFA)

• Both operate along opponent-color coordinates

• Both incorporate results from electrophysiological and

psychophysical exp.

• They differ in a similar way as VDP vs. IFA

CIFA has closer link between the structure of the model and the psychophysical data used by the model

• CIFA normalize the chromatic responses

This discounts luminance effect in chromatic channels

This reduces the dimension of psychometric LUT

Purdue UniversityPage 6

IntroductionIntroduction(Overview of CIFA)(Overview of CIFA)

• Color extension of Taylor’s achromatic IFA

• The model predicts perceived image fidelity

Assesses visible differences in the opponent channels

Explains the nature of visible difference (luminance change vs. color shift)

Color ImageFidelityAssessor(CIFA)

Ideal

Rendered

Viewing parameters

Image mapsof predicted

visibledifferences

Purdue UniversityPage 7

Chromatic differenceChromatic difference(Definition)(Definition)

• Objective: evaluate the spatial interaction between colors

• First transform CIE XYZ to opponent color space (O2,O3) *

* X. Zhang and B.A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97

tieschromaticiopponent )/,/(),( 3232 YOYOoo

.3,22

minmax

ioo

ci

• Then normalize to obtain opponent chromaticities (o2,o3)

• Define chromatic difference (analogous to luminance contrast c1)

Z

Y

X

O

O

Y

501.059.0086.0

077.029.0449.0

010

3

2

Luminance Red-Green

Blue-Yellow

Purdue UniversityPage 8

Opponent color representationOpponent color representation

(13.3,o2,0.17) (13.3,0.24,o3)(Y,0.24,0.17)

(Y,o2,o3)

Purdue UniversityPage 9

)16/cos(2.0 J)16/cos(1.0 J

Chromatic differenceChromatic difference(illustration)(illustration)

• Chromatic difference is a measure of chromaticity variation

• Chromatic difference is a spatial feature derived from opponent

chromaticity that has little dependence upon luminance

174438.0,235924.0,885.6),,( :)( pixelat gray Floyd"" 32 ooYI.J

)16/cos(05.0 J

0.1 0.10.20.05

• Chromatic difference is the amplitude of the sinusoidal grating

)16/cos(1.0 J

Purdue UniversityPage 10

CIFACIFA

Ideal Y Image

Rendered Y Image

Ideal O2 Image

Rendered O2 Image

Ideal O3 Image

Rendered O3 Image

Blue-yellowIFA

Red-greenIFA

Achromatic*IFA

Chromatic IFAs

* Previous work of Taylor et al

(Y,O2,O3): Opponent representation of an image

Multi-resolution Y images

Image map of predictedvisible luminance

differences

Image map of predictedvisible blue-yellow

differences

Image map of predictedvisible red-green

differences

Purdue UniversityPage 11

PsychometricLUT (f,o2,c2)

Chromatic diff.discriminationRed-green IFARed-green IFA

PsychometricSelector

ChannelResponsePredictor

LimitedMemory

Prob. Sum.

LowpassPyramid

LowpassPyramid

Chromatic Diff.Decomposition

Chromatic Diff.Decomposition

c

+

–

Adaptation level

Contrast Decomposition

Contrast Decomposition

Achromatic IFAAchromatic IFAPsychometric

LUT (f,Y,c1)

Lum. contrastdiscrimination

Contrast: luminance contrast & chromatic difference

Purdue UniversityPage 12

IFA componentsIFA components

• Psychometric LUT Results from psychophysical experiment

Stored in the form of Lookup-Table: (f,Y,c1), (f,o2,c1), (f,o3,c1)

Time consuming, but it is done off-line

• Image processing: Lowpass pyramid: create 5 multi-resolution images

» Lowpass filtering + 2 in horizontal and vertical direction

» Normalized by Y images if it is a chromatic IFA

Signal decomposition: create 8 orientation-specific contrast or chromatic-difference images at each resolution

Lowpass pyramid + Signal decomposition: 40 (5 levels 8 orientations) visual channels for each image pixel

Purdue UniversityPage 13

IFA componentsIFA components(cont’d)(cont’d)

• Image processing (continued):

Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT

Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability

Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel

3,2,1645.12

erf5.05.0

i

cp

i

5

1

)1(1k

kpP

Purdue UniversityPage 14

Estimating parameters of LUTEstimating parameters of LUT(Stimulus: Isoluminant Gabor patch)(Stimulus: Isoluminant Gabor patch)

• Red-green (O2 or o2) stimulus

Keep Y, O3 (o3) constant

Let O2=Yo2+Yc2cos(.)e(.) or equivalently o2’ =o2+c2cos(.)e(.)

• (Y,o2,o3) specifies the background color, c2 is the chromatic difference

3.0,2.0,885.6),,( :RG1 Floyd"" 32 ooY

)()cos(2.0 e)cos(2.0

Gabor patch f, o2, c2

Purdue UniversityPage 15

away m 1.5 from viewed

1.0 ,1.0 ee,cycle/degr 4 ,)3.0,2.0,885.6(),,( 232 ccfooY

Estimating parameters of LUTEstimating parameters of LUT(Psychophysical method)(Psychophysical method)

• Red-green stimulus: (Y,o2,o3) specifies the background color, c2 is

the ref. chromatic difference

• Which stimulus has less chromatic difference?

3.0,2.0,885.6 3.0,2.0,885.6

)(2 )cos( ec )(

2 )cos()( ecc

-0.02 -0.01 0 0.01 0.020

0.2

0.4

0.6

0.8

1

c

Purdue UniversityPage 16

-0.02 -0.01 0 0.01 0.020

0.2

0.4

0.6

0.8

1Subject WW’s responses

prob

abil

ity

c

)0.0065,0.0011(N

Estimating parameters of LUT Estimating parameters of LUT (Data analysis)(Data analysis)

• Fit subject’s responses to a Normal distribution using probit analysis

• Record the standard deviation as the discrimination threshold

• LUT: rg(f,o2,c2)

Purdue UniversityPage 17

Estimating parameters of LUTEstimating parameters of LUT(List of experimental conditions)(List of experimental conditions)

indicate spatial frequency of 1, 2, 4, 8, 16 cpd

Purdue UniversityPage 18

Representative resultsRepresentative results

• Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black.

• Threshold is not affected strongly by the reference chromatic difference

• Chromatic channels function like low-pass filters

-0.2 0 0.2 0.4 0.60

0.05

0.1

0.15

0.2

-0.1 0 0.1 0.2 0.30

0.02

0.04

0.06

Reference c3Reference c2

Thr

esho

ld

Thr

esho

ld

Red-green discrimination atRG1:(Y,o2,o3)=(5,0.2,-0.3)

Blue-yellow discrimination atBY1:(Y,o2,o3)=(5,0.3,0.2)

Purdue UniversityPage 19

CIFA output for example distortionsCIFA output for example distortions(Hue change)(Hue change)

Luminance R-G B-Y

Purdue UniversityPage 20

CIFA output for example distortionsCIFA output for example distortions(Blurring)(Blurring)

Luminance R-G B-Y

Purdue UniversityPage 21

CIFA output for example distortionsCIFA output for example distortions(Limited gamut)(Limited gamut)

Luminance R-G B-Y

Purdue UniversityPage 22

ConclusionConclusion

• CIFA provides good assessment of the perceived visible

differences over a range of image contents and distortion types

• Chromatic difference describes the color percept of HVS

efficiently

• Suggestions on future directions

Add DC component in the LUT in chromatic IFAs

Subjective validation

Improve spatial localization

Take dependency between visual channels into account (in prob. Sum. stage)

Purdue UniversityPage 23

CIFA output for example distortionsCIFA output for example distortions (Limited color quantization)(Limited color quantization)

Luminance R-G B-Y

Purdue UniversityPage 24

CIFA output for example distortionsCIFA output for example distortions (Limited gamut)(Limited gamut)

Luminance R-G B-Y

Purdue UniversityPage 25

CIFA output for example distortionsCIFA output for example distortions(Increased saturation)(Increased saturation)

Luminance R-G B-Y