color image fidelity assessor *
DESCRIPTION
Color Image Fidelity Assessor *. Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University) Jan P. Allebach (Purdue University). * Research supported by HP Company while Wencheng Wu was at Purdue. Outline. Introduction Spatial color descriptor: chromatic difference - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/1.jpg)
Purdue UniversityPage 1
Color Color Image Fidelity Assessor Image Fidelity Assessor *
Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University)Jan P. Allebach (Purdue University)
* Research supported by HP Company while Wencheng Wu was at Purdue
![Page 2: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/2.jpg)
Purdue UniversityPage 2
OutlineOutline
• Introduction
• Spatial color descriptor: chromatic difference
• Structure of Color Image Fidelity Assessor (CIFA)
• Psychophysical experiment and its results
• Test examples
• Conclusion
![Page 3: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/3.jpg)
Purdue UniversityPage 3
IntroductionIntroduction(Motivation)(Motivation)
• Image fidelity assessment is important in the development of
imaging systems and image processing algorithms
Create visually lossless reproduction
Allocate efforts on most visible area
• Subjective evaluation is expensive and slow.
![Page 4: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/4.jpg)
Purdue UniversityPage 4
IntroductionIntroduction(Prior work)(Prior work)
• Simple but not working
Root-Mean-Square Error
• Consider structure of HVS and perceptual process
Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA)
Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab)
![Page 5: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/5.jpg)
Purdue UniversityPage 5
IntroductionIntroduction(CVDM vs. CIFA)(CVDM vs. CIFA)
• Both operate along opponent-color coordinates
• Both incorporate results from electrophysiological and
psychophysical exp.
• They differ in a similar way as VDP vs. IFA
CIFA has closer link between the structure of the model and the psychophysical data used by the model
• CIFA normalize the chromatic responses
This discounts luminance effect in chromatic channels
This reduces the dimension of psychometric LUT
![Page 6: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/6.jpg)
Purdue UniversityPage 6
IntroductionIntroduction(Overview of CIFA)(Overview of CIFA)
• Color extension of Taylor’s achromatic IFA
• The model predicts perceived image fidelity
Assesses visible differences in the opponent channels
Explains the nature of visible difference (luminance change vs. color shift)
Color ImageFidelityAssessor(CIFA)
Ideal
Rendered
Viewing parameters
Image mapsof predicted
visibledifferences
![Page 7: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/7.jpg)
Purdue UniversityPage 7
Chromatic differenceChromatic difference(Definition)(Definition)
• Objective: evaluate the spatial interaction between colors
• First transform CIE XYZ to opponent color space (O2,O3) *
* X. Zhang and B.A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97
tieschromaticiopponent )/,/(),( 3232 YOYOoo
.3,22
minmax
ioo
ci
• Then normalize to obtain opponent chromaticities (o2,o3)
• Define chromatic difference (analogous to luminance contrast c1)
Z
Y
X
O
O
Y
501.059.0086.0
077.029.0449.0
010
3
2
Luminance Red-Green
Blue-Yellow
![Page 8: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/8.jpg)
Purdue UniversityPage 8
Opponent color representationOpponent color representation
(13.3,o2,0.17) (13.3,0.24,o3)(Y,0.24,0.17)
(Y,o2,o3)
![Page 9: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/9.jpg)
Purdue UniversityPage 9
)16/cos(2.0 J)16/cos(1.0 J
Chromatic differenceChromatic difference(illustration)(illustration)
• Chromatic difference is a measure of chromaticity variation
• Chromatic difference is a spatial feature derived from opponent
chromaticity that has little dependence upon luminance
174438.0,235924.0,885.6),,( :)( pixelat gray Floyd"" 32 ooYI.J
)16/cos(05.0 J
0.1 0.10.20.05
• Chromatic difference is the amplitude of the sinusoidal grating
)16/cos(1.0 J
![Page 10: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/10.jpg)
Purdue UniversityPage 10
CIFACIFA
Ideal Y Image
Rendered Y Image
Ideal O2 Image
Rendered O2 Image
Ideal O3 Image
Rendered O3 Image
Blue-yellowIFA
Red-greenIFA
Achromatic*IFA
Chromatic IFAs
* Previous work of Taylor et al
(Y,O2,O3): Opponent representation of an image
Multi-resolution Y images
Image map of predictedvisible luminance
differences
Image map of predictedvisible blue-yellow
differences
Image map of predictedvisible red-green
differences
![Page 11: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/11.jpg)
Purdue UniversityPage 11
PsychometricLUT (f,o2,c2)
Chromatic diff.discriminationRed-green IFARed-green IFA
PsychometricSelector
ChannelResponsePredictor
LimitedMemory
Prob. Sum.
LowpassPyramid
LowpassPyramid
Chromatic Diff.Decomposition
Chromatic Diff.Decomposition
c
+
–
Adaptation level
Contrast Decomposition
Contrast Decomposition
Achromatic IFAAchromatic IFAPsychometric
LUT (f,Y,c1)
Lum. contrastdiscrimination
Contrast: luminance contrast & chromatic difference
![Page 12: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/12.jpg)
Purdue UniversityPage 12
IFA componentsIFA components
• Psychometric LUT Results from psychophysical experiment
Stored in the form of Lookup-Table: (f,Y,c1), (f,o2,c1), (f,o3,c1)
Time consuming, but it is done off-line
• Image processing: Lowpass pyramid: create 5 multi-resolution images
» Lowpass filtering + 2 in horizontal and vertical direction
» Normalized by Y images if it is a chromatic IFA
Signal decomposition: create 8 orientation-specific contrast or chromatic-difference images at each resolution
Lowpass pyramid + Signal decomposition: 40 (5 levels 8 orientations) visual channels for each image pixel
![Page 13: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/13.jpg)
Purdue UniversityPage 13
IFA componentsIFA components(cont’d)(cont’d)
• Image processing (continued):
Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT
Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability
Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel
3,2,1645.12
erf5.05.0
i
cp
i
5
1
)1(1k
kpP
![Page 14: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/14.jpg)
Purdue UniversityPage 14
Estimating parameters of LUTEstimating parameters of LUT(Stimulus: Isoluminant Gabor patch)(Stimulus: Isoluminant Gabor patch)
• Red-green (O2 or o2) stimulus
Keep Y, O3 (o3) constant
Let O2=Yo2+Yc2cos(.)e(.) or equivalently o2’ =o2+c2cos(.)e(.)
• (Y,o2,o3) specifies the background color, c2 is the chromatic difference
3.0,2.0,885.6),,( :RG1 Floyd"" 32 ooY
)()cos(2.0 e)cos(2.0
Gabor patch f, o2, c2
![Page 15: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/15.jpg)
Purdue UniversityPage 15
away m 1.5 from viewed
1.0 ,1.0 ee,cycle/degr 4 ,)3.0,2.0,885.6(),,( 232 ccfooY
Estimating parameters of LUTEstimating parameters of LUT(Psychophysical method)(Psychophysical method)
• Red-green stimulus: (Y,o2,o3) specifies the background color, c2 is
the ref. chromatic difference
• Which stimulus has less chromatic difference?
3.0,2.0,885.6 3.0,2.0,885.6
)(2 )cos( ec )(
2 )cos()( ecc
-0.02 -0.01 0 0.01 0.020
0.2
0.4
0.6
0.8
1
c
![Page 16: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/16.jpg)
Purdue UniversityPage 16
-0.02 -0.01 0 0.01 0.020
0.2
0.4
0.6
0.8
1Subject WW’s responses
prob
abil
ity
c
)0.0065,0.0011(N
Estimating parameters of LUT Estimating parameters of LUT (Data analysis)(Data analysis)
• Fit subject’s responses to a Normal distribution using probit analysis
• Record the standard deviation as the discrimination threshold
• LUT: rg(f,o2,c2)
![Page 17: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/17.jpg)
Purdue UniversityPage 17
Estimating parameters of LUTEstimating parameters of LUT(List of experimental conditions)(List of experimental conditions)
indicate spatial frequency of 1, 2, 4, 8, 16 cpd
![Page 18: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/18.jpg)
Purdue UniversityPage 18
Representative resultsRepresentative results
• Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black.
• Threshold is not affected strongly by the reference chromatic difference
• Chromatic channels function like low-pass filters
-0.2 0 0.2 0.4 0.60
0.05
0.1
0.15
0.2
-0.1 0 0.1 0.2 0.30
0.02
0.04
0.06
Reference c3Reference c2
Thr
esho
ld
Thr
esho
ld
Red-green discrimination atRG1:(Y,o2,o3)=(5,0.2,-0.3)
Blue-yellow discrimination atBY1:(Y,o2,o3)=(5,0.3,0.2)
![Page 19: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/19.jpg)
Purdue UniversityPage 19
CIFA output for example distortionsCIFA output for example distortions(Hue change)(Hue change)
Luminance R-G B-Y
![Page 20: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/20.jpg)
Purdue UniversityPage 20
CIFA output for example distortionsCIFA output for example distortions(Blurring)(Blurring)
Luminance R-G B-Y
![Page 21: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/21.jpg)
Purdue UniversityPage 21
CIFA output for example distortionsCIFA output for example distortions(Limited gamut)(Limited gamut)
Luminance R-G B-Y
![Page 22: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/22.jpg)
Purdue UniversityPage 22
ConclusionConclusion
• CIFA provides good assessment of the perceived visible
differences over a range of image contents and distortion types
• Chromatic difference describes the color percept of HVS
efficiently
• Suggestions on future directions
Add DC component in the LUT in chromatic IFAs
Subjective validation
Improve spatial localization
Take dependency between visual channels into account (in prob. Sum. stage)
![Page 23: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/23.jpg)
Purdue UniversityPage 23
CIFA output for example distortionsCIFA output for example distortions (Limited color quantization)(Limited color quantization)
Luminance R-G B-Y
![Page 24: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/24.jpg)
Purdue UniversityPage 24
CIFA output for example distortionsCIFA output for example distortions (Limited gamut)(Limited gamut)
Luminance R-G B-Y
![Page 25: Color Image Fidelity Assessor *](https://reader036.vdocuments.mx/reader036/viewer/2022062301/568157a7550346895dc53686/html5/thumbnails/25.jpg)
Purdue UniversityPage 25
CIFA output for example distortionsCIFA output for example distortions(Increased saturation)(Increased saturation)
Luminance R-G B-Y