14th european signal processing conference 4-8 september 2006 · 14th european signal processing...
TRANSCRIPT
![Page 1: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/1.jpg)
![Page 2: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/2.jpg)
14th European Signal Processing ConferenceEUSIPCO 2006, 4-8 September 2006
Special Session, Image and Video Quality Evaluation
Task impact on the visual attention in subjective image
quality assessment
A. Ninassi ¹�², O. Le Meur ¹, P. Le Callet ², D. Barba ², A. Tirel ²
¹THOMSON R&D, VCL, Cesson-Sevigné
²IRCCyN UMR 6597 CNRS, Nantes
![Page 3: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/3.jpg)
3
Introduction
“ The most efficient quality metrics are based on the Human Visual System ”
An important component of the HVS=
Visual attention
![Page 4: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/4.jpg)
4
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 5: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/5.jpg)
5
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 6: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/6.jpg)
6
Visual attention : Definition
“ Visual attention allows us to select the relevant information in our environment, in connection or not with a particular task ”
Two mechanisms are involved in visual attention controlBottom-up
(voluntary attention)Salient elements of our visual
field catch our attention�
Mechanism based on signal
Top-down(involuntary attention)
Our attention is guided by the task to accomplish
�
Mechanism based on task
![Page 7: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/7.jpg)
7
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 8: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/8.jpg)
8
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 9: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/9.jpg)
9
Some questions … 1/3
Does the quality assessment task affect the visual strategy?
Importance of the mechanisms :– Bottom-up?– Top-down?
It is fundamental if we want to elaborate a quality metric based on visual attention model : Which visual strategy must be imitated?
![Page 10: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/10.jpg)
10
Some questions … 2/3
Does a learning process exist during a quality assessment campaign ?
If the same picture is shown several times (DSIS protocol as an example) :
– Do we learn how to evaluate the picture?– Do we learn where to look ?
![Page 11: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/11.jpg)
11
Some questions … 3/3
Do the artifacts modify the visual strategy ?
It is fundamental if we want to elaborate a quality metric based on visual attention model : Which visual strategy must be imitated?
![Page 12: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/12.jpg)
12
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 13: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/13.jpg)
13
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 14: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/14.jpg)
14
Experiments : The ground truth 1/3
Eye gaze tracking experiments
An eye-tracker is a device elaborated to measure and to record the position where
a human observer is looking at
Experiments :– 20 observers– 120 still pictures– Visualization distance 4H
Fixation density mapOriginal picture
![Page 15: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/15.jpg)
15
Experiments : The ground truth 2/3
Free viewing task 20 pictures (10 unimpaired references, 10 impaired versions)
Picture quality assessment task – 130 pictures (10 unimpaired references,120 impaired versions)– DSIS protocol (Double Stimulus Impairment Scale)– Constraint : 2 sequences from the same reference picture must not
follow themselves
![Page 16: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/16.jpg)
16
Experiments : The ground truth 3/3
Degradation category rating (5 scores) :– Imperceptible– Not annoying– Slightly annoying– Annoying– Very annoying
How to rate ?– Scoring screen– selection and confirmation based on eye
gaze positionScore is Not annoying
![Page 17: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/17.jpg)
17
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 18: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/18.jpg)
18
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 19: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/19.jpg)
19
Results : Analysis
– Fixation duration– Saliency map (average fixation density map)
� Kullback-Leibler divergence KL� Correlation coefficient CC
![Page 20: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/20.jpg)
20
Results : Some answers …
Does the quality assessment task affect the visual strategy?– Fixation duration :
Reference in quality task > Reference in free taskObservers try to accurately memorize some parts of the picture
– Saliency map :
Notable dissimilarity
� The quality assessment task affects the visual strategy
[0.77,0.92][0.3,0.5]Reference in quality task vs reference in free task[0.66,0.9][0.42,0.95]Impaired in quality task vs reference in free task
CC rangeKL rangeComparison A) and C)
![Page 21: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/21.jpg)
21
Results : Some answers …
Average fixation duration by picture for :– � The reference pictures in free task– � The reference pictures in quality task– � The impaired pictures in quality task
150
200
250
300
350
400
450
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rag
e fi
xati
on
du
rati
on
(m
s)
![Page 22: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/22.jpg)
22
Results : Some answers …
� A) Reference in quality task vs reference in free task
� C) Impaired in quality task vs reference in free task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL
![Page 23: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/23.jpg)
23
Results : Some answers …
•Illustration of the comparison A) :Reference in quality task (1,2 and 5th) vs reference in free task(Difference between saliency maps)
![Page 24: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/24.jpg)
24
Results : Some answers …
Does a learning process exist during a quality assessment campaign ?
Saliency map :
No notable dissimilarity
� No significant learning process
[0.9,0.97][0.12,0.18]Reference in quality task
vs first reference in quality task
CC rangeKL rangeComparison B)
![Page 25: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/25.jpg)
25
Results : Some answers …
� B) Reference in quality task vs first reference in quality task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL
![Page 26: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/26.jpg)
26
Results : Some answers …
Do the artifacts modify the visual strategy ?– Saliency map comparison :
Notable dissimilarity
– Saliency map cross analysis (confidence intervals ci) :
Confidence interval increase between A) et C)
� The artifacts do modify the visual strategy
[0.8,0.96][0.11,0.4]Impaired in quality task
vs its associated reference in quality task
CC rangeKL rangeComparison D)
[0.008,0.1][0.04,0.13]Impaired in quality task vs reference in free task[0.007,0.04][0.02,0.09]Reference in quality task vs reference in free taskCCci rangeKLci rangeComparison A) and C)
![Page 27: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/27.jpg)
27
Results : Some answers …
� D) Impaired in quality task vs its associated reference in quality task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL
![Page 28: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/28.jpg)
28
Results : Some answers …
•Illustration of the comparison D) : Impaired in quality task vs its associated reference in quality task(Difference between saliency maps)
![Page 29: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/29.jpg)
29
Results : Some answers …
� A) Reference in quality task vs reference in free task
� C) Impaired in quality task vs reference in free task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL
![Page 30: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/30.jpg)
30
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 31: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/31.jpg)
31
Synopsis
1. Visual attention2. Some questions …3. Experiments4. Results : Some answers …5. Conclusion
![Page 32: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/32.jpg)
32
Conclusion
� The quality assessment task affects the visual strategy– Fixation duration– Saliency map
� No significant learning processObservers are not more competitive at the end of the test than at the
beginning. No visual adaptation or task learning.
To elaborate a quality metric based on visual attention model, we must be conscious that :
– We can use a bottom-up model (free view condition)– But, In a quality assessment campaign the visual strategy is different
Future work :Study the dependence between visual attention and type of degradation� The artifacts do modify the visual strategy
![Page 33: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/33.jpg)
33
Appendix
![Page 34: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/34.jpg)
34
Appendix : Results
� A) Reference in quality task vs reference in free task
� B) Reference in quality task vs first reference in quality task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL
![Page 35: 14th European Signal Processing Conference 4-8 September 2006 · 14th European Signal Processing Conference EUSIPCO 2006, 4-8 September 2006 Special Session, Image and Video Quality](https://reader036.vdocuments.mx/reader036/viewer/2022070916/5fb6298824c60e75755a854a/html5/thumbnails/35.jpg)
35
Appendix : Results
� C) Impaired in quality task vs reference in free task
� D) Impaired in quality task vs its associated reference in quality task
0
0,2
0,4
0,6
0,8
1
1,2
Barba Isabe Clown Mandrill Boats Lena Peppers Plane House Fruit
Ave
rage
KL