application of scalable visual sensitivity profile in image and video coding
DESCRIPTION
Qian Chen, Guangtao Zhai , Xiaokang Yang, and Wenjun Zhang ISCAS,2008. Application of Scalable Visual Sensitivity Profile in Image and Video Coding. Outline. Introduction Scalable visual sensitivity profile (SVSP) SVSP in noise-shaping SVSP in ROI coding of JPEG2000 - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/1.jpg)
Application of Scalable Visual Sensitivity Profile in Image and Video Coding
Qian Chen, Guangtao Zhai, Xiaokang Yang, and Wenjun ZhangISCAS,2008
![Page 2: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/2.jpg)
Outline Introduction Scalable visual sensitivity profile
(SVSP) SVSP in noise-shaping SVSP in ROI coding of JPEG2000 SVSP in ROI scalable video coding Conclusion
![Page 3: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/3.jpg)
Introduction Computational visual attention
models have been developed over the last 20 years and have already facilitated various aspects of the evolution in visual communication systems.
Its important applications is to enhance the image and video compression algorithms perceptually.
![Page 4: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/4.jpg)
SVSP
Feature extraction
down-sampling filter
Center surround receptive field simulation
Cross level addition and normalize
Non-linear feature combination
![Page 5: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/5.jpg)
SVSP (1) Low-level Feature Detection
Intensity channel :
Color channels :
Orientation channel :
motion channel :optical flow
Gabor filter
![Page 6: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/6.jpg)
SVSP (2) By iteratively down-sampling for L times of
these channels ,we can create pyramids for each of these
channels of the framei
Center-surround Receptive Field Simulation
c ∈ [0, 8], s = c + δ,δ ∈ [−3,−2,−1, 1, 2, 3] and s is thrown away if s ∈ [0, 8].
![Page 7: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/7.jpg)
SVSP(3) Cross level addition and normalize
Non-linear Feature Combination
![Page 8: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/8.jpg)
SVSP
Skin & caption detection
Down-sampling filter
SVSP integration
Post-processing
![Page 9: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/9.jpg)
SVSP(4) Skin Color Detection
The skin color area indicates the appearance of people and often attracts human attention.
Hsu’s [5] skin model
Caption Detection Luo’s [6]
![Page 10: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/10.jpg)
SVSP(5) SVSP integration
Considering the fact that human face by its nature attracts more low-level human attention, we emphasize skin map more and α = 1.5, β = 1.2
Ref.G. T. Zhai, Q. Chen, X. K. Yang, W. J. Zhang,”Scalable Visual Significance Profile Estimation”, submitted to International Conference on Acoustics, Speech, and Signal Processing, April, 2008, Las Vegas, US.
![Page 11: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/11.jpg)
Noise-shaping To validate the effectiveness of the
proposed model. JND (Just-noticeable
distortion/difference) :refers to the visibility threshold below which changes cannot be perceived by human.
Noise shaping is a popular way to evaluate the correctness of JND models.
![Page 12: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/12.jpg)
Noise-shaping Noise-injection process is :
The proposed VSP-based JND model is :
We will compare it with Chou’s JND model [8] JNDC and the JND model we previously proposed [9] JNDY
![Page 13: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/13.jpg)
Noise-shaping
(a)Luminance of frame 51 in president debate.(b)Chou’s JND model, PNSR=25.99 dB.
(c)Yang’s JND model, PNSR=25.99 dB. (d)proposed VSP-based JND model, PNSR=25.99 dB.
![Page 14: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/14.jpg)
ROI coding of JPEG2000 We define the arbitrary ROIa in an image as
areas that take half the top values in .
To generate a rectangular ROIr , we explore a seeded region growing algorithm , seed is placed at the most saliency point in and then expands to surroundings. The stopping criterion is that the pixel value on region borders falls below 60% of the starting seed-value.
![Page 15: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/15.jpg)
ROI coding of JPEG2000
(a) Details of the most sensitiveof frame 51 in president debate.
(b) Details of image coded at 0.1bpp witharbitrary ROI defined in VSP, PSNR-Y=27.2dB.
(c) Details of image coded at 0.1bpp with rectangular-shaped ROI defined in SVP, PSNR-Y=32.6dB.
(d)Details of image coded at 0.1bpp without ROI, PSNR-Y=24.0dB.
![Page 16: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/16.jpg)
ROI scalable video codingSVSP
Filter out isolated
Most saliency point
Sensitive region
![Page 17: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/17.jpg)
ROI scalable video coding
(a) Average PSNR-Y vs. bit rate of president debate.
(b) Average PSNR-Y vs. bit rate of foreman.
(c) Average PSNR-Y vs. bit rate of crew. (d)Average PSNR-Y vs. bit rate of coastguard.
![Page 18: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/18.jpg)
ROI scalable video coding Visual comparison in saliency area of
frame 60 in president debate, CIF size coded at 900 kbps.
(a)without ROI (b)with SVSP defined ROI
![Page 19: Application of Scalable Visual Sensitivity Profile in Image and Video Coding](https://reader035.vdocuments.mx/reader035/viewer/2022070422/56816380550346895dd46232/html5/thumbnails/19.jpg)
Conclusion This paper applies the proposed
computational model for scalable visual sensitivity profile (SVSP) to image/video processing.
Extensive experimental results have justified the effectiveness of the proposed SVSP model.