ai-mei huang and truong nguyen image processing (icip), 2009 16th ieee international conference on 1
Post on 19-Dec-2015
218 views
TRANSCRIPT
![Page 1: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/1.jpg)
1
MOTION VECTOR PROCESSING USING THE
COLOR INFORMATION
Ai-Mei Huang and Truong NguyenImage Processing (ICIP), 2009 16th IEEE International Conference
on
![Page 2: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/2.jpg)
2
CONTENTSINTRODUCTION
COLOR INFORMATION
MV PROCESSING FOR MCFI USING THE COLOR INFORMATION
SIMULATIONS
CONCLUSIONS
![Page 3: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/3.jpg)
3
Introduction(1/2)
Color information has been shown to be effective in the object edges detection Due to its insensitivity on specular reflection Prevent false edge detection as compared to
luminance-based methods
Y1 Y2
Y3 Y4Cb Cr8
8
8
8
16
16
MB (Macro Block)
Chrominance(color information)Luminance(intensity)
![Page 4: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/4.jpg)
4
Introduction(2/2)Color has sharper and more consistent
variations between object boundaries Applications often take the color information to assist
the image segmentation process.In our previous work [8]
The color information was found very useful for the unreliable MV detection
Especially in the areas where the luminance component tends to distribute uniformly.
In this paper, we would like to Examine the color information Analyze how the chrominance components can be
used to assist the MV processing in MCFI.[8] A.-M. Huang and T. Nguyen, “A novel multi-stage motion vectorprocessing method for motion compensated frame interpolation,”in Proc. ICIP’07, pp. 389–392, 2007.
![Page 5: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/5.jpg)
5
Color Information(1/5)
The luminance components have stronger intensity distribution than the chrominance components Conventional motion estimation often ignore the color
information due to the complexity.
Y
Cb
Cr
![Page 6: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/6.jpg)
6
Color Information(2/5)
Color characteristics are distinct from luminance Such as the insensitivity in highlight or shadow areas Used in preventing the false edge detection
The chrominance improves the edge identification for the static text, face features, the cap, and the shirts
![Page 7: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/7.jpg)
7
Color Information(3/5)
If the moving objects have sharp edges, the ambiguous motions seem more unlikely to appear.
From Fig. 1(b), we can observe that the luminance has very smooth variations around the face and shirts areas. Since the motion is mainly determined using the
luminance difference, the motion can be easily wrong in these areas.
![Page 8: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/8.jpg)
8
Color Information(4/5)
Many artifacts around the nose and the shirts in (a)MVs around the shirts can only be detected in (c)
The luminance have uniformly distribution so that the encoder always chooses the face motion. The color has strong gradients.
![Page 9: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/9.jpg)
9
Color Information(5/5) Generally, chrominance residual distribution is similar to the luminancecomponents. The pavement and lawn have very similar intensity. The color difference will become relatively large.
![Page 10: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/10.jpg)
10
Motion Vector AnalysisThe residual energy with color consideration
be represented as follows:
rY (i, j), rCb(i, j), and rCr(i, j) are the reconstructed residual signals of Y, Cb and Cr components of the 8×8 block, bm,n
α is the weight used to emphasize the degree of color difference. Empirically set α=8 for 4:2:0 YUV
The residues are embedded in the reconstructed signals during the decoding process.
![Page 11: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/11.jpg)
11
MV classification process
Compare Em,n to a predefined threshold, ε1, based on the combined residual information.
The adjacent MBs will be merged as a group using the residual energy distribution.
16
16
MB
![Page 12: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/12.jpg)
12
Motion Vector Correction using the Color Information
Minimizing the absolute bidirectional prediction difference (ABPD) between forward and backward predictions.
![Page 13: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/13.jpg)
13
SIMULATIONSTwo video sequences, FOREMAN and FORMULA 1
CIF frame resolutionall encoded using H.263
with even frames skipped and the skipped frames are interpolated
![Page 14: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/14.jpg)
14
Visual Comparisons(1/2)
Fig. 4(c), the artifacts around the nose and the eye are reduced. These artifacts are removed in Fig. 4(d).
Since the chrominance information sharpens the residual energy. Unreliable MVs around the shirts and face areas can be identified and be corrected accordingly.
![Page 15: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/15.jpg)
15
Visual Comparisons(2/2)
The intensity between grass and pavement is very similar.
So the motion estimation easily fails on the white lines areas.
![Page 16: Ai-Mei Huang and Truong Nguyen Image Processing (ICIP), 2009 16th IEEE International Conference on 1](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d2d5503460f94a03c24/html5/thumbnails/16.jpg)
16
CONCLUSIONSWe present using color information for the
MV reliability classification.
Unreliable MVs with small luminance difference can be effectively detected.