robust face hallucination using quantization-adaptive ... · robust face hallucination using...
Post on 04-Aug-2018
222 Views
Preview:
TRANSCRIPT
Robust Face Hallucination using Quantization-Adaptive Dictionaries
Reuben FarrugiaChristine Guillemot
IEEE Int. Conf. on Image Processing, Arizona, USA 26th September 2016
Outline
• Limitations of existing Face Hallucination Methods• Analysis of Compression artefacts on faces• Proposed Robust Face Hallucination Methods• Experimental Results• Robust Face Hallucination in the Wild
Limitations of existing Face Hallucination methods
Face Hallucination
The low-quality face image 𝑿 isdivided into overlapping patchesෝ𝒙1,ෝ𝒙2,…ෝ𝒙𝑷.
These weights 𝒘𝑝are then used to
combine the corresponding high-
quality dictionary 𝑯𝑝 to restore 𝒚𝑝.
The optimal weights 𝒘𝑝 suitable to
reconstruct ෝ𝒙𝑝 are derived using the
low-quality dictionary 𝑳𝑝.
Face Hallucination Dictionary Construction
Let X and Y denote the high- andlow- quality facial images,respectively. Face Hallucinationmethods formulate the acquisitionmodel as
𝑿 =↓ 𝑩𝒀 + 𝜖 where 𝑩 is the blurring function and ↓ is thedownscaling operator.
However, this formulation is valid for uncompressed images and is noteffective when the low-quality image is compressed.
Non-Robust Face Hallucination
Comparing the original high quality face image (left), the downscaled and compressed low-quality face image (centre) and the super-resolved image using a state-of-the-art face hallucination method (right). The original image is down-scaled such that they have inter-eye distance of 10 pixels and compressed using H.264/AVC Intra with a quantization parameter of 20.
Analysis of Compression artefacts on Faces
Compression Artefacts on FacesH
.26
4/A
VC
Co
de
c d
x =
20
H.2
64
/AV
C C
od
ec d
x =
10
QP = 10 QP = 35 QP = 10 QP = 35
Compression Artefacts on FacesJP
EG C
od
ec
dx
= 2
0
JPEG
Co
dec
dx
= 1
0
Quality = 70 Quality = 30 Quality = 70 Quality = 50
Proposed Robust Face Hallucination Method
Robust Face Hallucination Dictionary Const.
The acquisition model considered in this work to construct thecoupled dictionaries includes the codec following the blurring anddownscaling operators and can be formulated using𝑿 = 𝑻−1𝑄 𝑻 ↓ 𝑩𝒀 + 𝜖 where T is the transform and Q is thequantization function.
Robust Position Patch (RPP)
The Robust Position Patch derives the combination weights byminimising the following optimization
𝑤𝑝 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑤𝑝ො𝑥𝑝 − 𝐿𝑝𝑤𝑝 2
𝑠. 𝑡. 𝑤𝑝 2= 1
Robust Position Patch (RPP)
The Robust Position Patch derives the combination weights byminimising the following optimization
𝑤𝑝 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑤𝑝ො𝑥𝑝 − 𝐿𝑝𝑤𝑝 2
𝑠. 𝑡. 𝑤𝑝 2= 1
Robust Sparse Position Patch (RSPP)
The Robust Sparse Position Patch derives the combination weights byminimising the following optimization
𝑤𝑝 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑤𝑝𝑤𝑝 1
𝑠. 𝑡. ො𝑥𝑝 − 𝐿𝑝𝑤𝑝 2< 𝜖
Experimental Results
PSNR Analysis
20 25 30
Baseline 10 27.59 27.33 26.63
PP 10 25.74 25.73 25.53
RPP 10 30.19 29.73 28.65
SPP 10 25.52 25.43 25.20
RSPP 10 30.12 29.54 28.51
LINE 10 25.32 25.22 25.01
RLINE 10 29.95 29.39 28.30
Baseline 20 29.63 29.46 29.03
PP 20 34.13 33.29 31.70
RPP 20 34.11 33.36 32.05
SPP 20 33.91 32.89 31.22
RSPP 20 34.01 33.21 31.88
LINE 20 33.90 32.88 31.25
RLINE 20 33.94 33.09 31.64
Quantization ParameterMethod dx
H.264/AVC Intra Codec
50 60 75
Baseline 10 26.78 27.02 27.48
PP 10 27.36 27.71 28.50
RPP 10 28.34 28.58 29.18
SPP 10 26.82 27.16 27.83
RSPP 10 28.13 28.40 28.96
LINE 10 26.67 27.02 27.74
RLINE 10 27.99 28.23 28.83
Baseline 20 30.05 30.30 30.75
PP 20 31.07 31.39 32.23
RPP 20 31.83 32.09 32.75
SPP 20 30.54 30.86 31.59
RSPP 20 31.62 31.88 32.47
LINE 20 30.52 30.84 31.65
RLINE 20 31.51 31.81 32.47
Quality ParameterMethod dx
JPEG Codec
Recognition Analysis
20 25 30
Baseline 10 0.538 0.498 0.354
PP 10 0.416 0.377 0.281
RPP 10 0.553 0.486 0.318
SPP 10 0.469 0.399 0.304
RSPP 10 0.619 0.561 0.385
LINE 10 0.485 0.424 0.293
RLINE 10 0.607 0.556 0.394
Baseline 20 0.722 0.693 0.625
PP 20 0.748 0.714 0.640
RPP 20 0.745 0.700 0.623
SPP 20 0.780 0.732 0.667
RSPP 20 0.761 0.723 0.634
LINE 20 0.779 0.744 0.681
RLINE 20 0.770 0.724 0.654
Quantization ParameterMethod dx
H.264/AVC Intra Codec
50 60 75
Baseline 10 0.362 0.380 0.450
PP 10 0.424 0.433 0.517
RPP 10 0.382 0.436 0.479
SPP 10 0.415 0.436 0.525
RSPP 10 0.428 0.461 0.526
LINE 10 0.444 0.481 0.532
RLINE 10 0.417 0.455 0.523
Baseline 20 0.663 0.685 0.699
PP 20 0.673 0.689 0.719
RPP 20 0.668 0.672 0.711
SPP 20 0.675 0.707 0.731
RSPP 20 0.687 0.705 0.727
LINE 20 0.683 0.719 0.726
RLINE 20 0.679 0.698 0.726
Quality ParameterMethod dx
JPEG Codec
Subjective Evaluation
Comparing the original high quality face image (first column), the downscaled and compressed low-quality face image(second column), the super-resolved image using a state-of-the-art face hallucination method (third column) and thesuper-resolved face image using the proposed robust method, where the LM-CSS face SR method is used. The originalimage is down-scaled such that dx = 10 and compressed using H.264/AVC Intra with a quantization parameter of 20.
Robust Face Hallucination in the Wild
Robust Face Hallucination in the WildTraining Images
Low-quality image
Upscale
Warping
HallucinationDegradation
Robust Face Hallucination in the Wild
HQ Image LQ Image RLM-CSS HQ Image LQ Image RLM-CSS
Conclusion
• This work presents a new dictionary construction methodwhich employs the syntax available in the image file tomodel the distortions affecting the low-quality image.
• Experimental Results show a significant gain in terms ofquality with respect to the non-robust hallucinationmethods.
• The proposed method can be extended to super-resolvefacial images in the wild.
Thank You
Reuben Farrugiahttps://www.um.edu.mt/staff/reuben.farrugiareuben.farrugia@um.edu.mt
More information about this project can be found onhttps://www.um.edu.mt/staff/reuben.farrugia/projects/face_hallucination
top related