image interpolation
DESCRIPTION
Image Interpolation. Introduction What is image interpolation? Why do we need it? Interpolation Techniques 1D zero-order, first-order, third-order 2D zero-order, first-order, third-order Directional interpolation* Interpolation Applications Digital zooming (resolution enhancement) - PowerPoint PPT PresentationTRANSCRIPT
EE465: Introduction to Digital Image Processing
1
Image InterpolationImage Interpolation• Introduction
– What is image interpolation?– Why do we need it?
• Interpolation Techniques– 1D zero-order, first-order, third-order– 2D zero-order, first-order, third-order– Directional interpolation*
• Interpolation Applications– Digital zooming (resolution enhancement)– Image inpainting (error concealment)– Geometric transformations
EE465: Introduction to Digital Image Processing
2
IntroductionIntroduction• What is image interpolation?
– An image f(x,y) tells us the intensity values at the integral lattice locations, i.e., when x and y are both integers
– Image interpolation refers to the “guess” of intensity values at missing locations, i.e., x and y can be arbitrary
– Note that it is just a guess (Note that all sensors have finite sampling distance)
EE465: Introduction to Digital Image Processing
3
Introduction (Con’t)Introduction (Con’t)• Why do we need image
interpolation?– We want BIG images
• When we see a video clip on a PC, we like to see it in the full screen mode
– We want GOOD images• If some block of an image gets damaged
during the transmission, we want to repair it
– We want COOL images• Manipulate images digitally can render fancy
artistic effects as we often see in movies
EE465: Introduction to Digital Image Processing
4
Scenario I: Resolution Scenario I: Resolution EnhancementEnhancement
Low-Res.
High-Res.
EE465: Introduction to Digital Image Processing
5
Scenario II: Image InpaintingScenario II: Image Inpainting
Non-damaged Damaged
EE465: Introduction to Digital Image Processing
6
Scenario III: Image WarpingScenario III: Image Warping
EE465: Introduction to Digital Image Processing
7
Image InterpolationImage Interpolation• Introduction
– What is image interpolation?– Why do we need it?
• Interpolation Techniques– 1D zero-order, first-order, third-order– 2D zero-order, first-order, third-order– Directional interpolation*
• Interpolation Applications– Digital zooming (resolution enhancement)– Image inpainting (error concealment)– Geometric transformations
EE465: Introduction to Digital Image Processing
8
1D Zero-order (Replication)
n
f(n)
x
f(x)
EE465: Introduction to Digital Image Processing
9
1D First-order Interpolation (Linear)
n
f(n)
x
f(x)
EE465: Introduction to Digital Image Processing
10
Linear Interpolation FormulaLinear Interpolation Formula
a 1-a
f(n)
f(n+1)f(n+a)
f(n+a)=(1-a)f(n)+af(n+1), 0<a<1
Basic idea: the closer to a pixel, the higher weight is assigned
Note: when a=0.5, we simply have the average of two
EE465: Introduction to Digital Image Processing
11
Numerical ExamplesNumerical Examples
f(n)=[0,120,180,120,0]
f(x)=[0,60,120,150,180,150,120,60,0], x=n/2
f(x)=[0,20,40,60,80,100,120,130,140,150,160,170,180,…], x=n/6
Interpolate at 1/2-pixel
Interpolate at 1/3-pixel
EE465: Introduction to Digital Image Processing
12
n
x
f(n)
f(x)
Cubic spline fitting
1D Third-order Interpolation (Cubic)
EE465: Introduction to Digital Image Processing
13
From 1D to 2D
Just like separable 2D transform (filtering) that can be implemented by two sequential 1D transforms (filters) along row and column direction respectively, 2D interpolation can be decomposed into two sequential 1D interpolations.
The ordering does not matter (row-column = column-row)
Such separable implementation is not optimal but enjoys low computational complexity
EE465: Introduction to Digital Image Processing
14
Graphical InterpretationGraphical Interpretationof Interpolation at Half-pelof Interpolation at Half-pel
row column
f(m,n) g(m,n)
EE465: Introduction to Digital Image Processing
15
Numerical ExamplesNumerical Examplesa b
c d
a a b b
a a b b
c c d d
c c d d
zero-order
first-order
a (a+b)/2b(a+c)/2 (a+b+c+d)/4 (b+d)/2c (c+d)/2 d
EE465: Introduction to Digital Image Processing
16
Numerical Examples (Con’t)Numerical Examples (Con’t)
X(m,n)row m
row m+1
Col n Col n+1
X(m,n+1)
X(m+1,n+1)X(m+1,n)
Ya 1-a
b
1-b
Q: what is the interpolated value at Y?Ans.: (1-a)(1-b)X(m,n)+(1-a)bX(m+1,n)+a(1-b)X(m,n+1)+abX(m+1,n+1)
EE465: Introduction to Digital Image Processing
17
Bicubic Interpolation*Bicubic Interpolation*
EE465: Introduction to Digital Image Processing
18
Limitation with Limitation with bilinear/bicubicbilinear/bicubic
• Edge blurring• Jagged artifacts
X Z
Jagged artifacts
X
Z
Edge blurring
EE465: Introduction to Digital Image Processing
19
Directional Interpolation*Directional Interpolation*
Step 1: interpolate the missing pixels along the diagonal
black or white?
Step 2: interpolate the other half missing pixels
a b
c d
Since |a-c|=|b-d|x
x has equal probabilityof being black or white
a
b
c
dSince |a-c|>|b-d|
x=(b+d)/2=black
x
EE465: Introduction to Digital Image Processing
20
Image InterpolationImage Interpolation• Introduction
– What is image interpolation?– Why do we need it?
• Interpolation Techniques– 1D zero-order, first-order, third-order– 2D zero-order, first-order, third-order– Directional interpolation*
• Interpolation Applications– Digital zooming (resolution enhancement)– Image inpainting (error concealment)– Geometric transformations
EE465: Introduction to Digital Image Processing
21
Pixel Replication
low-resolutionimage (100×100)
high-resolutionimage (400×400)
EE465: Introduction to Digital Image Processing
22
low-resolutionimage (100×100)
Bilinear Interpolation
high-resolutionimage (400×400)
EE465: Introduction to Digital Image Processing
23
Bicubic Interpolation
low-resolutionimage (100×100)
high-resolutionimage (400×400)
EE465: Introduction to Digital Image Processing
24
Edge-Directed InterpolationEdge-Directed Interpolation(Li&Orchard’2000)(Li&Orchard’2000)
low-resolutionimage (100×100)
high-resolutionimage (400×400)
EE465: Introduction to Digital Image Processing
25
Image DemosaicingImage Demosaicing((Color-Filter-Array InterpolationColor-Filter-Array Interpolation))
Bayer Pattern
EE465: Introduction to Digital Image Processing
26
Image ExampleImage Example
Ad-hoc CFA Interpolation Advanced CFA Interpolation
EE465: Introduction to Digital Image Processing
27
Error ConcealmentError Concealment
damaged interpolated
EE465: Introduction to Digital Image Processing
28
Image InpaintingImage Inpainting
EE465: Introduction to Digital Image Processing
29
Geometric TransformationGeometric Transformation
Widely used in computer graphics to generate special effects
MATLAB functions: griddata, interp2, maketform, imtransform
EE465: Introduction to Digital Image Processing
30
Basic PrincipleBasic Principle• (x,y) (x’,y’) is a geometric
transformation• We are given pixel values at (x,y)
and want to interpolate the unknown values at (x’,y’)
• Usually (x’,y’) are not integers and therefore we can use linear interpolation to guess their valuesMATLAB implementation: z’=interp2(x,y,z,x’,y’,method);
EE465: Introduction to Digital Image Processing
31
RotationRotation
y
x
y
x
cossin
sincos
'
'
x
y
x’
y’
θ
EE465: Introduction to Digital Image Processing
32
MATLAB ExampleMATLAB Example
% original coordinates[x,y]=meshgrid(1:256,1:256);
z=imread('cameraman.tif');
% new coordinatesa=2;for i=1:256;for j=1:256;x1(i,j)=a*x(i,j);y1(i,j=y(i,j)/a;end;end
% Do the interpolation z1=interp2(x,y,z,x1,y1,'cubic');
EE465: Introduction to Digital Image Processing
33
Rotation ExampleRotation Example
θ=3o
EE465: Introduction to Digital Image Processing
34
ScaleScale
y
x
a
a
y
x
/10
0
'
'
a=1/2
EE465: Introduction to Digital Image Processing
35
Affine TransformAffine Transform
y
x
d
d
y
x
aa
aa
y
x
2221
1211
'
'
square parallelogram
EE465: Introduction to Digital Image Processing
36
Affine Transform ExampleAffine Transform Example
1
0
25.
15.
'
'
y
x
y
x
EE465: Introduction to Digital Image Processing
37
ShearShear
y
x
d
d
y
x
sy
x
1
01
'
'
square parallelogram
EE465: Introduction to Digital Image Processing
38
Shear ExampleShear Example
1
0
15.
01
'
'
y
x
y
x
EE465: Introduction to Digital Image Processing
39
Projective TransformProjective Transform
1'
87
321
yaxa
ayaxax
1'
87
654
yaxa
ayaxay
quadrilateralsquare
A B
CD
A’
B’
C’
D’
EE465: Introduction to Digital Image Processing
40
Projective Transform ExampleProjective Transform Example
[ 0 0; 1 0; 1 1; 0 1] [-4 2; -8 -3; -3 -5; 6 3]
EE465: Introduction to Digital Image Processing
41
Polar TransformPolar Transform
22 yxr
x
y1tan
EE465: Introduction to Digital Image Processing
42
Iris Image UnwrappingIris Image Unwrapping
r
EE465: Introduction to Digital Image Processing
43
Use Your ImaginationUse Your Imaginationr -> sqrt(r)
http://astronomy.swin.edu.au/~pbourke/projection/imagewarp/
EE465: Introduction to Digital Image Processing
44
Free Form DeformationFree Form Deformation
Seung-Yong Lee et al., “Image Metamorphosis Using Snakes and Free-Form Deformations,”SIGGRAPH’1985, Pages 439-448
EE465: Introduction to Digital Image Processing
45
Application into Image Application into Image MetamorphosisMetamorphosis
EE465: Introduction to Digital Image Processing
46
Summary of Image Summary of Image InterpolationInterpolation
• A fundamental tool in digital processing of images: bridging the continuous world and the discrete world
• Wide applications from consumer electronics to biomedical imaging
• Remains a hot topic after the IT bubbles break