image interpolation

EE465: Introduction to Digital Image Processing

1

Image InterpolationImage Interpolation• Introduction

– What is image interpolation?– Why do we need it?

• Interpolation Techniques– 1D zero-order, first-order, third-order– 2D zero-order, first-order, third-order– Directional interpolation*

• Interpolation Applications– Digital zooming (resolution enhancement)– Image inpainting (error concealment)– Geometric transformations


2

IntroductionIntroduction• What is image interpolation?

– An image f(x,y) tells us the intensity values at the integral lattice locations, i.e., when x and y are both integers

– Image interpolation refers to the “guess” of intensity values at missing locations, i.e., x and y can be arbitrary

– Note that it is just a guess (Note that all sensors have finite sampling distance)


3

Introduction (Con’t)Introduction (Con’t)• Why do we need image

interpolation?– We want BIG images

• When we see a video clip on a PC, we like to see it in the full screen mode

– We want GOOD images• If some block of an image gets damaged

during the transmission, we want to repair it

– We want COOL images• Manipulate images digitally can render fancy

artistic effects as we often see in movies


4

Scenario I: Resolution Scenario I: Resolution EnhancementEnhancement

Low-Res.

High-Res.


5

Scenario II: Image InpaintingScenario II: Image Inpainting

Non-damaged Damaged


6

Scenario III: Image WarpingScenario III: Image Warping


7






8

1D Zero-order (Replication)

n

f(n)

x

f(x)


9

1D First-order Interpolation (Linear)

n

f(n)

x

f(x)


10

Linear Interpolation FormulaLinear Interpolation Formula

a 1-a

f(n)

f(n+1)f(n+a)

f(n+a)=(1-a)f(n)+af(n+1), 0<a<1

Basic idea: the closer to a pixel, the higher weight is assigned

Note: when a=0.5, we simply have the average of two


11

Numerical ExamplesNumerical Examples

f(n)=[0,120,180,120,0]

f(x)=[0,60,120,150,180,150,120,60,0], x=n/2

f(x)=[0,20,40,60,80,100,120,130,140,150,160,170,180,…], x=n/6

Interpolate at 1/2-pixel

Interpolate at 1/3-pixel


12

n

x

f(n)

f(x)

Cubic spline fitting

1D Third-order Interpolation (Cubic)


13

From 1D to 2D

Just like separable 2D transform (filtering) that can be implemented by two sequential 1D transforms (filters) along row and column direction respectively, 2D interpolation can be decomposed into two sequential 1D interpolations.

The ordering does not matter (row-column = column-row)

Such separable implementation is not optimal but enjoys low computational complexity


14

Graphical InterpretationGraphical Interpretationof Interpolation at Half-pelof Interpolation at Half-pel

row column

f(m,n) g(m,n)


15

Numerical ExamplesNumerical Examplesa b

c d

a a b b

a a b b

c c d d

c c d d

zero-order

first-order

a (a+b)/2b(a+c)/2 (a+b+c+d)/4 (b+d)/2c (c+d)/2 d


16

Numerical Examples (Con’t)Numerical Examples (Con’t)

X(m,n)row m

row m+1

Col n Col n+1

X(m,n+1)

X(m+1,n+1)X(m+1,n)

Ya 1-a

b

1-b

Q: what is the interpolated value at Y?Ans.: (1-a)(1-b)X(m,n)+(1-a)bX(m+1,n)+a(1-b)X(m,n+1)+abX(m+1,n+1)


17

Bicubic Interpolation*Bicubic Interpolation*


18

Limitation with Limitation with bilinear/bicubicbilinear/bicubic

• Edge blurring• Jagged artifacts

X Z

Jagged artifacts

X

Z

Edge blurring


19

Directional Interpolation*Directional Interpolation*

Step 1: interpolate the missing pixels along the diagonal

black or white?

Step 2: interpolate the other half missing pixels

a b

c d

Since |a-c|=|b-d|x

x has equal probabilityof being black or white

a

b

c

dSince |a-c|>|b-d|

x=(b+d)/2=black

x


20






21

Pixel Replication

low-resolutionimage (100×100)

high-resolutionimage (400×400)


22


Bilinear Interpolation



23

Bicubic Interpolation




24

Edge-Directed InterpolationEdge-Directed Interpolation(Li&Orchard’2000)(Li&Orchard’2000)




25

Image DemosaicingImage Demosaicing((Color-Filter-Array InterpolationColor-Filter-Array Interpolation))

Bayer Pattern


26

Image ExampleImage Example

Ad-hoc CFA Interpolation Advanced CFA Interpolation


27

Error ConcealmentError Concealment

damaged interpolated


28

Image InpaintingImage Inpainting


29

Geometric TransformationGeometric Transformation

Widely used in computer graphics to generate special effects

MATLAB functions: griddata, interp2, maketform, imtransform


30

Basic PrincipleBasic Principle• (x,y) (x’,y’) is a geometric

transformation• We are given pixel values at (x,y)

and want to interpolate the unknown values at (x’,y’)

• Usually (x’,y’) are not integers and therefore we can use linear interpolation to guess their valuesMATLAB implementation: z’=interp2(x,y,z,x’,y’,method);


31

RotationRotation

y

x

y

x

cossin

sincos

'

'

x

y

x’

y’

θ


32

MATLAB ExampleMATLAB Example

% original coordinates[x,y]=meshgrid(1:256,1:256);

z=imread('cameraman.tif');

% new coordinatesa=2;for i=1:256;for j=1:256;x1(i,j)=a*x(i,j);y1(i,j=y(i,j)/a;end;end

% Do the interpolation z1=interp2(x,y,z,x1,y1,'cubic');


33

Rotation ExampleRotation Example

θ=3o


34

ScaleScale

y

x

a

a

y

x

/10

0

'

'

a=1/2


35

Affine TransformAffine Transform

y

x

d

d

y

x

aa

aa

y

x

2221

1211

'

'

square parallelogram


36

Affine Transform ExampleAffine Transform Example

1

0

25.

15.

'

'

y

x

y

x


37

ShearShear

y

x

d

d

y

x

sy

x

1

01

'

'

square parallelogram


38

Shear ExampleShear Example

1

0

15.

01

'

'

y

x

y

x


39

Projective TransformProjective Transform

1'

87

321

yaxa

ayaxax

1'

87

654

yaxa

ayaxay

quadrilateralsquare

A B

CD

A’

B’

C’

D’


40

Projective Transform ExampleProjective Transform Example

[ 0 0; 1 0; 1 1; 0 1] [-4 2; -8 -3; -3 -5; 6 3]


41

Polar TransformPolar Transform

22 yxr

x

y1tan


42

Iris Image UnwrappingIris Image Unwrapping

r


43

Use Your ImaginationUse Your Imaginationr -> sqrt(r)

http://astronomy.swin.edu.au/~pbourke/projection/imagewarp/


44

Free Form DeformationFree Form Deformation

Seung-Yong Lee et al., “Image Metamorphosis Using Snakes and Free-Form Deformations,”SIGGRAPH’1985, Pages 439-448


45

Application into Image Application into Image MetamorphosisMetamorphosis


46

Summary of Image Summary of Image InterpolationInterpolation

• A fundamental tool in digital processing of images: bridging the continuous world and the discrete world

• Wide applications from consumer electronics to biomedical imaging

• Remains a hot topic after the IT bubbles break

image interpolation

Documents

digital image processing1d

digital image processingfrom

image fx

image warpingee465

order interpolation

orderdirectional interpolation

interpolation techniques1d

integersimage interpolation