image analysis - lecture 1 - lth · general research image models repetition image analysis...

General Research Image models Repetition

Image Analysis - Lecture 1

Kalle Åström

21 Juli 2014

Kalle Åström Image Analysis - Lecture 1


Lecture 1

I Administrative thingsI What is image analysis?I Examples of image analysisI Image models


General Research Image models Repetition Image analysis Computer vision Perceptual problems

Information

Lectures: 15⇥ 2h, tue 8.15 (not week 2), thu 10.15 and fri10.15 (weeks 2 and 5)Assignments: 5 (compulsory)Exercises: 7⇥ 2h, wed 13.15 or thu 10.15Lab sessions: 4⇥ 2h, weeks 2, 3, 4 and 6Project: Next study period (optional)Credits: 7.5Pass on course (grade 3): Assignments okPass on course (grades 4 and 5): Assignments ok + Writtenexam (hemtenta) + Oral exam



The Course

F1 - Introduction, image modelsF2 - Linear algebra, algebra of images, Fourier transformF3 - Linear filters, convolutionF4 - Scale space theory, edge detectionF5 - Machine learning 1F6 - Multispectral ImagingF7 - Segmentation: FittingF8 - Machine learning 2F9 - ApplicationsF10 - TextureF11 - Segmentatoin: ClusteringF12 - Segmentation: Active contoursF13 - Statistical Image AnalysisF14 - SystemsF15 - Medical Image Analysis.



Image analysis

Image processing: Enhance the image (image -> image)Image analysis: Interpret the image (image -> interpretation)Computer vision: Mimic human vision, geometry, interpretationComputer graphics: Generate images from models



Computer vision

Computer vision - attempt to mimic human visual functionExamples:

I RecognitionI NavigationI ReconstructionI Scene understanding



Perceptual problems

Example 1:

What is true ?1. In the figure a = b.2. In the figure a > b.



Perceptual problems (ctd.)

Exemple 2:

1. This is an image of a vase2. This is an image of two faces.


General Research Image models Repetition Mathematical Imaging Group Related courses Research areas

Mathematical Imaging Group,Centre for mathematical sciences

IResearch projects: EU, VR, SSF, Industry

IMasters thesis projects

ISSBA

IIndustry research: NDC, Decuma, Ludesi, Gasoptics,Exini, Cellavision, Precise Biometrics, Anoto, Wespot,Cognimatics, Polar Rose, Nocturnal Vision



Related courses

I Multispectral imaging 6hpI Computer vision 7.5 hpI Medical Image Analysis 7.5 hpI Statistical Image Analysis 7.5 hpI Digital pictures – compression 9hpI Courses in Copenhagen and MalmÃu



Research areas

I Geometry and computer visionI Medical image analysisI Cognitive vision


General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation

Continuous model

An image can be seen as a function

f : ⌦ 7! R+ ,

where ⌦ = { (x , y) | a x b, c y d} ✓ R2 andR+ = {x 2 R | x � 0}. f (x , y) = intensity at point (x , y) =gray-level(f does not have to be continuous)0 L

min

f L

max

1[L

min

, L

max

] = gray-scale



Continuous model (ctd.)

Change to gray-scale [0, L] where 0=’black’ and L=’white’.



Discrete model

Discretise x , y , called sampling.Discretise f , called quantification.Sampling:Point grid in xy -plane.



Sampling

f (x , y) 7!

0

B@f0,0 . . . f0,N�1... f

j,k...

f

M�1,0 . . . f

M�1,N�1

1

CA



Quantification

Use G gray-levelsUsually G = 2m for some m.NMm bits are required for storing an imageEx: 512 · 512 · 8 ⇠ 262kB(256 gray-levels)M, N decreased ) Chess-patternm decreased ) False contours



Sampling

Given an image with continuous representation it isstraightforward to convert it into a discrete one by sampling.

Common model for image formation is smoothing followed bysampling



Interpolation

Given an image with discrete representation one can obtain acontinuous version by interpolation.

Problem: (Interpolation)Given f (i , j), i , j 2 Z2.”compute” f (x , y), x , y 2 R2



Re-sampling



Re-sampling (ctd.)

Problem: (Re-sampling)Given f (i , j), i , j 2 Z2.”Compute” f (x , y), x , y 2 R2

Discrete image -> Interpolation -> continuous image ->sampling -> New discrete image in different resolution

Used frequently on computers when displaying an image in adifferent size, thus needing a different resolution.



Nearest neighbour (pixel replication)

f (x , y) = f (i , j),

where (i , j) is the grid point closest to (x , y).

Called pixel replication.



Nearest neighbour

Pixel replication can be seen as interpolation with

f (x , y) =X

i,j

h(x � i , y � j)f (i , j),

where

Notice the similaraty to convolution.



Linear interpolation

In one dimension

f (x) = (x � i)f (i + 1) + (i + 1� x)f (i), i < x < i + 1

Called linear inperpolation.



Linear interpolation (ctd.)

Linear interpolation can bee seen as convolution

f (x , y) =X

i,j

h(x � i , y � j)f (i , j),

with a different interpolation function h:



Two dimensions

f (x , y) =(i + 1� x)(j + 1� y)f (i , j)+

+ (x � i)(j + 1� y)f (i + 1, j)+

+ (i + 1� x)(y � j)f (i , j + 1)+

+ (x � i)(y � j)f (i + 1, j + 1),

i < x < i + 1, j < y < j + 1

Called bilinear interpolation. Between grid points the intensityis

f (x , y) = ax + by + cxy + d ,

where a, b, c, d is determined by the gray-levels in the cornerpoints.



Bilinear interpolation

For two-dimensional signals (images) we can apply linearinterpolation, first in x-direction and then y -direction.



Cubic interpolation (Cubic spline)

Define a function k such that

k(x) =

8><

>:

a3x

3 + a2x

2 + a1x + a0 x 2 [0, 1]

b3x

3 + b2x

2 + b1x + b0 x 2 [1, 2]

0 x 2 [2,1)

andI

k symmetric around the originI

k(0) = 1, k(1) = k(2) = 0I

k and k

0 continuous at x = 1I

k

0(0) = k

0(2) = 0



Cubic spline function



Determination of a

i

and b

i

These conditions give

k(x) =

((a + 2)x3 � (a + 3)x2 + 1 x 2 [0, 1]

ax

3 � 5ax

2 + 8ax � 4a x 2 [1, 2]

where a is a free parameter.Common choice is a = �1.Interpolation is done using the convolution

f (x) =X

i

f (i)k(x � i)



Cubic interpolation for images

For images one first interpolates in x-direction and then iny -direction.



Sinc interpolation

Assume that f (x) is a band-limited signal.Sampling theorem:

f (x) =X

k

sinc(2⇡(x � k))f (k)

Sketch of proof: Fouriertransform F (!) is band limited. Thus itcan be written as a fourier series, where the coefficients aref (k). Inverse fouriertransform completes the proof.Drawback: sinc has unlimited support ) large filter ) timeconsuming.Solution: Cut sinc after the first or the first few oscilations )almost like cubic interpolation.



Sinc interpolation for images

For images one interpolates first in x-direction and then iny -direction.



Gauss interpolation

Interpolate with

f (x) =X

k

e

�(x�k)2/a

2f (k)

where a determines ’scale/resolution/blurriness’.



Scale selection

Gives a scale-space pyramid with the same image at differentscales by changing a. More about this later.

This is called Gaussian pyramid or scale space pyramid orscale space representation.



Digital Geometry

Let Z be the set of integers 0,±1,±2, . . . .

Grid: Z2,

· · · ·· · · ·· · · ·· · · ·

Grid point: (x , y)

Definition4-neigbourhood to (x , y):

N4(x , y) =

0

@· ⇥ ·⇥ (x , y) ⇥· ⇥ ·

1

A .



Neighbours, connectedness, paths

Definitionp and q are 4-neighbours if p 2 N4(q).

DefinitionA 4-path from p to q is a sequence

p = r0, r1, r2, . . . , r

n

= q ,

such that r

i

and r

i+1 are 4-neighbours.

DefinitionLet S ✓ Z2. S is 4-connected if for every p, q 2 S there is a4-path in S from p to q.There are efficient algorithms for dividing sets M ✓ Z2 inconnected components. (For example, see MATLAB’s bwlabel).



D- and 8-neighbourhoods

Similar definitions with other neighbourhood structuresDefinitionD-neighbourhood to (x , y):

N

D

(x , y) =

0

@⇥ · ⇥· (x , y) ·⇥ · ⇥

1

A .

Definition8-neighbourhood to (x , y):

N8(x , y) = N4(x , y) [ N

D

(x , y) =

0

@⇥ ⇥ ⇥⇥ (x , y) ⇥⇥ ⇥ ⇥

1

A .



Gray-level transformation

A simple method for image enhancementDefinitionLet f (x , y) be the intensity function of an image. A gray-level

transformation, T , is a function (of one variable)

g(x , y) = T (f (x , y))

s = T (r) ,

that changes from gray-level f to gray-level g. T usually fulfilsI

T (r) increasing in L

min

r L

max

,I 0 T (r) L.

In many examples we assume that L

min

= 0 och L

max

= L = 1.The requirements on T being increasing can be relaxed, e.g.with inversion.



Thresholding

Let

T (r) =

(0 r m

1 r > m,

for some 0 < m < 1.



Thresholding (ctd.)

i.e.f (x , y) m ) g(x , y) = 0 (black),

f (x , y) > m ) g(x , y) = 1 (white).

The result is an image with only two gray-levels, 0 and 1. This iscalled a binary image.

The operation is called thresholding.



Continuous images

I Let s = T (r) be a gray-scale transformation (r = T

�1(s))I Let p

r

(r) be the frequency function for the original image.I Let p

s

(s) be the frequency function for the resulting image.

It follows that Zs

0p

s

(t)dt =

Zr

0p

r

(t)dt .



Continuous images (ctd.)

Differentiate with respect to s

p

s

(s) = p

r

(r)dr

ds

(s = T (r)) .



Histogram equalization

Take T so that p

s

(s) = 1 (constant).Z

r

0p

r

(t)dt =

Zs

01dt = s ) s = T (r) =

Zr

0p

r

(t)dt

ords

dr

= p

r

(r)



Histogram equalization (ctd.)

This transformation is called histogram equalization.



Histogram equalization for digital images

p

r

(rk

) =n

k

n

,

whereI

n=number of pixelsI

n

k

=number of pixels with intensity r

k

i.e. a histogram.Histogram equalization is obtained by

s

k

= T (rk

) =kX

j=0

n

j

n

.

Note that s

k

does not have to be an allowed gray-scale )perfect equalization cannot be obtained.



Example OCR (Optical Character Recognition)

I Image of textI Image enhancement, filtering.I Segmentation

I ThresholdingI Connected components with digital metrics.

I Classification



Images show how a system for OCR (Optical CharacterRecognition) can be used in a mobile telephone.The binary image is interpreted into ascii characters.

Original image and rectified image.



Cut-out of OCR number after thresholding.



Masters thesis suggestion of the day: The automaticbook database

Create a system for taking inventory of your books by takingimages of them and analysing the images.Images - segmentation - OCR - Database - Search - what ismissing? - etc.



Repetition - Lecture 1

I What is image analysis?I Image models (continuous - discrete - sampling -

quantification, sampling and interpolation)I Digital geometry (4-, D-, 8- neighbours, paths, connected

components)I Gray-level transformations (thresholding, histogram

equalization)



Recommended reading

I Forsyth & Ponce: 1. Cameras.I Szeliski: 1. Introduction and 3.1 Point operators.



i.e. id est that is det vill sägae.g. exempli gratia for example till exempelcf. confer compare with (see) jämför, se


image analysis - lecture 1 - lth · general research image models repetition image analysis...

Documents