image sampling and quantisation - unibas.ch · image sampling and quantisation introduction to...

Image Samplingand Quantisation

Introduction to Signaland Image Processing

Prof. Dr. Philippe Cattin

MIAC, University of Basel

March 29th, 2016

March 29th, 2016Introduction to Signal and Image Processing

1 of 46 22.02.2016 09:17

Contents

3

4

5

8

9

10

11

12

13

14

15

16

17

18

20

21

22

23

24

25

26

27

28

Ph. Cattin: Image Sampling and Quantisation

Contents

1 Motivation

Introduction and Motivation

Sampling Example

Quantisation Example

2 Sampling

2.1 Tessellation

Tessellation

Tessellation Examples by M.C. Escher (1)

Tessellation Examples by M.C. Escher (2)

Tessellation Basics

Tessellation Claim

How Many Tessellations Exist with RegularPolygons?

Combinatorial Analysis

All Semi-Regular Tessellations

All Regular Tessellations

Tessellation Rules

Advantages of Square Tessellation

2.2 A Sampling Model

A Sampling Model

The Neighbourhood Function

Fourier Transform of the NeighbourhoodFunction

Filtering with the Neighbourhood Function

Sampling of a Continuous 1D Function

Sampling of a Continuous 1D Function (2)

Sampling of a Discrete 1D Function

An Alternative Reasoning for Periodicity in theDFT

Sampling of Two-Dimensional FunctionsMarch 29th, 2016Introduction to Signal and Image Processing

2 of 46 22.02.2016 09:17

29

30

31

32

33

34

35

36

37

39

40

41

42

43

(Images)

Summary Sampling Theorem

Aliasing Example 1

Aliasing Example 2

Aliasing Example 3

Remark on the Discrete Fourier Transform

Linear, Shift-Invariant Operators

Linear, Shift-Invariant Operators (2)

Liner, Shift-Invariant Operators (3)

Liner, Shift-Invariant Operators (4)

3 Quantisation

Quantisation

Lloyd-Max Quantisation


Quantisation Example (2)



3 of 46 22.02.2016 09:17

Motivation


(3)Introduction and Motivation

In order for computers to process an image, this imagehas to be described as a series of numbers, each of finiteprecision

This calls for two kinds of discretisation:

Sampling, and

Quantisation

By sampling is meant that the brightness information is onlystored at a discrete number of locations. Quantisation indicatesthe discretisation of the brightness levels at these positions.

4 of 46 22.02.2016 09:17


Motivation

(4)


Sampling Example

Sampling is the process of measuring the brightnessinformation only at a discrete number of locations

Fig 4.1: Hight profile of Switzerland Fig 4.2: Sampled hight profile

5 of 46 22.02.2016 09:17


Motivation

(5)



Quantisation is the process of discretising the brightnessat a finite number of positions

Height map with grey values with grey values

with grey values with grey values

Fig 4.3:

6 of 46 22.02.2016 09:17

Sampling

Tessellation


(8)Tessellation

Definition

Tessellations are patterns that cover a plane withrepeating figures so there is no overlapping or emptyspaces

Sampling is best performed following a regular tessellation ofthe image:

Brightness is integrated over cells of same size1.

Cells should cover the whole image2.

These cells are usually referred to as picture elements or pixels.

7 of 46 22.02.2016 09:17


Tessellation

(9)


Tessellation Examples byM.C. Escher (1)

Fig 4.4: Sample Escher images

8 of 46 22.02.2016 09:17


Tessellation

(10)


Tessellation Examples byM.C. Escher (2)

Fig 4.5: Sample Escher images

9 of 46 22.02.2016 09:17


Tessellation

(11)


Tessellation Basics

Three types of tessellations with polygons exist

regular tessellations (using the same regular polygon)1.

semi-regular tessellations (using various regular

polygons)

2.

hyperbolic tessellations (they use non-regular polygons)3.

They are formed by translating, rotating, and reflecting

polygons

Fig 4.6: regular Fig 4.7: semi-regular Fig 4.8: hyperbolic

10 of 46 22.02.2016 09:17


Tessellation

(12)


Tessellation Claim

There exist only 11 possible tessellations with regularpolygons that can cover the entire image

11 of 46 22.02.2016 09:17


Tessellation

(13)


How Many TessellationsExist with Regular Polygons?

Observation 1:

Since the regular polygons in atessellation must fill the plane ateach vertex, the interior angle mustbe an exact divisor of

Observation 2:

A regular -gon has an internal angle

of degrees Fig 4.9:

Of the regular polygons, only triangles ( ), squares ( ),pentagons ( ), hexagons ( ), octagons ( ), decagons (

) and dodecagons ( ) can be used for tiling around acommon vertex - again because of the angle value

12 of 46 22.02.2016 09:17


Tessellation

(14)


Combinatorial Analysis

A combinatorial analysis of these base polygons produces thefollowing 14 solutions

RegularTessellations

4.4.46.6.6

Semi-regularTessellations

3.3.43.6.33.4.63.3.34.8.83.124.6.1

Semi-regularTessellations thatcan not be extendedinfinitely 3.4.45.5.1

Fig 4.10: Tessellations

13 of 46 22.02.2016 09:17


Tessellation

(15)


All Semi-RegularTessellations

Eight semi-regular tessellations exist

Snub hexagonal Trihexagonal Prismatic trisquare Snub square

Small

rhombitrihexagonalTruncated square

Truncated

hexagonal

Great

rhombitrihexagonal

Fig 4.11:

14 of 46 22.02.2016 09:17


Tessellation

(16)


All Regular Tessellations

But only three regular tessellations exist

Triangular tiling Square tiling Hexagonal tiling

Fig 4.12:

15 of 46 22.02.2016 09:17


Tessellation

(17)


Tessellation Rules

For practical applications in computer vision the tessellation hasto adhere to the following rules

The tessellation must tile an infinite area with no gaps or

overlapping

Each vertex must look the same

The tiles must all be the same regular polygon

This leaves us with the following three regular tessellations

RegularTessellations 4.4.4

6.6.6Although the hexagonal tessellation offers some substantialadvantages (e.g. no ambiguities in defining connectedness,closer spatial organisation as found in mammalian retinas), thesquare tessellation is the most commonly used.

16 of 46 22.02.2016 09:17


Tessellation

(18)


Advantages of SquareTessellation

They directly support operations in the Cartesian coordinate

frame

Most algorithms (FFT, Image pyramids) are based on square

tessellations

The resolution is often a power of 2: e.g. 16x16, 32x32,

..., 256x256, 512x512

17 of 46 22.02.2016 09:17

A Sampling Model


(20)A Sampling Model

As we have seen,

The intensity value attributed to a pixel corresponds to the

integration of the incoming irradiance over a cell of the

tessellation

The cells are only located at discrete locations

The sampling process can thus be modeled in a 2-step scheme:

Integrate brightness over regions of the pixel size,1.

Read out values only at the pixel positions.2.

18 of 46 22.02.2016 09:17


A Sampling Model

(21)


The NeighbourhoodFunction

First a neighbourhood function has to be

defined, that is 1 inside a region with the shape of apixel/cell and 0 outside.

Integrating the incoming intensity over such a

region then yields

(4.1)

rewriting this expression as

(4.2)

we recognise it as the convolution of with

which can also be written as

. Since is symmetric we can

equally well write .

Fig 4.13:

Neighbourhood

function

for square

pixels

19 of 46 22.02.2016 09:17


A Sampling Model

(22)


Fourier Transform of theNeighbourhood Function

To gain a deeper understand of the sampling model we need itsFourier Transform :

(4.3)

Fig 4.14: , the

Fourier Transform of

the neighbourhood

function (notice

the negative values)

Because is real and even its Fourier Transform is too

→ the neighbourhood filter will not change the phase but only

their amplitude.

Since becomes negative for some some

frequencies undergo a complete phase reversal (shift over -

see next slide).

20 of 46 22.02.2016 09:17


A Sampling Model

(23)


Filtering with theNeighbourhood Function

As the Fourier Transform of the neighbourhood function

has negative amplitudes for some frequencies, complete phasereversals can be observed for higher frequencies:

Fig 4.15: Star pattern that increases its

frequency towards the centre

Fig 4.16: Complete phase reversals

occur at higher frequencies

21 of 46 22.02.2016 09:17


A Sampling Model

(24)


Sampling of a Continuous1D Function

As the second step after filtering with the neighbourhoodfunction we have to select values only at discrete pixel

positions. This is modelled as a multiplication with a 1D or 2Dpattern (train) of Dirac impulses at these discrete positions.

Consider the real neighbourhood function

filtered

Suppose its Fourier Transform is band

limited and thus vanishes outside the

interval

To obtain a sampled version of simply

involves multiplying it by a sampling

function , which consists of a train of

Dirac impulses apart

Its Fourier Transform is also a train

of Dirac impulses with a distance inversely

proportional to , namely apart

By the convolution theorem multiplication

in the image domain is equivalent to

convolution in the frequency domain

The transform is periodic, with period

, and the individual repetitions of

can overlap → aliasing!!!

The centre of the overlap occurs at

To avoid these problems, the sampling

interval has to be selected so that

, or

(4.4)

22 of 46 22.02.2016 09:17


Once the individual are separated a

multiplication with the window function

yields a completely isolated

The inverse Fourier Transform then yields

the original continuous function

Complete recovery of a band-limited

function that satisfies the above

inequality is known as the Whittaker-

Shannon Sampling Theorem

23 of 46 22.02.2016 09:17


A Sampling Model

(25)


Sampling of a Continuous1D Function (2)

All the frequency domain information of a band-limited

function is contained in the interval

If the Whittaker-Shannon Sampling Theorem or Nyquist

Sampling Theorem

(4.5)

is not satisfied, the transform in this interval is corrupted by

contributions from adjacent periods. This phenomenon is

frequently referred to as aliasing.

24 of 46 22.02.2016 09:17


A Sampling Model

(26)


Sampling of a Discrete 1DFunction

The preceding example applies to functions of unlimited durationin the spatial domain. For practical examples only functionssampled over a finite region are of interest. This situation isshown graphically below

Consider a real neighourhood-

function-filtered function

Suppose its Fourier Transform is

band limited and thus vanishes

outside the interval

The sampling function fulfils

the Whittaker-Shannon Theorem

As the Whittaker-Shannon

Sampling Theorem (aka Nyquist

Criterion) is fulfilled, the are

well separated and no aliasing is

present

The Sampling Window

and its Fourier

Transform

has Frequency components

that extend to infinity

Because has frequency

components that extend to infinity,

the convolution of these functions

introduces a distortion in the

frequency domain representation

of a function that has been

sampled and limited to a finite

region by

25 of 46 22.02.2016 09:17


These considerations lead to the important conclusion that

No function of finite duration can be band limited

Conversely,

A function that is band limited must extend from to in the spatial domain

These important practical results establish fundamentallimitations to the treatment of digital functions.

26 of 46 22.02.2016 09:17


A Sampling Model

(27)


An Alternative Reasoningfor Periodicity in the DFT

So far, all the results in the Fourier domain have been of acontinuous nature. To obtain a discrete Fourier Transform simplyrequires to sample it with a train of Dirac impulses that are units apart.

Consider the signals and

as the results of the operation

sequence on the previous slide

To sample we multiply it with

a train of Dirac impulses that

are units apart

The inverse Fourier Transform of

yields , an other train of

Dirac impulses with inversely

spaced pulses

The graph shows the

result of sampling

As the equivalent of a

multiplication in the Fourier

domain is a convolution in the

spatial domain, it yields a periodic

function, with period

If samples of and are taken and the spacings

between samples are selected so that a period in each domain iscovered by uniformly spaced samples, then in thespatial domain and in the frequency domain. The

latter equation is based on the periodic property of the FourierTransform of a sampled function, with period , as shown

earlier. The Sampling Theorem for discrete signals can thus beformulated as

27 of 46 22.02.2016 09:17


(4.6)

28 of 46 22.02.2016 09:17


A Sampling Model

(28)


Sampling ofTwo-Dimensional Functions(Images)

The preceding sampling concepts (after some

modifications in notation) are directly applicable to

2D functions

The sampling process for these functions can be

formulated making use of a 2D train of Dirac

impulses

For a function , where and are

continuous, a sampled function is obtained by

forming the product . The equivalent

operation in the Frequency domain is the

convolution of and , where is

a train of Dirac impulses with separation and

. If is band limited it might look like

shown on the right

Let and represent the widths in and

direction that completely enclose the band-limited

function

No aliasing is present if and

The 2D sampling theorem can thus be formulated as

(4.7)

and

(4.8)

A periodicity analysis similar to the discrete 1D case

shown previously would yield a 2D Sampling Theorem

of

(4.9)

and

29 of 46 22.02.2016 09:17


(4.10)

30 of 46 22.02.2016 09:17


A Sampling Model

(29)


Summary SamplingTheorem

The One-Dimensional Sampling Theorem states that

If the Fourier Transform of a function is zero for all

Frequencies beyond , i.e. the Fourier Transform isband-limited, then the continuous function can be

completely reconstructed as long as .

The Two-Dimensional Sampling Theorem states that

If the Fourier Transform of a function is zero for

all Frequencies beyond , i.e. the Fourier Transform isband-limited, then the continuous function can be

completely reconstructed as long as and

.

31 of 46 22.02.2016 09:17


A Sampling Model

(30)


Aliasing Example 1

The input image containsregions with clearly differentfrequency content. Going fromthe centre to boundary, thefrequency increases. It can beseen that once the Nyquist rateis higher than the actualsampling, aliasing occurs.

(a) the 256x256 sample pattern

(b) the sinc function for a sampling

rate of (grey is zero,

brighter is positive, and darker is

negative)

(c) the original pattern is sampled

with

(d) the reconstructed pattern. In

regions where the Nyquist rate is

higher strong aliasing artefacts

are present

(a) Original pattern (b) Sinc size 5

(c) Sampled

pattern

(d) Reconstruction

Fig 4.17 Aliasing example

32 of 46 22.02.2016 09:17


A Sampling Model

(31)


Aliasing Example 2

This example shows thereconstruction of the rollingpattern for a sampling rate (

) that is well above theNyquist rate.

(a) the 128x128 sample rolling

pattern


rate of . The grey

background is zero, brighter is

positive, and darker is negative


with

(d) the reconstructed rolling

pattern. The reconstruction is

perfect (except for boundary

effects)

(a) Original pattern (b) Sinc of size 5

(c) Sampled

pattern

(d) Reconstruction

Fig 4.18 Aliasing example 2

33 of 46 22.02.2016 09:17


A Sampling Model

(32)


Aliasing Example 3

In this example the samplingrate ( ) is below theNyquist rate.

(a) the 128x128 sample rolling

pattern


rate of . The grey

background is zero, brighter is

positive, and darker is negative


with

(d) the reconstructed rolling

pattern is no longer valid. It is

interesting that not only the

frequency changed, but even the

orientation of the pattern.

(a) Original pattern (b) Sinc size 15

(c) Sampled

pattern

(d) Reconstruction

Fig 4.19 Aliasing example 3

34 of 46 22.02.2016 09:17


A Sampling Model

(33)


Remark on the DiscreteFourier Transform

As already noted,

Sampling in one domain impliesperiodicity in the other

If both domains are discretised and thusshould both the original image and itsFourier Transform be interpreted as periodsof periodic signals.

The discrete Fourier Transform istherefore not the Fourier Transformof the image as such, but rather ofthe periodic signal created byrepeating the image data bothhorizontally and vertically

Periodically repeated image

Flipped images

35 of 46 22.02.2016 09:17


A Sampling Model

(34)


Linear, Shift-InvariantOperators

Convolution theory is not only important in image acquisition butplays an important role at several other occasions. To fullybenefit from the convolution theorem a little bit morebackground theory is required. In fact, it will be explained that

Every linear, shift-invariant operation can be expressedas a convolution and vice versa.

Definition:

Consider a 2D system thatproduces output and

when given inputs

and respectively.

The system is called linear if

the output

is

produced when the input is

The system is called shift-invariant if

the output is

produced when the input is

Fig 4.20: Linear system

Fig 4.21: Shift-invariant system

36 of 46 22.02.2016 09:17


A Sampling Model

(35)


Linear, Shift-InvariantOperators (2)

Suppose a process, e.g. camera with lens system, can bemodeled as a linear, shift-invariant operation . As we haveseen, any image can be considered as a sum of point sources(Dirac impulses).

The output of for a singlepoint source is called Pointspread function (PSF) of which we denote as .

Fig 4.22: Point spread function

Knowledge of the PSF can be used to determine theoutput for

Assuming shift-invariance implies that the output to such a Diracpulse is always the same irrespective of its position. In terms ofimage acquisition, we assume that the light comming from apoint source will be distributed over the image following a fixedspatial pattern. The projection of such a point will thereforealways be blurred in the same way independent of its position inthe image.

37 of 46 22.02.2016 09:17


A Sampling Model

(36)


Liner, Shift-InvariantOperators (3)

Let us consider an input picture . It can be written as a

linear combination of point sources

(4.11)

For the linear and shift-invariant operation we obtain

(4.12)

The linear, shift-invariant operation has led to a convolutionoperation. This is true in general and every LSI operation can bewritten as a convolution and vice versa.

A simple variable substitution shows that the above expressioncan also be written as

(4.13)

so that

(4.14)

i.e. convolution is commutative (convolution is also associative).

38 of 46 22.02.2016 09:17


39 of 46 22.02.2016 09:17


A Sampling Model

(37)


Liner, Shift-InvariantOperators (4)

Suppose we would like to process an image by first convolvingwith , followed by a convolution with , thus

(4.15)

the global operation can therefore be interpreted as applying asingle (generally larger) filter .

The reverse analysis might be useful too, i.e. if a filter(separable) can be decomposed as a convolution of two simplerfilter efficiency can be increased by applying the smaller filterssequentially.

Example

The Figures on the right show a 2DGauss kernel and a 1D Gausskernel of size and respectively.

It can be easily shown numericallythat the kernel can be separatedinto two 1D kernels and

thus

(4.16)

Convolving the image sequentiallywith the 1D kernels is computationallymore efficient than convolving theentire image with the 2D kernel.

Fig 4.23: 2D Gauss kernel

Fig 4.24: 1D Gauss kernel

40 of 46 22.02.2016 09:17

Quantisation


(39)Quantisation

The subjective image quality depends on (1)the number of samples and (2) thenumber of grey-values . Figure 4.26 showsthis relation.

The key point of interest is, thatisopreference curves tend to become morevertical as the detail in the image increases→ images with large amount of detail requirefewer grey levels.

Fig 4.25: (a) Low detail face image, (b) Cameraman

with mid detail, and (c) crowd with high detail content

Fig 4.26: Isopreference

curves for the three

sample images

41 of 46 22.02.2016 09:17


Quantisation

(40)


Lloyd-Max Quantisation

In the Introduction of this Lecture wehave already shortly explained theeffect of using more or lessquantisation levels. This part isconcerned with the optimal placementof these quantisation levels

Suppose we create intervals in therange of possible intensities, definedby the decision levels

.

We therefore assign to all intensitiesin the interval the new grey

level . The mean-square

quantisation error between the inputand output of the quantiser for a givenchoice of boundaries and outputlevels is thus

Fig 4.27: Principle of the

Lloyd-Max quantiser

(4.17)

where is the probability density function for the input

sample value.

For a given number of output levels, we would like todetermine the output levels and interval boundaries that

minimise . The partial derivatives of with respect to and

must thus vanish:

(4.18)

42 of 46 22.02.2016 09:17


For not equal to zero we obtain the Lloyd-Max Quantiser

Equations

(4.19)

We see that

the decision levels are located halfway between the output

levels

whilst each is the centroid of the portion of between

and

If the sample values occur equally frequently, the optimalquantised will spread the values and uniformly, and the

Lloyd-Max Quantiser Equations can be simplified to

(4.20)

As can be seen from the following examples, improvement can bedisputed. The main problem is, that Lloyd-Max quantisation doesnot take local image structure or interpretation into account.

43 of 46 22.02.2016 09:17


Quantisation

(41)



Original image with 256

grey values

32 equally spaced grey

values

32 Lloyd-max quantised

grey values

Fig 4.28: Quantisation example with 32 grey values

44 of 46 22.02.2016 09:17


Quantisation

(42)




grey values


values


grey values


45 of 46 22.02.2016 09:17


Quantisation

(43)




grey values


values


grey values


46 of 46 22.02.2016 09:17

image sampling and quantisation - unibas.ch · image sampling and quantisation introduction to...

Documents