Lecture 2: 2D edge detectionand moving beyond edges

Dr. Richard E. Turner (ret26@cam.ac.uk)

May 2, 2014

Recap of last lecture (big picture)

• goal: match query image to a stored library of reference images

• cannot use raw pixels:

– too many– too unstable (lighting, shadow, occlusion, scale change, perspective

change...)

• three step plan:

1. identify interest points (easy to identify structures): visual words2. extract local feature descriptors (characterises interest points): word

definitions3. match images based by comparing sets of feature descriptors

Recap of last lecture (edge detection)

Recap of last lecture (edge detection)

image

Recap of last lecture (edge detection)

image Gaussianblur

Recap of last lecture (edge detection)

image Gaussianblur

differentiate

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

o frequency

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

o o frequency

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

o o

X frequency

=

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

o o

X frequency

=

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

linear operations are commutative and associative

o o

X frequency

=

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

linear operations are commutative and associative

o o

image Gaussianblur

differentiate

X frequency

=

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

linear operations are commutative and associative

o o

image differentiate

X frequency

=

Gaussianblur

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

linear operations are commutative and associative

o o

image blur with derivative of Gaussian

X frequency

=

Recap of last lecture (edge detection)

image Gaussianblur

differentiate locateextrema

threshold

convolutionlowpass filter

convolutionhighpass filter

bandpass filter

linear operations are commutative and associative

o o

image blur with derivative of Gaussian

X frequency

=

Today

• Is there a correct level of blur for edge detection?

• Edge detection in 2D

– Canny edge detection– Marr Hildreth edge detection

• Moving beyond edges

– why edges aren’t sufficient for computer vision

Discrete approximation of the Gaussian filter

1 2 3 4 5 6 7 8 9 ... N

N = number of pixels in 1D image

discreteapprox

continuousGaussian

=⇒ Trade-off between comp. complexity and high-frequency artifacts

Truncation of the Gaussian filter

1 2 3 4 5 6 7 8 9 ...K

N = number of pixels in 1D image

discreteapprox

continuousGaussian

peak height

peak height1000

K = truncated approximation

=⇒ Trade-off between comp. complexity and high-frequency artifacts

Truncation of the Gaussian filter

1 2 3 4 5 6 7 8 9 ...K

N = number of pixels in 1D image

discreteapprox

continuousGaussian

peak height

peak height1000

K = truncated approximation

=⇒ Trade-off between computational complexity and high-frequency artifacts

Convolution

imag

efil

ter aa b c

d e fg h i

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i

filtered image

Is this the right thing to do?

imag

efil

ter aa b c

d e fg h i a b c

d e fg h i

filtered image

Is this the right thing to do?

imag

efil

ter aa b c

d e fg h i a b c

d e fg h i

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i i h g

f e lc b a

flip

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i i h g

f e lc b a

flip

filtered image

convolution = correlation when the filter is symmetric e.g. isotropic Gaussian

Convolution

imag

efil

ter aa b c

d e fg h i i h g

f e lc b a

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i

i h gf e lc b a

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i

i h gf e lc b a

filtered image

Convolution

imag

efil

ter aa b c

d e fg h i

i h gf e lc b a

filtered image

What’s the computational cost of this naïve approach?

imag

efil

ter aa b c

d e fg h i i h g

f e lc b a

filtered image

K by K matrix N by M pixel image

What’s the computational cost of this naïve approach?

imag

efil

ter aa b c

d e fg h i i h g

f e lc b a

filtered image

K by K matrix N by M pixel image

NB. K = 2n+1 in your notes

computational cost = NMK2

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

image

filter aa b c

d e fg h i

c'b'a'

a'b'c'

a' b' c'

aa b cb d ec e f

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

Can we speed this up when the filter is separable?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

What is the new computational cost?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

c'b'a'

N by M pixel image

K dimensionalvectors

What is the new computational cost?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

c'b'a'

new computational cost is = 2NMK

N by M pixel image

K dimensionalvectors

What is the new computational cost?

imag

efil

ter aa b c

d e fg h i

c' b' a'

a'b'c'

a' b' c'

aa b cb d ec e f filtered

image

c'b'a'

old computational cost was = NMK

new computational cost is = 2NMK2

N by M pixel image

K dimensionalvectors

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Which way do you think the edge is moving?

a

b

c

de

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Aperture problem

Problem with edge-features

=⇒ Edges can only tell us about the component of motionperpedicular to the edge

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?

What about corners?