![Page 1: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/1.jpg)
Linear filters and edges
![Page 2: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/2.jpg)
Linear Filters General process:
Form new image whose pixels are a weighted sum of original pixel values, using the same set of weights at each point.
Properties Output is a linear function of
the input Output is a shift-invariant
function of the input (i.e. shift the input image two pixels to the left, the output is shifted two pixels to the left)
Example: smoothing by averaging form the average of pixels
in a neighborhood
Example: smoothing with a Gaussian form a weighted average of
pixels in a neighborhood
Example: finding a derivative form a weighted average of
pixels in a neighborhood
![Page 3: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/3.jpg)
Convolution
Represent these weights as an image, H
H is usually called the kernel
Operation is called convolution it’s associative
Result is:
Notice weird order of indices all examples can be put in
this form it’s a result of the
derivation expressing any shift-invariant linear operator as a convolution.
Rij H i u, j vFuvu,v
![Page 4: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/4.jpg)
c22
+ c22 f(i,j)
f(.)
o (i,j) =
f(.)
c12
+ c12 f(i-1,j)
f(.)
c13
+ c13 f(i-1,j+1) +
f(.)
c21
c21 f(i,j-1)
f(.)
c23
+ c23 f(i,j+1) +
f(.)
c31
c31 f(i+1,j-1)
f(.)
c32
+ c32 f(i+1,j)
f(.)
c33
+ c33 f(i+1,j+1)
f(.)
c11
c11 f(i-1,j-1)
Convolution
![Page 5: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/5.jpg)
Example: Smoothing by Averaging
![Page 6: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/6.jpg)
Smoothing with a Gaussian
Smoothing with an average actually doesn’t compare at all well with a de-focused lens Most obvious difference
is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square.
A Gaussian gives a good model of a fuzzy blob
![Page 7: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/7.jpg)
exp x2 y2
22
An Isotropic Gaussian The picture shows a
smoothing kernel proportional to
(which is a reasonable model of a circularly symmetric fuzzy blob)
![Page 8: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/8.jpg)
Smoothing with a Gaussian
![Page 9: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/9.jpg)
Differentiation and convolution
Recall
Now this is linear and shift invariant, so must be the result of a convolution.
We could approximate this as
(which is obviously a convolution; it’s not a very good way to do things, as we shall see)
fx
lim 0
f x , y
f x,y
fx
f xn1,y f xn , y
x
![Page 10: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/10.jpg)
Finite differences
![Page 11: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/11.jpg)
Noise
Simplest noise model independent stationary
additive Gaussian noise
the noise value at each pixel is given by an independent draw from the same normal probability distribution
Issues this model allows noise
values that could be greater than maximum camera output or less than zero
for small standard deviations, this isn’t too much of a problem - it’s a fairly good model
independence may not be justified (e.g. damage to lens)
may not be stationary (e.g. thermal gradients in the ccd)
![Page 12: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/12.jpg)
sigma=1
![Page 13: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/13.jpg)
sigma=16
![Page 14: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/14.jpg)
Finite differences and noise
Finite difference filters respond strongly to noise obvious reason: image
noise results in pixels that look very different from their neighbors
Generally, the larger the noise the stronger the response
What is to be done? intuitively, most pixels in
images look quite a lot like their neighbors
this is true even at an edge; along the edge they’re similar, across the edge they’re not
suggests that smoothing the image should help, by forcing pixels different to their neighbors (=noise pixels?) to look more like neighbors
![Page 15: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/15.jpg)
Finite differences responding to noise
Increasing noise ->(this is zero mean additive gaussian noise)
![Page 16: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/16.jpg)
The response of a linear filter to noise Do only stationary
independent additive Gaussian noise with zero mean (non-zero mean is easily dealt with)
Mean: output is a weighted sum
of inputs so we want mean of a
weighted sum of zero mean normal random variables
must be zero
Variance: recall
variance of a sum of random variables is sum of their variances
variance of constant times random variable is constant^2 times variance
then if is noise variance and kernel is K, variance of response is
2 Ku,v2
u,v
![Page 17: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/17.jpg)
Filter responses are correlated
over scales similar to the scale of the filter Filtered noise is sometimes useful
looks like some natural textures, can be used to simulate fire, etc.
![Page 18: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/18.jpg)
![Page 19: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/19.jpg)
![Page 20: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/20.jpg)
![Page 21: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/21.jpg)
Smoothing reduces noise
Generally expect pixels to “be like” their neighbors surfaces turn slowly relatively few reflectance
changes
Generally expect noise processes to be independent from pixel to pixel
Implies that smoothing suppresses noise, for appropriate noise models
Scale the parameter in the
symmetric Gaussian as this parameter goes up,
more pixels are involved in the average
and the image gets more blurred
and noise is more effectively suppressed
![Page 22: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/22.jpg)
The effects of smoothing Each row shows smoothingwith gaussians of differentwidth; each column showsdifferent realizations of an image of gaussian noise.
![Page 23: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/23.jpg)
Some other useful filtering techniques
Median filter Anisotropic diffusion
![Page 24: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/24.jpg)
Median filters : principle
non-linear filter
method :
1. rank-order neighborhood intensities 2. take middle value
no new gray levels emerge...
![Page 25: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/25.jpg)
Median filters : odd-man-out
advantage of this type of filter is its “odd-man-out” effect
e.g.
1,1,1,7,1,1,1,1
?,1,1,1.1,1,1,?
![Page 26: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/26.jpg)
Median filters : example
filters have width 5 :
![Page 27: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/27.jpg)
Median filters : analysis
median completely discards the spike,linear filter always responds to all aspects
median filter preserves discontinuities,linear filter produces rounding-off effects
DON’T become all too optimistic
![Page 28: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/28.jpg)
Median filter : images
3 x 3 median filter :
sharpens edges, destroys edge cusps and protrusions
![Page 29: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/29.jpg)
Median filters : Gauss revisitedComparison with Gaussian :
e.g. upper lip smoother, eye better preserved
![Page 30: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/30.jpg)
Example of median
10 times 3 X 3 median
patchy effectimportant details lost (e.g. ear-ring)
![Page 31: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/31.jpg)
Gradients and edges
Points of sharp change in an image are interesting: change in reflectance change in object change in illumination noise
Sometimes called edge points
General strategy determine image
gradient
now mark points where gradient magnitude is particularly large wrt neighbors (ideally, curves of such points).
![Page 32: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/32.jpg)
There are three major issues: 1) The gradient magnitude at different scales is different; which should we choose? 2) The gradient magnitude is large along thick trail; how do we identify the significant points? 3) How do we link the relevant points up into curves?
![Page 33: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/33.jpg)
Smoothing and Differentiation
Issue: noise smooth before differentiation two convolutions to smooth, then differentiate? actually, no - we can use a derivative of Gaussian filter
because differentiation is convolution, and convolution is associative
![Page 34: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/34.jpg)
The scale of the smoothing filter affects derivative estimates, and alsothe semantics of the edges recovered.
1 pixel 3 pixels 7 pixels
![Page 35: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/35.jpg)
We wish to mark points along the curve where the magnitude is biggest.We can do this by looking for a maximum along a slice normal to the curve(non-maximum suppression). These points should form a curve. There arethen two algorithmic issues: at which point is the maximum, and where is thenext one?
![Page 36: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/36.jpg)
Non-maximumsuppression
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
![Page 37: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/37.jpg)
Predictingthe nextedge point
Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).
![Page 38: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/38.jpg)
Remaining issues
Check that maximum value of gradient value is sufficiently large drop-outs? use hysteresis
use a high threshold to start edge curves and a low threshold to continue them.
![Page 39: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/39.jpg)
Notice
Something nasty is happening at corners Scale affects contrast Edges are not bounding contours
![Page 40: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/40.jpg)
![Page 41: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/41.jpg)
fine scalehigh threshold
![Page 42: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/42.jpg)
coarse scale,high threshold
![Page 43: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/43.jpg)
coarsescalelowthreshold
![Page 44: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/44.jpg)
Filters are templates
Applying a filter at some point can be seen as taking a dot-product between the image and some vector
Filtering the image is a set of dot products
Insight filters look like the effects
they are intended to find filters find effects they
look like
![Page 45: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/45.jpg)
Normalized correlation Think of filters of a dot
product now measure the angle i.e normalized correlation
output is filter output, divided by root sum of squares of values over which filter lies
Tricks: ensure that filter has a
zero response to a constant region (helps reduce response to irrelevant background)
subtract image average when computing the normalizing constant (i.e. subtract the image mean in the neighborhood)
absolute value deals with contrast reversal
![Page 46: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/46.jpg)
Positive responses
Zero mean image, -1:1 scale Zero mean image, -max:max scale
![Page 47: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/47.jpg)
Positive responses
Zero mean image, -1:1 scale Zero mean image, -max:max scale
![Page 48: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/48.jpg)
Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE
![Page 49: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/49.jpg)
The Laplacian of Gaussian
Another way to detect an extremal first derivative is to look for a zero second derivative
Appropriate 2D analogy is rotation invariant the Laplacian
Bad idea to apply a Laplacian without smoothing smooth with Gaussian, apply
Laplacian this is the same as filtering
with a Laplacian of Gaussian filter
Now mark the zero points where there is a sufficiently large derivative, and enough contrast
![Page 50: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/50.jpg)
sigma=2
sigma=4
contrast=1 contrast=4LOG zero crossings
![Page 51: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/51.jpg)
We still have unfortunate behaviorat corners
![Page 52: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/52.jpg)
Orientation representations
The gradient magnitude is affected by illumination changes but it’s direction isn’t
We can describe image patches by the swing of the gradient orientation
Important types: constant window
small gradient mags edge window
few large gradient mags in one direction
flow window many large gradient
mags in one direction corner window
large gradient mags that swing
![Page 53: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/53.jpg)
Representing Windows
Types constant
small eigenvalues Edge
one medium, one small Flow
one large, one small corner
two large eigenvalues
H I I Twindow
![Page 54: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/54.jpg)
![Page 55: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/55.jpg)
![Page 56: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/56.jpg)
![Page 57: Linear filters and edges. Linear Filters General process: Form new image whose pixels are a weighted sum of original pixel values, using the same set](https://reader034.vdocuments.mx/reader034/viewer/2022050714/56649d565503460f94a34938/html5/thumbnails/57.jpg)