selected topics in computer engineering (0907779)
TRANSCRIPT
Selected Topics in Computer
Engineering (0907779)Engineering (0907779)
Image Enhancement –Part III
Spatial Filtering
Dr. Iyad Jafar
Spatial Filtering
Outline
� Introduction
� Mechanics of Spatial Filtering � Mechanics of Spatial Filtering
� Correlation and Convolution
� Linear Spatial Filtering
�Spatial filters for Smoothing
�Spatial filters for Sharpening�Spatial filters for Sharpening
� Nonlinear Spatial Filtering
� Combining Spatial Enhancement Techniques 2
Background
� Filtering is borrowed from the frequency domainprocessing and refers to the process of passing orrejecting certain frequency components
�Highpass, lowpass, band-reject , and bandpassfilters
� Filtering is achieved in the frequency domain bydesigning the proper filter (Chapter 4)
� Filtering can be done in the spatial domain also byusing filter masks (kernels, templates, or windows)
�Unlike frequency domain filters, spatial filters canbe nonlinear !
3
Spatial Filtering Mechanics
� A spatial filter is characterized by � A rectangular neighborhood of size mxn (usually m and n are odd)
� A predefined operation that is specified by the mask values at each position. position.
3x3 filter mask example
w(-1,-1) w(-1,0) w(-1,1)
w(0,-1) w(0,0) w(0,1)
w(1,-1) w(1,0) w(1,1)
Origin y
(x, y)Neighbourhood
3x3 filter mask example
� Spatial filtering Operation� The filter mask is centered at each pixel in the image and the output
pixel value is computed based on the operation specified by themask
4
x Image f (x, y)
Spatial Filtering MechanicsOrigin
y
xImage f (x, y)
3x3 Filter
xw(-1,-1) w(-1,0) w(-1,1)
w(0,-1) w(0,0) w(0,1)
w(1,-1) w(1,0) w(1,1)
f(x-1,y-1) f(x-1,y) f(x-1,y+1)
f(x,y-1) f(x,y) f(x,y+1)
f(x+1,y-1) f(x+1,y) f(x+1,y+11)
5
Filter Original Image Pixels
a b
s a t b
g( x, y ) w( s,t ) f ( x s, y t )=− =−
= + +∑∑ ∑
Vector Representation of
Spatial Filtering
� The previous filtering equation can be written as
R w z w z ... w z= + + +
� If we represent the coefficients of the filter mask as a row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and
w1 w2 w3
w4 w5 w6
w7 w8 w9
1 1 2 2
1
mn mn
mn
k k
k
R w z w z ... w z
= w z =
= + + +
∑
z1 z2 z3
z4 z5 z6
z7 z8 z9
row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and the image pixels under the mask by z = [z1 z2 z3 z4 z5
z6 z7 z8 z9], then the filtering equation can be written as
6
TR wz=
Spatial Correlation� The filtering operation just described is called
correlation
� In 1-DOrigin
0 0 0 1 0 0 0 0
1 2 3 2 8
f wOrigin
0 0 0 1 0 0 0 0
1 2 3 2 81) Function and mask
2) Initial Alignment
7
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
3) Zero Padding for f by m-1
4) Position after one shift
Spatial Correlation
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts
.6) Repeat
.
.
.
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position
8
Full correlation result
Cropped correlation result
0 0 0 8 2 3 2 1 0 0 0 0
0 8 2 3 2 1 0 0
Spatial Convolution
� A strongly related operation to correlation isconvolution (an important tool in linear systemstheory)theory)
� The mechanics of convolution is similar to thoseof correlation, except that the filter mask isrotated by 180o before sliding
9
Spatial Convolution
0 0 0 1 0 0 0 0
f wOrigin
1 2 3 2 81) Function and mask
8 2 3 2 1
0 0 0 1 0 0 0 0
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
2) Initial Alignmentafter rotation ofmask
3) Zero Padding for f by m-1
10
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift
Spatial Convolution
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts
.6) Repeat
.
.
.
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position
11
Full convolution result
Cropped convolutionresult
0 0 0 1 2 3 2 8 0 0 0 0
0 1 2 3 2 8 0 0
Extension to 2D� Extension to 2D is straight forward
� Zero padding is done in both directions by m-1 and n-1
� In convolution, the mask is flipped vertically and horizontally horizontally
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 0
f
Zero padding by m-1 and n-1
12
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0
1 2 3
4 5 6
7 8 9
9 8 7
6 5 4
3 2 1
w Mask rotation
Extension to 2D
9 8 7 0 0 0 0 0 01 2 3 0 0 0 0 0 0
6 5 4 0 0 0 0 0 0
3 2 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
4 5 6 0 0 0 0 0 0
7 8 9 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
13
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Initial position in convolution
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Initial position in correlation
Extension to 2D
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 9 8 7 0 0 0
0 0 0 6 5 4 0 0 0
0 0 0 3 2 1 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 1 2 3 0 0 0
0 0 0 4 5 6 0 0 0
0 0 0 7 8 9 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
14
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Full convolution result
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Full correlation result
Extension to 2D
0 0 0 0 0
0 1 2 3 0
0 4 5 6 0
0 0 0 0 0
0 9 8 7 0
0 6 5 4 0
Cropped convolution result
0 7 8 9 0
0 0 0 0 0
Cropped correlation result
0 6 5 4 0
0 3 2 1 0
0 0 0 0 0
Correaltion Convolution
15
a b
s a t b
Correaltion
g( x, y ) w( s,t ) f ( x s, y t )=− =−
= + +∑∑a b
s a t b
Convolution
g( x,y ) w( s,t ) f ( x s, y t )=− =−
= − −∑∑
Treatment of Pixels at Edges
� In the previous slides, we padded the image with zerosin both directions in order to compensate forunavailable values.
xunavailable values.
� Other approaches
� Replicate edge pixels
� Consider only available
pixels that fall under the
mask in the computation
xe
e
e
e
mask in the computation
of the new values
� Allow pixels to wrap
around16
yImage f (x, y)
ee e
Treatment of Pixels at Edges
� The result depends on the used approach
� Example 3.6
Filtered Image:
OriginalImage
Filtered Image: Zero Padding
Filtered Image: Replicate Edge Pixels
17
Filtered Image: Wrap Around Edge Pixels
Smoothing Spatial Filters
� A smoothing (averaging, blurring) filter replaces eachpixel with the average value of all pixels under themaskmask
� Uses of smoothing filters
� The smoothed image corresponds to the gross detailsof the image or the low frequency content of the image
� Can be used to reduce noise as it is characterized withsharp transitionssharp transitions
� However, smoothing usually results in degradation ofedge quality (sharpness of the image)
18
Smoothing Spatial Filters
� Common smoothing masks
1 1 1 1 2 1
� Notes
1 1 1
1 1 1
1 1 1
X 1/9
1 2 1
2 4 2
1 2 1
X 1/16
Standard averaging mask Weighted average mask
� Notes
� The weighted average filter gives more weight to pixelsnear the center
� It is hard to see the visual difference between theprocessing results of the two filters; however, the weightedaverage filter coefficients sum up to 16, which makes itmore attractive for computers19
� Example: 3.7 Smoothing with different mask sizes. Notice the loss of details as the mask size increases
Smoothing Spatial Filters
Original Smoothing by 3x3 Mask Smoothing by 5x5 Mask
20Smoothing by 9x9 Mask Smoothing by 15x15 Mask Smoothing by 35x35 Mask
� Example 3.8: smoothing highlights gross details. Could be useful in providing better segmentation results
Smoothing Spatial Filters
results
21
Original Image Smoothed Image Thresholded Image
� Example 3.9: Noise Reduction
Smoothing Spatial Filters
22
Original Smoothed
Sharpening Spatial Filters
� The principle objective of sharpening is to highlighttransitions in intensity which usually correspond toedges in images; thus sharpening is the opposite ofsmoothingedges in images; thus sharpening is the opposite ofsmoothing
� If we examine the smoothing operation we can thinkof it as integration
� Thus to perform sharpening in the spatial domain, it isintuitive to use differentiationintuitive to use differentiation
� In the following few slides we examine thefundamental properties of the first and secondderivatives when applied on different intensitytransitions
23
Sharpening Spatial Filters� Lets discuss differentiation for the1-D case
� We are concerned about the behavior of 1st and 2nd
derivatives in the following areas � Constant intensity� Constant intensity
� Onset and end of discontinuities (ramps and steps discontinuities)
� Intensity ramps
� Properties of 1st derivative� Zero in areas of constant intensity
� Nonzero at the onset of a step and intensity rampNonzero at the onset of a step and intensity ramp
� Nonzero along intensity ramp
� Properties 2nd derivative� Zero in areas of constant intensity
� Nonzero at the onset and end of a step and intensity ramp
� Zero along intensity ramp 24
Sharpening Spatial Filters
� Derivatives can be approximated as differences
� 1st derivative at x � 1st derivative at x
2nd derivative at x
1f
f ( x ) f ( x )x
∂ = + −∂
� 2nd derivative at x
25
2
21 1 2
ff ( x ) f ( x ) f ( x )
x
∂ = + + − −∂
Sharpening Spatial Filters� Investigation of the derivatives behavior
26
Sharpening Spatial Filters
� Notes
� Examining the 1st and 2nd derivatives plots showsthat all of their properties are satisfiedthat all of their properties are satisfied
� 1st derivative produces thicker edges than 2nd
derivatives
� 2nd derivative produces double edge separated by azero crossingzero crossing
� 2nd derivative is commonly used in sharpening sinceit has simpler implementation and finer edges
27
Sharpening Using 2nd
Derivative� The second derivative (Laplacian) in 2-D is defined as
2 22 f f
f∂ ∂∇ = +
If we define
Then the discrete second derivative can be approximated
22 2
f ff
x y
∂ ∂∇ = +∂ ∂
2
21 1 2
ff ( x , y ) f ( x , y ) f ( x, y )
x
∂ = + + − −∂
2
21 1 2
ff ( x, y ) f ( x, y ) f ( x, y )
y
∂ = + + − −∂
Then the discrete second derivative can be approximated by
2
1 1 1 1 4
f
f ( x , y ) f ( x , y ) f ( x, y ) f ( x, y ) f ( x, y )
∇ =+ + − + + + − −
28
Sharpening Using 2nd
Derivative� The Laplacian can be implemented as a filter mask
0 1 0
� Or
0 1 0
1 -4 1
0 1 0
1 1 1
1 -8 1
1 1 1
29
Sharpening Using 2nd
Derivative� Computing the Laplacian doesn’t produce a sharpened
image. However, grayish edge lines and discontinuitiessuperimposed on a dark backgroundsuperimposed on a dark background
� It is common practice to scale the Laplacian image to [0,255] for better display30
Sharpening Using 2nd
Derivative� Alternatively, to obtain a sharpened image g(x,y),
subtract the Laplacian image from the original image
2g( x, y ) f ( x, y ) f ( x, y )= − ∇
- =
31
- =
OriginalImage
LaplacianFiltered Image
SharpenedImage
Sharpening Using 2nd
Derivative�The two steps required to achieve sharpening can becombined into a single filtering operation
),1(),1([),( yxfyxfyxf −++−=)1,()1,( −+++ yxfyxf
)],(4 yxf−
fyxfyxg 2),(),( ∇−=
)],(4 yxf−),1(),1(),(5 yxfyxfyxf −−+−=)1,()1,( −−+− yxfyxf
32
Sharpening Using 2nd
Derivative
�As a filter mask, the previous equation can be represented as
0 -1 0as
0 -1 0-1 5 -10 -1 0
33
Sharpening Using 1st Derivative
�The first derivative in 2-D is implemented as a gradientwhich is defined as a vector
∂f
� The gradient points in the direction of greatest rate ofchange at location (x,y)
∂∂∂∂
=
=∇
y
fx
f
G
G
y
xf
change at location (x,y)
� The magnitude of the gradient is defined as
Or, approximately
34
12 22
f fM( x, y )
x y
∂ ∂ = + ∂ ∂ | | x yf G G∇ ≈ +
Sharpening Using 1st Derivative
� Computation of the gradient using Roberts cross-gradient operators
� Roberts operators for computing Gx and Gy are shownbelow. The upper-left pixel in the operator is overlaid overbelow. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5 in the original image
-1 0
0 1
0 -1
1 0
Horizontal Operator
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9Vertical Operator
35
[ ][ ]
29 5 9 5
28 6 8 6
9 5 8 6
x
y
G ( x, y ) z z z z
G ( x, y ) z z z z
M( x, y ) z z z z
= − ≈ −
= − ≈ −
≈ − + −
Horizontal Operator
Pixel z5 and its neighbours
Vertical Operator
� Computation of the gradient using Sobel operators
� Sobel operators for computing Gx and Gy are shownbelow. The center pixel in the operator is overlaid over the
Sharpening Using 1st Derivative
below. The center pixel in the operator is overlaid over thepixel (x,y) or z5
-1 0 1
-2 0 2
-1 0 1
-1 -2 -1
0 0 0
1 2 1
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9
36
7 8 9 1 2 3
3 6 9 1 4 7
2 2
2 2
x
y
x y
G ( x, y ) ( z z z ) ( z z z )
G ( x, y ) ( z z z ) ( z z z )
M( x, y ) G ( x, y ) G ( x, y )
≈ + + − + +
= + + − + +
= +
Mask to Compute GxPixel z5 and its neighbours Mask to Compute Gy
� Example 3.10
� Gradient is widely used in industrial inspection as itproduce thicker edges in the result, which make it easierfor machines to detect artifacts
Sharpening Using 1st Derivative
for machines to detect artifacts
37
Optical Image for a contact lens Gradient Image
Nonlinear (Order-statistic)
Spatial Filters
Order-statistic filters are nonlinear filters whoseresponse is based on ordering the pixels under the maskand then replacing the centre pixel with the valueand then replacing the centre pixel with the valuedetermined by the ranking result
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9
Rank
� Examples
� minimum filter, maximum filter, median filter � popular and useful in removing impulse or salt-and-pepper noise with less blurring than linear smoothing38
New value
Nonlinear (Order-statistic)
Spatial Filters
� Example 3.11
Note how the median filter has reduced the noise significantly with less blurring than smoothing filter
39
Combining Spatial
Enhancements
� It is very common to combine different enhancement techniques in order to achieve different enhancement techniques in order to achieve the desired goal
� Example 3.12
� We want to enhance/sharpen the bones in this imagebones in this image
� Direct application of the Laplacian may result in noise amplification
40
Combining Spatial
EnhancementsCompare to enhancement by single method
41
Processed by a combination of
methods
Sharpened by Laplacian
Histogram Equalization
Power-Law with gamma = 0.5
Combining Spatial
Enhancement
Enhancement by multiple methods
Laplacian filter of
(a)
Laplacian filter of bone scan (a)
Sharpened version of bone scan achieved by subtracting (a) and (b) Sobel filter of bone
scan (a)
(b)
(c)
(d)42
Combining Spatial
Enhancement
Sharpened image which is sum of (a)
Result of applying a power-law trans. to (g)
(g)
(h)
The product of (c) and (e) which will be used as a mask
which is sum of (a) and (f)
(e)
(f)
(g)
Image (d) smoothed witha 5*5 averaging filter
43
Combining Spatial
Enhancement
Compare the original and final images
44
Readings
� Read section 3.6.3 titled “Unsharp Masking and Highboost filtering”.
45
Related Matlab Functions
� Check Matlab documentation for the following functions
�conv2
�corr2
� fillter2
� imfilter� imfilter
�padarray
�median
�medfilt246