development of image restoration techniquesethesis.nitrkl.ac.in/4361/1/,development_of_image... ·...
TRANSCRIPT
DEVELOPMENT OF IMAGE RESTORATION
TECHNIQUES
THESIS REPORT SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF
Master of Technology
in
Computer Science and Engineering
by
TONY XAVIER
Department of Computer Science and Engineering
National Institute of Technology
Rourkela
May 2007
DEVELOPMENT OF IMAGE RESTORATION
TECHNIQUES
THESIS REPORT SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF
Master of Technology
in
Computer Science and Engineering
by
TONY XAVIER
Under the Guidance of
Prof.BANSHIDHAR MAJHI
Department of Computer Science and Engineering
National Institute of Technology
Rourkela
May 2007
National Institute of Technology
Rourkela
CERTIFICATE
This is to certify that the thesis entitled,Development of Image Restoration Techniques
submitted by Sri.Tony Xavier in partial fulfillment of the requirements for the award of
Master of Technology Degree in Computer Science and Engineering at National Institute
of Technology, Rourkela (Deemed University) is an authentic work carried out by him
under my supervision and guidance.
To the best of my knowledge, the matter embodied in the thesis has not been submitted
to any other University / Institute for the award of any Degree or Diploma.
Prof. Banshidhar Majhi
Professor
Dept. of Computer Science and Engg.
National Institute of Technology
Rourkela-769008
21 May 2007
Acknowledgments
There are a few people without whose help this thesis would have been incomplete. First
among them is of course Prof.Banshidhar Majhi for his excellent guidance and motivation
he offered from the beginning to the end of the thesis. Next, my gratitude goes to Pankaj
Kumar Sa, Lecturer of our department, for his guidance and help whenever I required.
I am thankful to Prof. S K Jena, Head of our department for providing with all the
facilities required. Also, I am thankful to all the Professors of our department for their
help and encouragement they offered all the time. Last but not the least is my friends
who helped throughout the year to get over all difficulties no matter technical or personal.
TONY XAVIER
Contents
1 INTRODUCTION 1
1.1 A MODEL OF IMAGE DEGRADATION . . . . . . . . . . . . . . . . . 1
1.2 POINT SPREAD FUNCTION . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 DECONVOLUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Classical Restoration Techniques . . . . . . . . . . . . . . . . . . 3
1.3.2 Blind Deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 DENOISING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.1 Noise Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.2 Denoising Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 BLIND DECONVOLUTION 12
2.1 APPROACHES IN BLIND DECONVOLUTION . . . . . . . . . . . . . 13
2.1.1 Classification of Blind Techniques . . . . . . . . . . . . . . . . . . 13
2.2 A PRIORI BLUR IDENTIFICATION . . . . . . . . . . . . . . . . . . . 14
2.2.1 Cepstrum Based Motion Blur Identification . . . . . . . . . . . . 15
3 NON PARAMETRIC TECHNIQUES BASED ON IMAGE CONSTRAINTS 17
3.1 DETERMINISTIC IMAGE CONSTRAINTS . . . . . . . . . . . . . . . 17
3.2 NON NEGATIVITY AND SUPPORT
CONSTRAINTS RECURSIVE INVERSE
FILTERING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.1 Limitations of NAS-RIF . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 ITERATIVE BLIND DECONVOLUTION . . . . . . . . . . . . . . . . . 22
3.3.1 Shortcomings of Iterative Blind Deconvolution . . . . . . . . . . . 24
3.4 IMPROVED ITERATIVE BLIND DECONVOLUTION . . . . . . . . . 25
3.4.1 Accelerating the convergence . . . . . . . . . . . . . . . . . . . . . 25
i
3.4.2 Definition of Stopping criteria . . . . . . . . . . . . . . . . . . . . 25
3.4.3 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.5 SIMULATION RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4 DENOISING 30
4.1 IMPULSE NOISE DETECTION . . . . . . . . . . . . . . . . . . . . . . 31
4.2 PERFORMANCE MEASURES . . . . . . . . . . . . . . . . . . . . . . . 31
4.3 IMPULSE NOISE DETECTION BASED ON
PIXEL WISE MAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.4 DOUBLE DERIVATIVE BASED IMPULSE DETECTION . . . . . . . 34
4.5 ITERATIVE NOISE FILTERING WITH EDGE RETRIEVAL . . . . . 35
4.5.1 Edge retrieval based on single derivative . . . . . . . . . . . . . . 36
4.5.2 Edge retrieval based on pixel wise MAD . . . . . . . . . . . . . . 37
4.5.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.6 SIMULATION RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5 CONCLUSIONS AND FUTURE WORK 40
5.1 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.2 LIMITATIONS AND FUTURE WORK . . . . . . . . . . . . . . . . . . 41
ABSTRACT
Image denoising and image deblurring are studied as part of the thesis. In deblurring,
blind deconvolution is investigated. Out of the several classes of blind deconvolution
techniques, Non parametric Methods based on Image Constraints are studied at greater
depth. A new algorithm based on the Iterative Blind Deconvolution(IBD) technique is
developed. The algorithm makes use of spatial domain constraints of non-negativity and
support. The Fourier-domain constraint may be described as constraining the product of
the Fourier spectra of the image f and the Fourier spectra of the point spread function h
to be equal to the convolution spectrum. Within each iteration, the algorithm switches
between spatial domain and frequency domain and imposes known constraints on each.
The convergence of the original IBD can be accelerated by limiting high magnitude val-
ues in frequency domain of both estimated image and point spread function. The new
algorithm converges within less than 25 iterations where as the original IBD took nearly
500 iterations. Inclusion of the support constraint in the spatial domain improves quality
of the restored image. Also, sum of the spatial domain values of the point spread function
should be made equal to one at the end of each iteration, for preventing the loss of image
intensity. PSNR values calculated for restored images show significant improvement in
image quality. A PSNR of 17.8dB is obtained for the improved scheme where as it is
14.3dB for the original IBD. A new stopping criteria based on standard deviation of the
image power for last k iterations is defined for stopping the algorithm when it converges.
In denoising, an edge retrieval technique is developed which preserves the image details
along with effectively removing impulse noise. Noisy pixels are detected in the first phase
and in the next phase those pixel values are replaced with an estimate of the actual value.
For dealing with the wrong classification of edge pixels as noisy pixels, an edge retrieval
technique based on pixel-wise MAD is defined. This scheme retrieves the pixels which
are wrongly classified as noise. The algorithm gives high PSNR values as well as very
good detail preservation.
List of Figures
1.1 A model of the image degradation/restoration process. . . . . . . . . . . 2
1.2 Convolution of the point spread function with a point object gives the
observed image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 (a)Lena image of size 256× 256. (b) Lena image blurred with motion blur
of length 31 pixels and angle 45 degree. . . . . . . . . . . . . . . . . . . 5
1.4 (a)Motion blurred Lena image in Fig.(1.3b) restored using direct inverse
filtering. (b) Image restored using direct inverse filtering when gaussian
noise with mean zero and variance 1 is added to the image. . . . . . . . 5
1.5 (a)Blurred image of figure(1.3b) deconvolved using weiner filter. (b)Blurred
image of figure(1.3b) deconvolved using constrained least square filtering. 6
1.6 The probability density function for Gaussian Distribution . . . . . . . . 8
1.7 (a)Probability density function for Uniform Noise. (b) Probability density
function for salt and pepper noise . . . . . . . . . . . . . . . . . . . . . . 9
2.1 The classification of blind deconvolution techniques . . . . . . . . . . . . 14
2.2 (a)The original image. (b) The original image blurred with a motion blur
of length 31pixels and angle 0o. (c) Image restored using cepstral method. 15
3.1 (a).An image with the support surrounding it. (b) The image gets spread
outside the support after blurring. . . . . . . . . . . . . . . . . . . . . . 18
3.2 Block Diagram of the NASRIF algorithm . . . . . . . . . . . . . . . . . . 19
3.3 The sequence of operations to be performed in the IBD algorithm. . . . . 23
3.4 (a)Original Image. (b)Motion blured image using 13x23 PSF . . . . . . 27
3.5 (a)Image Deconvolved using IBD with β = .5. (b)Image deconvolved with
IBD with β = 0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
iv
3.6 Results of Improved IBD: (a)Image Deconvolved without support informa-
tion. FMAX = 90000, DEV = 7.0e+006, HMAX = 1, hMIN=.0001. Result
obtained after 23 iterations. (b)Image deconvolved using IIBD algorithm
with support information and same parameters as (a). Result obtained
after 21 iterations. The PSNR value obtained is 17.8dB . . . . . . . . . 28
3.7 Results of Improved IBD: (a)Image blurred with camera defocus PSF of
radius 10. (b)Image deconvolved using IIBD algorithm with support infor-
mation and same parameters as figure(3.5). (b)Result obtained after 168
iterations. The PSNR value obtained is 16.7dB . . . . . . . . . . . . . . 29
3.8 Results of Improved IBD: (a)Image blurred with motion blur of length 21
pixels and angle 45 degree. (b)Image deconvolved using IIBD algorithm
with support information and same parameters as given in figure( 3.5).
Result obtained after 29 iterations. . . . . . . . . . . . . . . . . . . . . . 29
4.1 (a)Plot of pixel values in a horizontal line of a non-noisy image. (b) The
plot for a noisy image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2 (a)Original Lena image of size 256×256. (b) Lena image with 15% random
valued impulse noise added to it. . . . . . . . . . . . . . . . . . . . . . . 38
4.3 (a)Median filtered Image with 15% noise. PSNR=28.35dB(b) Image fil-
tered using double derivative method with Threshold 50. PSNR=28.94dB 38
4.4 (a)Lena image filtered using pixel-wise MAD algorithm. PSNR=28.75dB
(b) Image (a) after processing by passing the output of PWMAD through
the first phase of the proposed algorithm. PSNR=30.16dB . . . . . . . . 39
4.5 (a)Image filtered using the Two-phase scheme with single derivative used
for edge retrieval. PSNR=30.18dB (b) Image filtered using the Two-phase
scheme with PWMAD used for edge retrieval. PSNR=31.42dB . . . . . 39
Chapter 1
INTRODUCTION
Image Restoration is the process of reconstructing or recovering an image that has
been degraded by some degradation phenomenon. Restoration techniques are primarily
modeling of the degradation and applying the inverse process in order to recover the
original image. Image restoration techniques exist both in spatial and frequency domain.
1.1 A MODEL OF IMAGE DEGRADATION
An image may be described as a two-dimensional function I
I = f(x, y) (1.1)
where x and y are spatial coordinates. Amplitude of f at any pair of coordinates (x, y)
is called intensity I or gray value of the image. When spatial coordinates and amplitude
values are all finite, discrete quantities, the image is called digital image.If f(x, y) is the
original image, h(x, y) is a degradation function and η(x, y) is the additive noise then the
degraded image g(x, y) is given as [9]:
g(x, y) = f(x, y)∗h(x, y) + η(x, y) (1.2)
where the symbol ∗ indicates spatial convolution. Since convolution in spatial domain
is equal to multiplication in the frequency domain, the corresponding frequency domain
representation is given as:
G(u, v) = F (u, v)H(u, v) + N(u, v) (1.3)
where the terms in capital letters are the Fourier transforms of the corresponding terms
in equation( 1.2)
1
Figure 1.1: A model of the image degradation/restoration process.
Many types of the degradations can be approximated by linear, position invariant pro-
cesses. Since degradations are modeled as being the result of convolution, and restoration
seeks to find filters that apply the process in reverse, the term image deconvolution is
used to signify linear image restoration.
1.2 POINT SPREAD FUNCTION
The linear position-invariant function h(x, y) in equation(1.2) is known as a point spread
function. The point spread function gets convolved with the original image to give the
degraded image. Some commonly occurring image degradations, which are linear and
position-invariant are given below.
Motion Blur
We often see images blurred because of camera movement during image capture. Suppose
the relative motion is of velocity ν at an angle θ with the horizontal axis and if T is the
duration of exposure, then the blur length is L = νT and the motion blur PSF can be
expressed as
h(x, y) =
1/L if 0 ≤ |x| ≤ L cos θ; y = Lsinθ
0 otherwise(1.4)
2
Figure 1.2: Convolution of the point spread function with a point object gives the ob-
served image.
Camera Defocus
Another commonly occurring blur is because of improperly focussed camera. Assuming
the lens system is of circular aperture, with radius r the point spread function can be
expressed as
h(x, y) =
0 if√
x2 + y2 > r
1/πr2 otherwise(1.5)
1.3 DECONVOLUTION
If we can estimate the point spread function, h(x, y) which caused the degradation, then
we can get back the original image by deconvolution. There are two classes of deconvo-
lution techniques. Classical image restoration techniques and blind deconvolution tech-
niques. In classical restoration, we need to have prior knowledge of the PSF which caused
the degradation. Blind deconvolution techniques are used when we don’t have any prior
knowledge of the degradation process.
1.3.1 Classical Restoration Techniques
Classical restoration techniques need prior knowledge of the degradation process. The
degradation can be estimated by one of the several techniques [9] given below.
• Estimation by image observation.
3
• Estimation by experimentation
• Estimation by modeling
After estimating the point spread function, one of the three techniques [9] given below
can be used for deconvolution.
Inverse Filtering
Direct inverse filtering is the simplest approach to restoration. In this method, an esti-
mate of the Fourier transform of the image F (u, v) is computed by dividing the Fourier
transform of the degraded image by the Fourier transform of the degradation function.
F (u, v) =G(u, v)
H(u, v)(1.6)
This method works well when there is no additive noise in the degraded image. That
is, when the degraded image is given by g(x, y) = f(x, y)∗h(x, y). But if noise gets added
to the degraded image then the result of direct inverse filtering is very poor. Equation
1.3 gives the expression for G(u, v). Substituting for G(u, v) in the above equation, we
get
F (u, v) = F (u, v) +N(u, v)
H(u, v)(1.7)
The above equation shows that direct inverse filtering fails when additive noise is
present in the degraded image. Because noise is random and so we can not find the noise
spectrum N(u, v).
Minimum Mean Square Error Filtering
Minimum mean square filtering, also known as Wiener Filtering, is more robust in
the presence of additive noise. Wiener filtering incorporates both degradation function
and statistical characteristics of noise into the restoration process. The objective of this
technique is to find an estimate f of the original image f such that the mean square error
between them is minimized. This error measure is given by
e2 = E
(f − f)2
(1.8)
where E . is the expected value of the argument. The minimum of the error function
in the above equation is given in the frequency domain by the expression
4
Figure 1.3: (a)Lena image of size 256× 256. (b) Lena image blurred with motion blur of
length 31 pixels and angle 45 degree.
Figure 1.4: (a)Motion blurred Lena image in Fig.(1.3b) restored using direct inverse
filtering. (b) Image restored using direct inverse filtering when gaussian noise with mean
zero and variance 1 is added to the image.
F (u, v) =
[H∗(u, v)Sf (u, v)
Sf (u, v)|H(u, v)|2 + Sη(u, v)
]G(u, v)
=
[H∗(u, v)
|H(u, v)|2 + Sη(u, v)/Sf (u, v)
]G(u, v)
=
[1
H(u, v)
|H(u, v)|2|H(u, v)|2 + Sη(u, v)/Sf (u, v)
]G(u, v) (1.9)
5
where H∗(u, v) is the complex conjugate of H(u, v) and |H(u, v)|2 = H∗(u, v)H(u, v)
Sη(u, v) = |N(u, v)|2 = power spectrum of the noise
Sf (u, v) = |F (u, v)|2 =power spectrum of the original image.
If the noise is zero, then the noise power spectrum vanishes and the Wiener filter reduces
to the inverse filter. But since it is not possible to get the power of the original image,
power spectrum of the degraded image can be used.
Figure 1.5: (a)Blurred image of figure(1.3b) deconvolved using weiner filter. (b)Blurred
image of figure(1.3b) deconvolved using constrained least square filtering.
Constrained Least Squares Filtering
The method of using a constant for the ratio of power spectra is not a suitable solu-
tion always. Constrained least square filtering require knowledge of only the mean and
variance of the noise. The method finds the minimum of a criterion function C, defined
as
C =M−1∑x=0
N−1∑y=0
[O2f(x, y)
]2(1.10)
subject to the constraint
‖ g −Hf ‖2=‖ η ‖2 (1.11)
where ‖ . ‖ is the Euclidean vector norm and f is the estimate of the original image. The
laplacian operator O2 is defined as
O2f =∂2f
∂x2+
∂2f
∂y2(1.12)
6
The frequency domain solution for this optimization problem is given by the expression
F (u, v) =
[H∗(u, v)
|H(u, v)|2 + γ|P (u, v)|2]
G(u, v) (1.13)
where γ is a parameter that must be adjusted so that the constraint mentioned above is
satisfied, and P (u, v) is the Fourier transform of the function
p(x, y) =
0 −1 0
−1 4 −1
0 −1 0
(1.14)
which is actually the Laplacian operator.
1.3.2 Blind Deconvolution
All the techniques described so far, requires the knowledge of the exact degradation func-
tion. Then the restoration algorithm is applied to invert the degradation process. Those
techniques are called the Classical Restoration Techniques. If the deconvolution is per-
formed without having prior knowledge of the degradation function, then it is called blind
deconvolution [12]. In blind deconvolution, both the degradation function and the true
image are estimated from the degraded image characteristics. Partial information about
the imaging system may also be utilized if available.
Blind deconvolution is of great interest because in most of the practical cases, knowing
the degradation function is not possible. For example, in applications like remote sensing
and astronomy, it is difficult to statistically model the original image or even know specific
information about scenes never imaged before.
1.4 DENOISING
Another type of image degradation is due to additive noise. Random values get added
to the intensity values. Denoising involves the application of some filtering technique to
get the true image back. The denoising algorithms [20] vary greatly depending on the
type of noise present in the image. Each type of image is characterized by a unique noise
model. Each noise model corresponds to a probability density function which describes
the distribution of noise within the image.
7
1.4.1 Noise Models
The spatial component of noise is based on the statistical behaviour of the intensity
values. These may be considered as random variables , characterized by a probability
density function (PDF). Some commonly found noise models [9] and their corresponding
PDFs are given below.
Gaussian Noise
Gaussian noise is noise that has a probability density function of the normal distribution
(also known as Gaussian distribution). In other words, the values that the noise can take
on are Gaussian distributed. If z is a Gaussian random variable representing noise, then
its PDF is given by
p(z) =1√2πσ
e−(z−µ)2/2σ2
(1.15)
Figure 1.6: The probability density function for Gaussian Distribution
Gaussian noise is additive noise. That is the Gaussian distributed noise values get
added to the intensity values of the image.
8
Uniform Noise
Another commonly observed image noise is uniform noise. In this case the noise can take
on values in an interval [a, b] with uniform probability. The PDF of uniform noise is given
by
p(z) =
1b−a
ifa ≤ z ≤ b
0 otherwise
(1.16)
The plot of the probability density function for uniform noise distribution is given in
figure(1.7a).
Figure 1.7: (a)Probability density function for Uniform Noise. (b) Probability density
function for salt and pepper noise
Impulse Noise
Impulse noise is characterized by a noise spike replacing the actual pixel value. Impulse
noise is further divided into two classes. Random valued impulse noise(RVIN) and salt
& pepper noise(SPN). In RVIN, the impulse value at a particular pixel may be a random
value between a particular interval. But in SPN the impulses are either the minimum
value or the maximum value allowed in the intensity values. For example 0 and 255 in
the case of 8-bit image.
9
1.4.2 Denoising Techniques
When the only degradation present in an image is noise, then equation( 1.2) becomes
g(x, y) = f(x, y) + η(x, y) (1.17)
and Eq(1.3) becomes
G(u, v) = F (u, v) + N(u, v) (1.18)
Denoising techniques exist in both spatial domain as well as frequency domain.
Spatial Filtering
Spatial filtering is preferred when only additive noise is present. The different classes [9]
of filtering techniques exist in spatial domain filtering.
• Mean Filters
• Order-Statistics Filters
• Adaptive Filters
Arithmetic mean filter
This belongs to the category of mean filters. In this method the middle pixel value of the
filter window is replaced with the arithmetic mean of all the pixel values within the filter
window. A mean filter simply smoothes local variations in an image. Noise is reduced as
a result of this smoothening, but edges within the image get blurred.
Median Filter
Median filter comes under the class of order-statistics filters. Response of Order-statistics
filters is based on ordering the pixels contained in the filter window. Median filter replaces
the value of a pixel by the median of the gray levels within the filter window. Median
filters are particularly effective in the presence of impulse noise.
Adaptive Filters
Adaptive filters change its behaviour based on the statistical characteristics of the image
inside the filter window. Adaptive filter performance is usually superior to non-adaptive
counterparts. But the improved performance is at the cost of added filter complexity .
10
Mean and variance are two important statistical measures using which adaptive filters
can be designed. For example if the local variance is high compared to the overall image
variance, the filter should return a value close to the present value. Because high variance
is usually associated with edges and edges should be preserved.
11
Chapter 2
BLIND DECONVOLUTION
In the first chapter we have seen that if we can estimate the point spread function,
h(x, y) which caused the degradation, then we can get back the actual original image
by deconvolution. But unfortunately, in many practical situations, the blur is often un-
known, and little information is available about the true image. Therefore, the true image
f(x, y) must be identified directly from g(x, y) by using partial or no information about
the blurring process and the true image. Such an estimation problem, assuming the linear
degradation model of equation(1.2) is called blind deconvolution.
There are several motivating factors behind the use of blind deconvolution for image
processing applications. In practice, it is often costly, dangerous, or physically impossi-
ble to obtain a priori information about the imaged scene. For example, in applications
like remote sensing and astronomy, it is difficult to statistically model the original image
or even know specific information about scenes never imaged before . In addition, the
degradation from blurring cannot be accurately specified. In aerial imaging and astron-
omy, the blurring cannot be accurately modeled as a random process, since fluctuations
in the PSF are difficult to characterize. In real-time image processing, such as medical
video-conferencing, the parameters of the PSF cannot be pre-determined to instanta-
neously deconvolve images. Moreover, on-line identification techniques used to estimate
the degradation may result in significant error, which makes the restored image useless.
In other applications, the physical requirements for improved image quality are un
realizable. For instance, in space exploration, the physical weight of a high resolution
camera exceeds practical constraints. Similarly, in x-ray imaging, improved image qual-
12
ity occurs with increased incident x-ray beam intensity, which is hazardous to a patient’s
health. Thus, blurring is unavoidable. In such situations, the hardware available to
measure the PSF of an imaging system is often difficult to use. Although these meth-
ods work well to identify the PSF, they are esoteric, which limits their wide use. Blind
deconvolution is a viable alternative.
2.1 APPROACHES IN BLIND DECONVOLUTION
There are two main approaches to blind deconvolution of images:
1. Identifying the PSF separately from the true image, in order to use it later with one of
the known classical image restoration methods. Estimating the PSF and the true image
are disjoint procedures. This approach leads to computationally simple algorithms.
2. Incorporating the identification procedure with the restoration algorithm. This merge
involves simultaneously estimating the PSF and the true image, which leads to the de-
velopment of more complex algorithms.
2.1.1 Classification of Blind Techniques
Blind deconvolution techniques can be further divided into five broad categories as shown
in figure(2.1). Each of those categories contain several algorithms for blind deconvolution.
Some of the algorithms are in frequency domain and some are in spatial domain. A few
algorithms make use of both frequency domain and spatial domain data. The five classes
of algorithms are given below. A detailed classification of blind deconvolution techniques
can be found in [12].
• A priori Blur Identification Methods
• Zero Sheet Separation
• ARMA Parameter Estimation Methods
• Non parametric Methods Based on Image Constraints
• Non Parametric methods Based on Higher Order Statistics
13
Figure 2.1: The classification of blind deconvolution techniques
2.2 A PRIORI BLUR IDENTIFICATION
A priori blur identification methods perform blind deconvolution by identifying the
PSF prior to restoration. This general class of techniques makes assumptions on the
characteristics of the PSF such as symmetry, and availability of a known parametric form
of the blur. Popular parametric models include PSFs resulting from linear camera mo-
tion or an out-of-focus lens system. Based on these assumptions, an attempt is made to
completely characterize the PSF using special features of the true/blurred image. Once
the PSF has been completely identified, one of the classical restoration techniques is used
to estimate the true image using deconvolution.
A priori blur identification techniques are the simplest class of blind deconvolution
methods to implement and have low computational requirements. They are applicable
to situations in which the true image is known to possess special features, or when the
PSF is known to be of a special parametric form. For more general situations or when
14
less information is available other deconvolution algorithms must be used.
2.2.1 Cepstrum Based Motion Blur Identification
A method for identifying linear motion blur is to compute the two-dimensional cepstrum
of the blurred image g(x, y) [11]. The cepstrum of g(x, y) is given by
C(g(x, y)) = F−1(log(|F(g(x, y))|). (2.1)
An important property of the cepstrum is that it is additive under convolution. Thus,
ignoring noise, we have
C(g(x, y)) = C(f(x, y)) + C(h(x, y)) (2.2)
C(h(x, y)) = F−1(log(|F(h(x, y))|) has large negative spikes at a distance L from the
origin. By the additivity of the cepstrum, this negative peak is preserved in C(g(x, y)),
also at a distance L from the origin. If the noise level of the blurred image is not too high,
there will be two pronounced peaks in the cepstrum. To estimate the angle of motion
blur, draw a straight line from the origin to the first negative peak. The angle of motion
blur is approximated by the inverse tangent of the slope of this line.
Figure 2.2: (a)The original image. (b) The original image blurred with a motion blur of
length 31pixels and angle 0o. (c) Image restored using cepstral method.
Limitations of the Cepstrum Based Technique
From figure( 2.2), it is evident that the cepstrum based method gives excellent results for
horizontal motion blur. Similar results were obtained for vertical motion blur also. The
15
restoration is perfect since the length estimation was accurate. But the length estimate
shows significant error when the motion blur direction is not horizontal or vertical.
16
Chapter 3
NON PARAMETRIC
TECHNIQUES BASED ON IMAGE
CONSTRAINTS
The algorithms of this class do not assume parametric models for either the image
or the blur. Deterministic constraints such as non negativity, known finite support, and
existence of blur invariant edges are assumed for the true image. A number of blind
deconvolution techniques for images fall into this class, which include the Iterative Blind
Deconvolution algorithm , Simulated Annealing algorithm , Non negativity And support
constraints Recursive Inverse Filtering (NAS-RIF)algorithm.
The methods are iterative and simultaneously estimate the pixels of the true image and
the PSF (or its inverse). The constraints on the true image and the PSF are incorporated
into an optimality criterion which is minimized using numerical techniques.
3.1 DETERMINISTIC IMAGE CONSTRAINTS
The deterministic constraints used in this particular class of techniques include non
negativity and known finite support. Non negativity can be assumed for both image pixel
values as well as PSF coefficients. Another constraint that can be used is a known finite
support. Support is the smallest rectangle which known to encompass the true image.
Blurring causes the image to spread outside the support. Restoration algorithm may
make use of the fact that true image pixels can not lie outside the support. The concept
17
of support is illustrated in figure( 3.1). Another spatial domain constraint which is found
to be useful is the fact that the sum of the PSF coefficients will always be equal to one.
This constrained may be enforced in each iteration of the algorithm.
Figure 3.1: (a).An image with the support surrounding it. (b) The image gets spread
outside the support after blurring.
3.2 NON NEGATIVITY AND SUPPORT
CONSTRAINTS RECURSIVE INVERSE
FILTERING
Non negativity and support constraints recursive inverse filtering (NAS-RIF) [13]
technique makes the assumption of non negativity on the true image. That is, the algo-
rithm assumes the true image pixel intensity values will be nonnegative. The knowledge
of a known support is also assumed. The only assumptions made on the PSF, however,
is that it is absolutely summable and that it has an inverse h−1(x, y) which is also ab-
solutely summable. An advantage of this method is that it does not require the PSF to
be of known finite extent, as do the other methods; this information is often difficult to
obtain.
The NAS-RIF technique is shown in figure( 3.2). It consists of a variable FIR fil-
ter u(x, y) with the blurred image g(x, y) as input. The output of this filter represents
18
an estimate of the true image f(x, y). This estimate is passed through a nonlinear filter,
which uses a non-expansive mapping to project the estimated image into the space repre-
senting the known characteristics of the true image. The difference between the projected
image fNL(x, y) and f(x, y) is used as the error signal to update the variable filter u(x,y).
Figure 3.2: Block Diagram of the NASRIF algorithm
If we assume the image is nonnegative with known support, the NL block of Fig.
3.2 represents the projection of the estimated image onto the set of images that are
nonnegative with given finite support. Thus, the negative pixel values within the region
of support must be zero, and the pixel values outside the region of support are the
background grey-level, LB. The cost function used in the restoration procedure is defined
as
J =∑
∀(x,y)
[fNL(x, y)− f(x, y)
]2
(3.1)
where
19
fNL(x, y) =
f(x, y) if f(x, y) > 0 and (x, y) ∈ SUP
0 if f < 0 and (x, y) ∈ SUP
LB if (x, y) ∈ SUP
(3.2)
Where SUP is the region of the image within the support and SUP is the region
outside the support. Substituting for fNL in equation( 3.1) gives
J =∑
(x,y)∈SUP
f 2(x, y)
[1− sgn(f(x, y))
2
]
+∑
(x,y)∈SUP
[f(x, y)− LB
]2
(3.3)
where the definition for sgn(.) is
sgn(f) =
1 if f ≥ 0
−1 if f < 0(3.4)
The parameter set u(x, y) = 0 for all (x, y) globally minimizes J . This results in a
restored image f(x, y) = 0 for all (x, y) , which is the all black solution. To avoid this
trivial solution, we make use of an assumption, that the sum of all the PSF coefficients
is positive, i.e.,∑
∀(x,y)
h(x, y) > 0 (3.5)
Also the sum of inverse filter coefficients is assumed to be 1, so that the filtering doesn’t
cause any loss of total image intensity. One option for constraining the parameters to
fulfill this condition is to normalize u(x, y) at every iteration. Another option is to use a
penalty method and add a third term to the cost function. The overall function is then
represented by
J =∑
(x,y)∈SUP
f 2(x, y)
[1− sgn(f(x, y))
2
]
+∑
(x,y)∈SUP
[f(x, y)− LB
]2
+ γ
∑
∀(x,y)
u(x, y)− 1
2
(3.6)
The cost function consists of three components. The first penalizes the negative pixels
of the image estimate inside the region of support, and the second penalizes the pixels of
20
the image estimate outside the region of support that are not equal to the background
color. The first component prevents the pixels of the intermediate restorations from
becoming highly negative. The conjugate gradient minimization routine is used for the
minimization of the cost function. NAS-RIF algorithm is summarized below.
• Set initial conditions:
u0 = [u0(1, 1), . . . , u0((Ux + 1)/2, (Uy + 1)/2), . . . , u0(Ux, Uy)] = [0, . . . , 1, . . . , 0]
tolerance : δ > 0
• At iteration k,(k=1,2,3,. . . )
1. If J(uk) ≤ δ then stop
2. Calculate the gradient vector of J
fk(x, y) = uk(x, y)∗g(x, y)
[OJ(uk)j+(i−1)∗Ux,1
]=
∂J(uk)
∂u(i, j)
= 2∑
(x,y)εSUP
fk(x, y)cl(fk(x, y))g(x− i + 1, y − j + 1)
+ 2∑
(x,y)∈SUP
[fk(x, y)− LB] g(x− i + 1, y − j + 1)
+ 2γ
∑
∀(x,y)
u(x, y)− 1
(3.7)
3. If k = 0, dk = −OJ(uk).
Otherwise,
a) βk−1 = 〈OJ(uk)−OJ(uk−1),OJ(uk)〉‖OJ(uk−1)‖2
b) dk = −OJ(uk) + βk−1dk−1
4. Find tk such that
J(uk + tkdk) ≤ J(uk + tdk) for t ∈ R
5. uk+1 = uk + tkdk
21
3.2.1 Limitations of NAS-RIF
The NAS-RIF algorithm did not yield results as expected. NAS-RIF algorithm makes
use of constraints only in spatial domain, not in spatial domain. That may be the reason
for the unsatisfactory performance. Minimizing the cost function may result in one of
several solutions, most of which are physically meaningless to the problem.
3.3 ITERATIVE BLIND DECONVOLUTION
The degraded image g(x, y) is represented by the convolution of the original image f(x, y)
and the point spread function h(x, y).
g(x, y) = f(x, y)∗h(x, y) (3.8)
The equivalent equation in frequency domain is given by
G(x, y) = F (x, y)H(x, y) (3.9)
Iterative blind deconvolution technique [3] uses some general a priori information con-
cerning the original image f(x, y) and the PSF h(x, y). After a random initial guess is
made for the PSF , the algorithm alternates between the image and Fourier domains,
enforcing known constraints in each. The constraints are based on information available
about the image and PSF .
The basic deconvolution method consists of the following steps.
First, a nonnegative-valued initial estimate of the PSF h(x, y) is input into the iterative
scheme. This function is Fourier transformed to yield H(u, v). which is then inverted to
form an inverse filter and multiplied by G(u, v) to form a first estimate of the original
image spectrum F (u, v). This estimated Fourier spectrum is inverse transformed to give
f(x, y). The image-domain constraint of non negativity is now imposed by putting to
zero all pixels of the image f(x, y) that have a negative value. A positive constrained es-
timate f(x, y) is formed that is Fourier transformed to give the spectrum F (u, v). This is
inverted to form another inverse filter and multiplied by G(u, v) to give the next spectrum
estimate H(u, v). A single iterative loop is completed by inverse Fourier transforming
H(u, v) to give h(x, y) and by constraining this to be nonnegative, yielding the next func-
tion estimate h(x, y). The iterative loop is repeated until two positive functions with the
22
Figure 3.3: The sequence of operations to be performed in the IBD algorithm.
required convolution, g(x, y), have been found.
The image-domain constraint of non negativity is commonly used in iterative algo-
rithms associated with optical processing owing to the non negativity property of inten-
sity distributions. The complete image-domain constraint used in this technique not only
forces the function estimate to be positive but also conserves energy at each iteration.
The latter condition is realized by uniformly redistributing the sum of the function’s neg-
ative values over the function estimate.One major problem exists in the above procedure.
If H(u, v) contains very small values this will result in very high values in F (u, v) after
performing the operation G(u, v)./H(u, v).
The Fourier-domain constraint may be described as constraining the product of the
Fourier spectra of f and h to be equal to the convolution spectrum, in agreement with
equation(3.9). It should be noted that, at the kth iteration, two estimates for each Fourier
23
spectrum are available. For example, Hk(u, v) and the estimate Hk(u, v) in the as shown
in figure( 3.3). Both of these estimates have associated properties in common with the de-
sired deconvolved solution. Hk(u, v) has a nonnegative inverse transform, and the second
estimate Hk(u, v) obviously satisfies the Fourier-domain constraint. Therefore at each
iteration the two estimates are averaged to form a composite new estimate as given in
equation(3.11). This averaging is not essential for convergence; however, the convergent
rate is dependent on β, and a method of selecting the optimum value of β has not been
found. Small, confined regions of low or zero value present in G(u, v) are dealt with by
using only the estimate Hk(u, v). The estimate Hk(u, v) contributes no information to
the new estimate. The Fourier-domain constraint can be summarized as follows:
if |G(u, v)| < Td
Hk+1(u, v) = Hk(u, v) (3.10)
if |Fk(u, v)| ≥ |G(u, v)|
Hk+1(u, v) = (1− β)Hk(u, v) + βG(u, v)
Fk(u, v)(3.11)
if |Fk(u, v)| < |G(u, v)|,
1
Hk+1(u, v)=
(1− β)
Hk(u, v)+ β
Fk(u, v)
G(u, v)(3.12)
where 0 ≤ β ≤ 1. Now the problem of Hk(u, v) (or equivalently of Fk(u, v)) having
a small modulus is considered. If the modulus of Hk(u, v) is less than the modulus of
the convolution spectrum, then instead of performing the linear averaging previously
described, the inverses of the two function spectrum estimates are averaged. The new
composite estimate is now the inverse of this average. The rationale behind this averaging
is simply that large function estimate values, obtained when the inverse filter function
has a large value, are prevented from dominating in the average. This is intuitively
reasonable, as large inverse filter values are a consequence of small values of Hk(u, v).
3.3.1 Shortcomings of Iterative Blind Deconvolution
The main disadvantage of iterative blind deconvolution is its computational complexity.
It takes nearly 300 iterations for getting a useful deconvolved image. Also, there is no
convergence criteria defined for IBD. Visual inspection of the deconvolved image has to
be used for stopping the iterative process. Another shortcoming of IBD is that, the
24
convergence of the algorithm is dependent on the initial random estimate of the point
spread function.
3.4 IMPROVED ITERATIVE BLIND DECONVO-
LUTION
An enhanced iterative blind deconvolution scheme is proposed which can solve first two
of the above mentioned shortcomings.
3.4.1 Accelerating the convergence
Extremely high values in the Fourier transform of the estimated image as well as in the
point spread function was observed as one reason behind the slow convergence of IBD.
This problem is overcome by defining cut-off values for limiting the maximum value that
can appear in the Fourier transform of the estimated image and PSF. This significantly
reduces the convergence time.
Also, one more spatial domain constraint is defined for the image. This is the support
information explained in section( 3.1). One property of point spread functions is that,
the sum of all PSF coefficients should be equal to one. This property is used as another
constraint in the spatial domain. After each iteration, the sum of all PSF coefficients
is made equal to one. This is done by dividing each individual value by the sum of all
values. In fact this constraint is observed to be indispensable for getting a satisfactory
deconvolved image. The support information results in a faster convergence.
3.4.2 Definition of Stopping criteria
In the original IBD technique the iterative process it stopped by visual inspection of the
deconvolved image. This is difficult and may result in wrong judgment. To get rid of
this problem, a new stopping criteria based on the image power is defined. Power of the
image is defined as the sum of the square of the individual pixel values in the image. In
each iteration standard deviation of the image power for last n iterations is calculated. If
this standard deviation is less than a particular threshold value TD, then the algorithm
can be terminated. The definition of stopping criteria can be summarized as follows:
25
• Calculate the image power at ith iteration as:
powi =M∑
x=1
N∑y=1
fi(x, y)2 (3.13)
• Calculate the standard deviation SD of the last i− n power values.
• If SD < TD then stop the algorithm. TD is a predefined threshold value.
3.4.3 The algorithm
The improved iterative blind deconvolution algorithm can be summarized as follows:
1. Get the initial PSF h0(x, y) with random values. (Size of the PSF must be known
before starting the algorithm).
2. • Find Hk(u, v) by taking FFT of hk(x, y).
• Limit the high magnitude values within Hk(u, v) as:
If Hk(u, v) > HMAX then Hk(u, v) = HMAX
3. Compute Fk(u, v) as: Fk(u, v) = G(u, v)/H(u, v)
4. Compute the Inverse Fourier transform of Fk(u, v) to get fk(x, y).
5. Impose the image constraints of non negativity and support on fk(x, y) to get
fk(x, y).
6. • Compute FFT of fk(x, y) to get Fk(x, y).
• Limit the high magnitude values within Fk(u, v) as:
If Fk(u, v) > FMAX then Fk(u, v) = FMAX
7. Compute Hk(u, v) as: Hk(u, v) = G(u, v)/Fk(u, v)
8. • Compute IFFT of Hk(u, v) to get hk(x, y)
• Limit the low magnitude values within hk(x, y) as:
If hk(x, y) < hMIN then hk(x, y) = hMIN
• Make the sum of individual values of hk(x, y) equal to one, to get hk(x, y) as:
SUMh =∑m
x=1
∑ny=1 hk(x, y)
hk(x, y) = hk(x, y)/SUMh
26
9. • Compute the power of the image as given in equation 3.13.
• Compute the standard deviation, SD of the power for last t iterations.
• If SD < Threshold STOP the algorithm. Otherwise goto step 2.
3.5 SIMULATION RESULTS
Simulations were carried out by using MATLAB6.5. Different images and different types
of degradations were used in simulation. Some results are given below.
Figure 3.4: (a)Original Image. (b)Motion blured image using 13x23 PSF
27
Figure 3.5: (a)Image Deconvolved using IBD with β = .5. (b)Image deconvolved with
IBD with β = 0.1
Figure 3.6: Results of Improved IBD: (a)Image Deconvolved without support information.
FMAX = 90000, DEV = 7.0e + 006, HMAX = 1, hMIN=.0001. Result obtained after 23
iterations. (b)Image deconvolved using IIBD algorithm with support information and
same parameters as (a). Result obtained after 21 iterations. The PSNR value obtained
is 17.8dB
28
Figure 3.7: Results of Improved IBD: (a)Image blurred with camera defocus PSF of
radius 10. (b)Image deconvolved using IIBD algorithm with support information and
same parameters as figure(3.5). (b)Result obtained after 168 iterations. The PSNR
value obtained is 16.7dB
Figure 3.8: Results of Improved IBD: (a)Image blurred with motion blur of length 21
pixels and angle 45 degree. (b)Image deconvolved using IIBD algorithm with support
information and same parameters as given in figure( 3.5). Result obtained after 29 iter-
ations.
29
Chapter 4
DENOISING
The main challenge in developing denoising algorithms is to remove noise while preserving
image details. Image denoising schemes can be classified broadly into three categories
based on the basic methodologies applied to remove noise.
Filtering without Detection
In this type of schemes, the algorithm doesn’t try to discriminate noisy pixels from non-
noisy pixels. Some operation is defined on the pixels within the filter window. Filter
performs the same operation on all pixels within the image while moving from the first
pixel to last pixel. Median filter is a very good example of filtering without detection.
This kind of filters can successfully filter out noise. But these filters smoothes the edges
within the image because of it indiscriminate filtering.
Detection followed by Filtering
This type of filtering involves two steps. In first step it identifies noisy pixels and in
second step it filters those pixels. Since filtering is performed only on pixels which are
found to be noisy, smoothening of edges will be far less in these techniques.
Hybrid Filtering
In such filtering schemes, two or more filters are suggested to filter a corrupted location.
The decision to apply a particular filter is based on the noise level at the test pixel location
or performance of the filter on a filtering mask.
30
4.1 IMPULSE NOISE DETECTION
A switching filter detects noise before performing filtering and filters a particular pixel
only if it is noisy. Let xij and yij denote pixels with coordinates (i, j) in a noisy and a
filtered image, respectively. If the estimated value of the corrupted pixel xij is ξ(xij), the
switching filter concept can be defined by
yij = Mij.ξ(xij) + (1−Mij).xij (4.1)
where Mij is the binary noise map, with 1s at the positions of pixels detected as noisy
and 0 otherwise. Generally, it is produced by comparing the absolute difference between
the original pixel value xij and some local statistics Ω(xij) with a threshold Td as in
Mij =
1 if |xij − Ω(xij)| ≥ Td
0 if |xij − Ω(xij)| < Td
(4.2)
The problem of finding the optimal Td is fairly complex due to its correlation to the
image contents, noise probability, and noise distribution. Setting Td too high, leaves a
certain portion of the noisy pixels omitted from the noise map. If is too low, image details
will be treated as noise, and the overall image quality will be degraded.
4.2 PERFORMANCE MEASURES
The metrics used for performance comparison of different denoising techniques are defined
below
Peak Signal to Noise Ratio (PSNR)
PSNR analysis uses a standard mathematical model to measure an objective difference
between two images. It estimates the quality of a reconstructed image with respect to
an original image. The basic idea is to compute a single number that represents the
quality of the reconstructed image. Reconstructed images with higher PSNR are judged
better. Given an original image F of size M × N pixels and a reconstructed image F ,
the PSNR(dB) is defined as:
PSNR(dB) = 10 log10
2552
1MN
∑Mi=1
∑Nj=1
(Yij − Yij
)2
(4.3)
31
Percentage of Spoiled Pixels (PSP )
PSP is a measure of percentage of non-noisy pixels change their gray scale values in the
reconstructed image. In other words it measures the efficiency of noise detectors. Hence,
lower the PSP value better is the detection, in turn better is the filter performance.
PSP=((Number of non-noisy pixels changed their gray value)/(total number of non-
noisy pixels))x100.
Percentage of Hidden Noise(PHN) and Percentage of Faulty Detection(PFD)
Percentage of Hidden Noise is the ratio of Number of undetected noisy pixels to total
number of noisy pixels. For an ideal filter PHN should be zero. PHN=((Number of
undetected noisy pixels)/(Total number of noisy pixels))× 100
Percentage of faulty detection is the ratio of the number of pixels wrongly classified
as noise to the total number of non-noisy pixels. PFD=((Number of pixels wrongly
detected as noise)/(Total number of non-noisy pixels))× 100.
4.3 IMPULSE NOISE DETECTION BASED ON
PIXEL WISE MAD
The main drawback of the median filter is that it modifies pixels not contaminated by
noise, thus removing fine details in the image. Therefore, contemporary switching im-
pulse noise filters split the filtering process into two steps: (1) detection of impulses and
(2) replacement of impulses with estimated values, where the median is commonly used
as the estimator. Detection schemes are based on various concepts: weighted median ,
rank-order thresholding, local signal statistics , fuzzy reasoning , neural networks ,etc.
The impulse detection concept in pixel-wise MAD, does not require optimizing param-
eters or previous training. Still, it removes impulse noise very efficiently, while preserving
image details. Since both of its steps, detection and estimation, are based on the same
median structure, only a simple median filter is required for practical realization. There-
fore, the algorithm complexity is equivalent to that of the median filter.The median of
the absolute deviations from the median -MAD, is used to estimate the presence of image
details, thus providing their efficient separation from noisy image pixels. An iterative
32
pixel-wise modification of MAD (PWMAD) provides reliable removal of arbitrarily dis-
tributed impulse noise.
Most of the known filtering schemes exhibit satisfactory results for salt and pepper
noise. However, few of them perform well for random valued impulse noise. PWMAD
gives better results for images corrupted with random valued impulse noise.
Let xij,mij and dij represent pixels with coordinates (i, j) of the noisy image, median
image and absolute deviation image, respectively. Also let Xij,Mij, Dij denote matrices
whose elements are pixels of the corresponding images contained within the (2K + 1)×(2K +1) sized filter window centered at position (i, j). The PWMAD filtering procedure
can be described as given below.
1. Find the pixels of the median image as
mij = median(Xij) (4.4)
2. Find the pixels of the absolute deviation image as
dij = |xij −mij| (4.5)
3. Find MADij as
MADij = median(|Xij −median(Xij)|= median(|Xij −mij|) (4.6)
4. Compute PWMADij as
PWMADij = median(Dij) (4.7)
= median(|Xij −Mij|)
Intuitively we can see that MAD is the median of deviations of pixel values within a filter
window. So this can be used as a measure of local variance within the filter window.
Higher variance usually is a result of the presence of edges in that area. PWMAD
represents a good estimate of the local variance in the presence of impulse noise, i.e., the
image details. The absolute deviation image consists of noise and image details eliminated
from the noisy image by median filtering. If the details are extracted, only noise remains
33
and an accurate noise map can be generated. In order to make the robust estimate of
image details, the PWMAD image is computed. By subtracting the PWMAD image from
image dij, details are reduced while most of the noise remains. Since the subtraction is
performed through the absolute value, positive and negative impulses are treated equally.
The whole procedure can be described in an iterative manner as
dk+1ij = |dk
ij −median(dnij)| (4.8)
where d0ij is the initial deviation image. In each step, certain portions of details are elim-
inated, while noise remains. Due to the fact that most of the details are eliminated after
N steps, the noise map is obtained by comparing with a threshold close to zero.
NMAPij =
1 if dNij > Td
0 if dNij < Td
(4.9)
4.4 DOUBLE DERIVATIVE BASED IMPULSE DE-
TECTION
A double derivative based impulse noise removal scheme essentially involves two steps.
1. Detect the presence of an impulse.
2. Replace the pixel by a suitable gray scale value if a pixel is found to be corrupted.
The row wise double derivative can be expressed as:
D2R =
D21,1 D2
1,2 D21,3
D22,1 D2
2,2 D22,3
D23,1 D2
3,2 D23,3
(4.10)
where D2i,j = Di + 1, j −Di, j where Di,j is the single derivative, defined as
Di, j = xi + 1, j − xi, j
The following conclusions can be drawn from the above operations.
• D2i,j is a high magnitude negative quantity if xi+1,j is corrupted by a high value
impulse noise.
34
• D2i,j is a high magnitude positive quantity if xi+1,j is corrupted by a low value
impulse noise.
As detecting the presence of an impulse is more important than its value abs(D2i,j) can
be used. But the above method fails if impulses appear continuously in a row, in which
case the double derivative does not attain a high magnitude. This shortcoming can be
overcome by applying the column wise double derivative after applying the row wise
double derivative. A noise map indicating the presence of noisy pixels is obtained after
finding the column wise and row wise double derivative. This noise map is used by a
median filter to selectively filter only the noisy pixels.
4.5 ITERATIVE NOISE FILTERING WITH EDGE
RETRIEVAL
A new simple scheme for impulse noise removal is proposed. The scheme is based on cal-
culating the deviation of values in a straight line passing through the center pixel of the
filtering window. The fundamental concept is similar to the double derivative method.
But the double derivative method is performed only horizontally and vertically. The
diagonal neighbours are not considered in the double derivative.
Another improvement of the algorithm is the edge detection which follows the noise
detection. In the edge detection phase one unique characteristic of edges is used to re-
trieve edges which are wrongly detected as noise. The graph given in Fig.4.1 below is
completely illustrative of the similarity as well as differences between edges and noise.
It can be seen that edges are also characterized by sudden change of pixel values
like noise, but unlike noise the width of the spike in case of an edge is much wider
compared to that of a noise. That means, If we take the differences d1 = xi,j −xi,j−1 and
d2 = xi,j − xi,j+1
• For noisy pixel xi,j, both d1 and d2 will be of high magnitude.
• For edge pixel xi,j, only one among d1 and d2 will be high of high magnitude.
35
0 50 100 150 200 250 3000
50
100
150
200
250
x
inte
nsity
val
ues
0 50 100 150 200 250 3000
50
100
150
200
250
x
pixe
l int
ensi
ty v
alue
s
Figure 4.1: (a)Plot of pixel values in a horizontal line of a non-noisy image. (b) The plot
for a noisy image.
Also,images taken by a camera contain edges which are slightly blurred due to the
imperfections of the imaging device. In that case edges are not abrupt variations instead
the variations are gradual. This fact is used to find out pixels which are wrongly classified
as noisy. The noise map is modified according to this information.
The proposed algorithm is designed as an iterative scheme in which the image is first
filtered with a 3 × 3 filter window and then filtered with a 5 × 5 filter window. The
threshold value used to decide the presence of noise is initially set to a high value and it
is decreased in each iteration. This increases the probability that a noisy pixel is replaced
with a value so close to its original value.
4.5.1 Edge retrieval based on single derivative
According to the property of edges depicted in Fig.( 4.1) the pixel values gradually
increase or decrease while moving through an edge. That is, if the first difference is taken
on the edge pixels, all the values will be of the same sign. This fact is used to retrieve
edges after the noise detection phase. First derivative is calculated in a 5 × 5 window,
instead of calculating for the whole image. This makes it possible to calculate the first
difference through all the lines passing through the middle pixel of a window, instead of
just horizontal and vertical.
36
4.5.2 Edge retrieval based on pixel wise MAD
It has already been observed that PWMAD has very good edge preservation charac-
teristics. So PWMAD can be used as a statistic for retrieving edge pixels after noise
detection. It is observed that PWMAD based edge detection outperforms the first deriva-
tive based method both in PSNR as well as visual quality.
4.5.3 The Algorithm
The algorithm for Iterative noise filtering and edge retrieval can be summarized as follows:
1. • Initialize filter window size WSZ = 3× 3.
• Initialize a threshold TD1. (70 is a good initial value).
• If xi,j is the center of the filter window, there are four straight lines passing
through that pixel.
Find the difference between each end pixel and xi,j for each line.
• Count the number of lines which satisfies the noisy pixel criteria explained in
section( 4.5).
• If the the count CNT >= K1 then mark the pixel as noisy. (K1=3 gives good
results).
• Make the noise map for all pixels.
• Compute PWMAD as given in equation( 4.7).
• Compute Ei,j = abs(Di,j − PWMADi,j). If Ei,j > TD3 mark the pixel xi,j as
non-noisy.
• Decrease the value of TD1 as TD1 = TD1− C1
• Repeat step 1 until TD1 < t
2. Perform the operations mentioned in STEP 1, with window size changed to 5x5.
Use different threshold values TD2, (TD2 = 30 gives good result);
4.6 SIMULATION RESULTS
37
Figure 4.2: (a)Original Lena image of size 256× 256. (b) Lena image with 15% random
valued impulse noise added to it.
Figure 4.3: (a)Median filtered Image with 15% noise. PSNR=28.35dB(b) Image filtered
using double derivative method with Threshold 50. PSNR=28.94dB
38
Figure 4.4: (a)Lena image filtered using pixel-wise MAD algorithm. PSNR=28.75dB (b)
Image (a) after processing by passing the output of PWMAD through the first phase of
the proposed algorithm. PSNR=30.16dB
Figure 4.5: (a)Image filtered using the Two-phase scheme with single derivative used
for edge retrieval. PSNR=30.18dB (b) Image filtered using the Two-phase scheme with
PWMAD used for edge retrieval. PSNR=31.42dB
39
Chapter 5
CONCLUSIONS AND FUTURE
WORK
5.1 CONCLUSIONS
For deconvolution, non parametric techniques give promising results. But use of image
constraints alone does not seem to yield a satisfactory restored image. That may be the
reason for the unsatisfactory performance of the NAS-RIF algorithm. Also in the NAS-
RIF algorithm the minimization of the cost function always does not mean a meaningful
solution.
The Iterative Blind deconvolution scheme shows some promise, even though quality of
the restored image is not satisfactory. The main disadvantage of IBD is its slow conver-
gence rate. The convergence can be accelerated by limiting the high magnitude values in
the frequency domain of estimated image as well as the estimated PSF. It was observed
that forcing the sum of the PSF coefficients at the end of each iteration improves the
quality of the restored image to a great extend. The original IBD algorithm didn’t have a
stopping criteria. A new stopping criteria can be defined based on the standard deviation
of image power for previous k iterations. The values of the initial PSF does not have any
impact on the performance of the algorithm. But the size of the PSF must be known
before starting the algorithm.
A priori algorithms for blind deconvolution was also observed to give promising results.
40
In the cepstrum based motion blur estimation scheme, some excellent results can be
obtained for a limited domain of motion blur.
In denoising, pixel-wise MAD was observed to be a very good measure of the edges
in the image. So, this statistic can be used for an edge retrieval scheme which follows
the noise detection. The edge retrieval scheme improves the PSNR values as well as the
detail preservation characteristics of a denoising algorithm.
5.2 LIMITATIONS AND FUTURE WORK
The Improved Iterative deconvolution scheme works well on binary images. But the
performance is not satisfactory on gray scale images. This is one area which needs im-
provement. Also, the algorithm shows variations in performance for different types of
point spread functions. These characteristics has to be more thoroughly studied and a
more general scheme has to be worked out. A more serious limitation is the requirement
that the size of the PSF must be known in advance. Some scheme has to be developed
which can at least guess the PSF with some accuracy and later refine it to estimate the
size accurately. A priori blur identification techniques may be used for this purpose.
In the denoising based on edge retrieval, the results of the scheme are satisfactory
except the time complexity of the scheme. A scheme to adaptively change the threshold
values may be designed which can improve the performance.
41
Bibliography
[1] “A switching median filter with boundary discriminative noise detection for ex-
tremely corrupted images,” IEEE Transactions on Image Processing, 2006.
[2] N. Alajlan and M. Kamel, “Detail preserving impulsive noise removal,” Signal Pro-
cessing: Image Communication, 2004.
[3] G. R. Ayers and J. C. Dainty, “Iterative blind deconvolution method and its appli-
cations,” Optics Letters, vol. 13, no. 7, 1988.
[4] J. Biemond, A. M. Tekalp, and R. L. Lagendijk, “Maximum likelihood image and
blur identification: a unifying approach,” Optical Engineering, vol. 29, May 1990.
[5] T. Chen and H. R. Wu, “Adaptive impulse detection using center-weighted median
filters,” IEEE Signal Processing Letters, vol. 8, no. 2, January 2001.
[6] D. G. Childers and D. P. Skinner, “The cepstrum: A guide to processing,” IEEE
Proceedings, vol. 65, no. 10, October 1977.
[7] V. Crnojevic and Z. Trpovski, “Advanced impulse detection based on pixel-wise
mad,” IEEE Signal Processing Letters, vol. 11, no. 7, July 2004.
[8] R. Fergus and B. Singh, “Removing camera shake from a single photograph,” MIT
CSAIL, Tech. Rep., 2004.
[9] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Prentice-Hall
India, 2005.
[10] N. Ishihara and S. Komatsu, “Blind recovery of truncated blurred image using adap-
tive masking method,” May 2001.
[11] F. Krahmer, Y. Lin, and D. Widemann, “Blind image deconvolution: Motion blur
estimation,” August 2006.
42
[12] D. Kundur and D. Hatzinakos, “Blind image deconvolution,” IEEE Signal Processing
Magazine, May 1996.
[13] ——, “A novel blind deconvolution scheme for image restoration using recursive
filtering,” IEEE Transactions on Signal Processing, vol. 46, no. 2, 1998.
[14] ——, “Robust classification of blurred imagery,” IEEE Transactions on Image Pro-
cessing, vol. 9, no. 2, February 2000.
[15] R. Lokhande and K. V. Arya, “Identification of parameters and restoration of motion
blurred images,” Indian Institute of Technology, Kanpur, Tech. Rep., 2003.
[16] Z. Lou, “Blind deconvolution using recursive filtering,” Master’s thesis, University
of Illinois at Urbana-Champaign, April 2005.
[17] Z. Mou-yan and R. Unbehauen, “An iterative method of blur identification and
image restoration,” Institute of Electronics Academia Sinica, Beijing, China, Tech.
Rep., 2002.
[18] G. Panda and S. K. Mahapatra, “Efficient filtering of image data corrupted by
impulse noise.”
[19] W. H. Press and B. P. Flannery, Numerical Recipes in C, 2nd ed. Cambridge, 1992.
[20] P. K. Sa, “On the development of impulsive noise removal schemes,” Master’s thesis,
NIT Rourkela, May 2006.
43