chapter 3 - hexagonal processing based interpolation...

79

CHAPTER 3 - HEXAGONAL PROCESSING BASED INTERPOLATION

AND ENHANCEMENT

3.1 Introduction

Image reconstruction through interpolation is a routine task during

processing all transformation that is made on an image. Such transformations

include scaling, rotation, registration and edge detection. Due to the interest in

human vision, some of the image acquisition systems are using hexagonal laid

pixels like imaging radars and nuclear medicine [16, 76 -78]. They also possess

better topological and geometrical properties resulting in a more efficient signal

representation in two dimensions.

Numerous works are available for interpolation in rectangular lattices

using parameterized wavelets and piecewise polynomial functions. B-splines is

one of the most important interpolation functions [53]. B-Spline methods enable

approximation in a simple way using the orthogonal properties of it. In

hexagonal grid three methods namely, box splines, hex-splines and generalized

splines of maximal order and minimum support were proposed [45, 53, 82, 83,

84-88]. The first one exploits the six fold symmetry and the second twelve fold

symmetry. However, the third one is a generalization to get best results out of

an interpolation with a tradeoff between the interpolated result and

computational complexity. This is achieved from error minimization using the

spectral components with an assumption that the signal energy is concentrated

over the low-frequency region.

The conventional interpolation techniques such as B-splines and H-

splines are reviewed in this section. The performance of B-spline was tested on

the hexagonal sub sampled image obtained using the half pixel shift approach.

The advantages and disadvantages are addressed based on the experimental

analysis. Hex-spline is a framework necessary for the regular hexagonal

arrangement, for which another alternative is only the spiral addressing scheme.

Therefore a change of platform was found necessary for utilizing the results in

conventional Image Processing. Spline interpolation is used through Hex-

splines for the purpose of shear interpolation by earlier workers [45, 82, 89].

Hexagonal interpolation may also used to gather mutual information between

three neighboring pixels and in three directions of the hexagonal lattice, if intra-

80

pixel information is resulting from directional filtering effects. With the same

structure, image enhancement and interpolation are attempted with Gabor filter

using the three directional filtering. From this, for regular lattices, Hex-Gabor is

proposed and is addressed in section 3.7.

3.2 Re-sampling and Interpolation

Re-sampling is the process of transforming a discrete image which is

defined at one set of coordinate locations to a new set of coordinate points. Re-

sampling is used for several different purposes in image processing: In remotely

sensed data, Image registration with different sensors [83], medical imaging,

image warping, photo mosaicing [64, 91, 92] etc., Re-sampling can produce

approximations at either lower or higher resolutions than that of the original

data, thus enabling to multi-resolution signal processing schemes [93, 94]. In

the case of hexagonal image processing, re-sampling can be done in two ways

(i) To represent acquired image using regular hexagonal pixels (ii) Taking the

samples to lie on an irregular hexagonal grid. There may be loss of image

quality due to re-sampling [91]. To preserve image quality a suitable

interpolating function should be used.

The choice of an interpolating function to be used for re-sampling

depends upon the task being performed. An understanding of the frequency

domain response of the interpolating functions may help in the choice of an

interpolating function [87, 88], especially when the re-sampling is used prior to

further image processing.

For a 2-D image, it is customary to apply the B-splines [52, 53, 54]

successively in 1-D along horizontal and vertical directions on rectangular

lattices [48, 49]. Hex-splines [45, 82] are proven for their quality for image

interpolation by re-sampling techniques on hexagonal lattices. Hex-splines were

already successfully applied to some printing applications [89]. Hex-spline by

Petrou [95], proposed bio-inspired algorithm for invariant image reconstruction

from irregular samples and used linear B-splines with control points on

hexagonal grids.

3.3 Implementation of resampling

In general the resampling can be divided conceptually into two

processes: interpolation of the discrete image to a continuous image and then

81

sampling the interpolated image (Figure 3.1). Frequently, resampling is used to

increase the number of points in an image to improve its appearance for display.

The process of interpolation is fitting a continuous function to the discrete

points in the digital image. This continuous function can then be sampled at

whatever points are necessary. In implementing resampling, interpolation and

sampling are often combined so that the signal is interpolated at only those

points which will be sampled [91].

Figure 3.1 Implementation of resampling [91]

From Figure 3.1, resampling can be thought of as interpolation from a discrete

to a continuous function followed by sampling (top row).The interpolation is

performed by convolving the signal, x[n], with a continuous interpolating

function, h(t). Sampling is performed by multiplying the interpolated signal by

a comb function. This process is, however, usually implemented by convolving

with a sampled interpolating function, h[n] (bottom row) which is equivalent to

interpolating and then sampling.

3.4 Interpolation Techniques – A Review

3.4.1 Ideal Interpolation

Following the sampling theory, the scanning of a continuous image

),( yxs yields infinite repetitions of its continuous spectrum ),( vus in the

Fourier domain, which do not overlap since the Nyquist criterion is satisfied.

82

Only then, the original image ),( yxs can be reconstructed perfectly from its

samples by multiplication of an appropriate rectangular prism in the Fourier

domain. The 1-D ideal interpolation equals the multiplication with a rectangular

function in the Fourier domain and can be realized in the spatial domain by a

convolution with the sinc function

)(sin

)sin()( xc

xx

xhideal ==ππ

(3.1)

Figure 3.2 Ideal interpolation (a) Kernel plotted for 3<x . (b) Magnitude of

Fourier Transform [96].

Figure 3.2(a) shows the ideal Infinite Impulse Response (IIR)

interpolator )(xhideal . The plot was truncated within the interval -3 < x < 3.

The magnitude )(ωidealH of the infinite kernel )(xhideal is plotted within

the interval ππωπ 424 ≤=≤− f (Figure 3.2(b)). The interval πωπ <<−

is called pass band and f = 1/2 or πω = is the cutoff point or Nyquist

frequency. The transfer function of the ideal interpolator is constant and one in

the pass band. Some fundamental properties such as smoothness or regularity

and vanishing moments can be derived from this ideal interpolator. Also, by

taking Fourier Transform of the ideal interpolator, the following Fourier

properties can be derived: 1) Deviation from the ideal constant gain within the

passband 2) The amplitude and slope of the kernel’s Fourier transform at the

83

cutoff frequency and 3) The occurrence and the amplitudes of ripples and

sidelobes in the stopband can be derived [96].

The ideal interpolation function )(xhideal is positive from zero to one,

negative from one to two, positive from two to three, and so on. For 1)0( =h

these zero crossings guarantee that the image is not modified if it is resampled

on the same grid. Therefore, kernels satisfying

=≡

≡

,.....2,1,0)(

1)0(

xxh

h

(3.2)

avoid smoothing and preserve high frequencies. They are called interpolators.

3.4.2 Nearest Neighbor Interpolation

The easiest interpolation algorithm to implement is the so-called nearest

neighbor algorithm, where each pixel is given the value of the sample which is

closest to it. This method interpolates the sampled image by convolving it with

a rectangle function and it is defined as

<≤

=.,0

5.00,1)(1

elsewhere

xxh

(3.3)

Convolution with a rectangle function in the spatial domain is equivalent to

multiplying the signal in the frequency domain by a sinc (sin (ω)/ω) function.

The sinc function is a poor low pass filter since it has prominent side lobes.

Therefore, the nearest neighbor algorithm has a poor frequency domain

response [96].

The nearest neighbor algorithm causes the resampled image to be shifted

with regard to the original image by the difference between the positions of the

coordinate locations. For example, the locations of the resampled points are half

way between the original points; the image will be shifted by one-half pixel.

This shift means that the nearest neighbor algorithm cannot be used when it is

necessary to preserve sub pixel image relations. Furthermore, the nearest

neighbor algorithm fails completely when resampling to a large image size

since the pixel values are merely replicated. There are some special properties

of the nearest neighbor algorithm. If the resampling is done on a coordinate

system with the same spacing as the original coordinate system, then, except for

84

the shift, the resampled data exactly reproduces the original data. That is, the

difference between the frequency spectrum of the original and the resampled

images is a pure linear phase shift.

3.4.3 Linear Interpolation

A slightly more complicated function to calculate is the linear

interpolation algorithm. Linear interpolation amounts to convolution of the

sampled image by a triangle function and it is defined as

<≤−

=.,0

10,1)(2

elsewhere

xxxh

(3.4)

This function corresponds in the frequency domain to a modestly good low-pass

filter. It attenuates frequencies near the cut-off frequency resulting in smoothing

of the image, and passes a significant amount of energy above the cut-off

frequency. Sampling an image with bilinear interpolation will cause the data

above the cutoff which has been passed by the interpolating function to be

aliased into the low frequencies [96].

3.4.4 Cubic B - spline Interpolation

B-splines are several convolutions of the rectangular function. The cubic

B-spline is four convolutions of the simple rectangular function. The B-splines

have an extent which is appropriate for interpolation over the two nearest

neighbors in each direction. They are reasonably good low-pass filters.

However, they are positive in the whole interval from 0 to 2; therefore, they

smooth somewhat more than is necessary below the cut-off frequency. They

have good efficiency in the stop band. Since cubic B-splines are symmetric,

they only need to be defined in the interval (0, 2). Mathematically, the cubic B-

spline can be written as [91],

(& ) '& * ' '' * + , -- . / -' * - & , -- . '+' , -- 0 (3.5)

When resampling using an interpolating function which extends over four

points, if the nearest point is at an offset d, where d is in the interval (0, 1), four

85

samples of the interpolating function will be used: f(d), f(1 - d), f(l + d) and f(2 -

d). For the cubic B-spline, the sum of the values of these four points for any

value of d is 1. Namely, for any offset the sum of the sampled interpolation

function points is equal to 1. Therefore, the DC amplification, the gain on the

interpolating function, will be unity. This remarkable property is quite valuable

when resampling is done because of geometric distortion or rotation. In these

cases, the offset is different at different points in the image.

3.4.5 Two Dimensional Interpolation

For picture processing, the one-dimensional interpolating functions must

be transformed into two-dimensional functions [96]. The general approach is to

define a separable interpolation function as the product of two one-dimensional

functions. Separability is attractive for implementation. In two dimensional

case, the interpolation step must reconstruct a two dimensional (2-D)

continuous signal ),( yxs from its discrete samples ),( lks with ℜ∈yxs ,, and

0, Nlk ∈ . Thus the amplitude at the position ),( yx must be estimated from its

discrete neighbors. This can be described formally as the convolution of the

discrete image samples with the continuous 2-D impulse response ),( yxh of a 2-

D reconstruction filter.

∑ ∑ −−=k l

lykxhlksyxs ),(),(),(

(3.6)

Usually, symmetrical and separable interpolation kernels are used to reduce the

computational complexity.

)()(),( yhxhyxh = (3.7)

Figure 3.3 illustrates the interpolation of the point ),( yx in a 44 ×

neighborhood. Interpolation is performed in the x direction first. The small grey

intermediate points are generated by four one dimensional interpolations. They

are used for the final one dimensional interpolation in the y direction.

86

Figure 3.3 One dimensional decomposition of the 2-D, N x N interpolation of

the point (x, y) [96]

3.4.6 Hex-Spline Interpolation

Hex spline is a new family of bivariate, non separable spline especially

designed for hexagonal lattices [45]. The starting point of the construction is the

indicator function of the Voronoi cell, which is used to define the first order hex

splines in a natural way. Higher order hex splines are obtained by successive

convolutions. 2-D periodic lattices are characterized by two (linearly

independent) vectors r1 and r2 (Figure 3.4).

Figure 3.4 (a) The regular hexagonal lattice and its Voronoi cell (b) The dual

lattice [45]

Any integer combination 2211

rkrkrk

+= of these vectors, points to a

lattice site. It is often convenient to group the lattice vectors r1 and r2 in a matrix

R = [ r1 r2] such that the lattice sites are given by Rk , where k = [ k1 k2 ] T .

Regular hexagonal lattice is characterized by matrices

87

.

10

33

332

ˆ

121

02

3

=

−

= RR

(3.8)

A well-defined and unique tiling cell is the Voronoi cell, which contains all

points that are closer to their lattice site than to any other site. The indicator

function for the Voronoi cell of the origin is defined as

∉

∈

∈

=

cellVoronoix

cellVoronoitheofedgexm

cellvoronoix

xR x,0

,/1

,1

)(χ . (3.9)

where mx is the number of lattice points to which x is adjacent. Note that )(xRχ

tiles the plane by definition. The dual or reciprocal lattice is specified by

TT RRrrR −− === )(]ˆˆ[ˆ 121 . According to [45], first-order hex-spline is defined

as )()(1 xx Rχη = . Hex splines of higher orders are constructed by successive

convolutions.

1,

)()(

*11 ≥

Ω=+ pX

Xp

p

ηηη

(3.10)

and the construction rule of hex spline is

xRkxk

p ∀=−∑ ,1)(η (3.11)

which holds for any order p. The hex spline basis functions are obtained by

shifting pη to each lattice site Rk.

3.4.6.1 Hex Splines Construction-Lattice Generating Functions:

88

For the generation of hex spline, a method described by M. Unser et.al [45]

is followed. Because of the three edge orientations (i.e., θ = 0, π/3, and – π/3) of

hexagon, two generating functions 0/)(Χ and 0

\)(Χ are considered as shown

below in Figure 3.5. The next is to localize these functions in order to obtain

the indicator function of the hexagon. The following steps were used to generate

the basic Voronoi cell.

(a) Place both generating functions on the outer left vertex of the hexagon.

(b) Create the horizontal edges by subtracting 0/X at the upper left vertex and

0\X at the lower left vertex (Figure 3.6)

(c) Form the right-hand edges by subtracting this time 0/X at the upper right

vertex and 0\X at the lower right vertex (Figure 3.6(a)).

(d) Compensate the dark region of figure by putting both the generating

functions at the outer right vertex (Figure 3.6(b)).

Figure 3.5 (a) Generating functions 0/)(Χ and 0

\)(Χ (b) Creation of horizontal

edges [45]

89

Figure 3.6 Formation of edges (a) Formation of right-hand edges (b)

compensation of dark region [45]

Thus the first-order hex-spline can be obtained by localizing the generating

functions 0/)(Χ and 0

\)(Χ

0\)(*\

0/)(*/)(1 XXX ∆+∆=η

(3.12)

where each generating function has been placed on four vertices through the

application of localization operators 0/)(Χ and 0

\)(Χ and are given by

( ) ( )n

n eeee

−∆+∆↔∆∧∧

ωω ,, 2121/ (3.13)

( ) ( )n

n eeee

−∆+∆↔∆∧∧

ωω ,, 3131\ (3.14)

Where )2/exp()2/exp()(ˆ ωωω jj −−=∆ is the classical 1-D B-spline

localization operator. The vectors e1, e 2 and e3 are defined as

.

21

63

,

21

63

,

0

33

321

−

−=

−

=

−

= eee

(3.15)

The higher order hex-splines were constructed by successive convolutions. For

the classical B-splines, we need formulas for the multiple convolutions between

the localization operators on one side, and the generating functions on the other

side.

The higher order hex spline is defined as

1)()(*1

1 ≥Ω

=+ pforxxp

p

ηηη and

90

( ) ( )( ) ( )( )

( )( )( )kpkp

uuukpk

p

kk

pp

kpuu

kuu

kpkp

kk

pp

xp

−−−−Χ−∆∆=

−Ω

=

−−Χ−Χ−∆∆=

−Ω

=

∑

∑

1,1,1,,*\*/

01

1

1,*1

,*\*/0

11

321

3121

η

(3.16)

Where .23

)det( ==Ω R For 1≥p , the generating functions to the power -1

neutralize the convolution, i.e.,

( )( ) ( )( ) ( )Χ=Χ=Χ −− δ1,

1, 3121 uuuu

(3.17)

Where 1u , 2u and u3 are the reciprocal vectors of e1, e 2 and e3 respectively.

Construction of hex-spline was performed in MAPLE software by Van

De Ville et.al. [45]. In this work, MATLAB software was used for simulation to

do further analysis and performance comparison with proposed Hex-Gabor.

Hex-spline kernels in spatial domain and frequency domain were constructed

and used for the applications such as (i) Tiling of the image in the form of

hexagon (ii) Enhancement of the image.

3.5 Performance Measures

The performance of image enhancement of Hex-Gabor kernel was compared

with the hex-spline kernel and the results are discussed in the section 3.6.

Performance measures used in this work are Mean Squared Error, Peak Signal-

to-Noise Ratio (PSNR) and computation time.

3.5.1 Mean Squared Error (MSE)

MSE indicates the average difference of the pixels throughout the image. A

higher MSE indicates a greater difference between the original and processed

image. The formula for the MSE calculation is given as:

91

∑ ∑ −=

i jijVijX

NMSE 21

(3.18)

Where N is the size of the image, V is the processed image, and X is the original

image.

3.5.2 Peak Signal-to-Noise Ratio (PSNR)

( )

−=

∑ ∑i j

ijij vX

NPSNR

2

2

10255*

log.10 (3.19)

PSNR is used for quantitative comparison.

3.6 Results and Discussions

3.6.1 Implementation Results of Interpolation Kernels

Initially, the interpolation kernels such as nearest neighbor, bilinear and

cubic B-spline (Section 3.4) were generated using MATLAB as shown in

Figure 3.7. Separability property was used by convolving the rows of the image

with the interpolating function and again convolving columns of the resultant

image with the same interpolation function. From the results, it was found that

as the order increases the side lobe magnitude is decreasing.

92

Figure 3.7 Interpolation functions in (a), (c), (e) spatial domain and (b), (d), (f)

Frequency domain, using nearest neighbor, linear and cubic B-spline

respectively.

The performances of the generated kernels on some standard images were

analyzed. For example, consider the image of Figure 3.8(a) which is a poor

resolution image and its hexagonal sub sampled image was obtained using half

pixel shift method. The interpolation results using nearest neighbor, bilinear and

cubic B-spline kernels are shown in Figure 3.8 (b), (c) and (d) respectively.

Figure 3.8 Effect of interpolation kernels (a) Original image (b) - (d) Images

after convoluting with nearest neighbor, linear interpolation and cubic B-spline

respectively.

3.6.2 Performance comparison of Interpolation kernels

The performance of the generated interpolation functions was compared in

terms of PSNR, MSE and computation time on three images (hexagonal

sampled) namely Cameraman, Cat and Lena. From the results, it was observed,

that PSNR value increases as the order i

order increases. But, computation time increases as the order increases which is

a disadvantage found in this method. This limitation can be overcome by using

Hex-spline.

(a) Mean Squared Error

(b) Peak Signal –

Figure 3.9 Performance measures of the interpolation kernels (Nearest neighbor,

linear and cubic spline) (a)

(c) Computation time (in seconds)

0.05

0.1

0.15

0.2

0.25

0.3

0

10

20

30

40

50

60

70

0

10

20

30

40

50

60

93

3.6.2 Performance comparison of Interpolation kernels




that PSNR value increases as the order increases. MSE value decreases as the



(a) Mean Squared Error

Noise Ratio

( c ) Computation time (in seconds)


linear and cubic spline) (a) Mean Squared Error (b) Peak signal

(c) Computation time (in seconds)

0

0.05

0.1

0.15

0.2

0.25

0.3

Cat Lena Camera

Nearest

Linear

Cubic

Cat Lena Camera

Nearest

Linear

Cubic

Cat Lena Camera

Nearest

Linear

Cubic




ncreases. MSE value decreases as the



( c ) Computation time (in seconds)


ared Error (b) Peak signal – Noise Ratio

Nearest

Linear

Cubic

Nearest

94

3.6.3 Implementation results of Hex-spline

Hex splines are most valuable kernels to represent the hexagonal tiles on the

image. As explained in the section 3.4.6, Hex-spline kernels for the order 1, 2

and 3 in spatial and frequency domain were generated and plotted in the Figure

3.10.

Figure 3.10 (a),(c),(e) Hex splines in spatial domain (b),(d),(f) Hex splines in

frequency domain for first, second and third order respectively.

95

3.6.4 Applications of Hex – Spline : Representation

Figure 3.11 shows the image Representation using hex splines of order p = 1, 2

and 3 for the barbara image. For performing Representation, the image pixels

are grouped equivalent to kernel size which contains the non zero values and

replaces all the pixel values with the same intensity value. It will tile the image

in the form of hexagons (Figure 3.11 (b)). To be more clearer, part of the image

(cat image – top left) of Figure 3.8 with hex-spline tiling is shown in Figure

3.11(f).

96

Figure 3.11 (a) Original Image (b) Tiling/Image representation by the hex spline

(c) - (e) Interpolated image with hex splines of order p = 1, 2 and 3 respectively

(f) Part of the image (Figure 3.8-top left ear of the cat image) to highlight the

tiling of hex – splines.

From the results, it was found that as the order of the hex spline increases, there

is an improvement in the image quality. Performance results are summarized in

Table 3.1 for the kernels of order 1, 2 and 3 respectively. The performance of

Hex-spline was found to be better than that of B-splines in terms of improved

PSNR and lower MSE.

Table 3.1 Performance measure of Hex-spline for order 1, 2 and 3

Image Type First order hex -

spline

Second order hex -

spline

Third order hex -

spline

PSNR MSE PSNR MSE PSNR MSE

Cat.jpg 57.3383 0.0973 72.5959 0.0036 72.7954 0.0034

Lena.png 59.1207 0.0796 68.4469 0.0093 68.8241 0.0085

Barbara.png 58.4814 0.0922 67.2120 0.0124 67.5346 0.0115

Peppers.png 60.0152 0.0648 70.9642 0.0052 71.4716 0.0046

97

3.7 Enhancement and Interpolation using Hex-Gabor Kernel

3.7.1 Overview of Gabor Filters

Gabor filters have desirable properties for picture analysis and feature

extraction: They are selective in space, spatial frequency and orientation

achieving the theoretical limit for conjoint resolution in the spatial and spatial

frequency domain [42]. Therefore, they have been widely used in these fields in

recent years [39, 97]. Those filters were also used to describe the behavior of

simple cells in area V1 of the human visual cortex, which has turned out to be

very successful [98]. Hubel and Wiesel [6] found simple cells in a monkey’s

visual cortex, which was sensitive to frequency and orientation of an image

perceived. Experiments revealed that a Gabor filter takes the form of a Gaussian

modulated complex sinusoid in the spatial domain. The Gaussian has a centre

frequency (u, v), two spatial extent parameters (σx , σy ) and the filter is rotated

through an angle of θ degrees about the origin. In the spatial domain the filter is

a Gaussian modulated sine wave grating. The spatial extent parameters of the

Gaussian are inversely proportional to σx and σy and the sine wave grating has a

frequency 22 vu + and orientation θ. These five parameters (σx , σy, u, v, θ )

uniquely determine the filter and are shown in Figure 3.12.

Figure 3.12 Five parameters defining each filter in the frequency domain

98

The standard definition of a two dimensional Gabor function used in this work

is from B. S. Manjunath and W. Y. Ma [38]. A two dimensional Gabor function

g(x, y) and its Fourier transform G(u ,v) can be written as:

1 2 3 456578 9 3* 3:56: ;:57:8 '%$<8 or

1 2 45:= >?@A* :B;:5:= C>?@$'%<DEFG 2FHG (3.20)

In the frequency domain,

IJ 9 K* LMNO :5P: Q:5R:ST

(3.21)

Where UM 456 and UQ 45657.

while σx and σy are the standard deviations of the elliptical Gaussian

along x and y axes and F is the radial frequency (Equation (3.20)).

3.7.2 Hex-Gabor and its Features (Proposed method)

In this section, an attempt to follow a truly hexagonal lattice method for

HVS applications by the Hex-Gabor kernel or Hex-Gabor idea is proposed. A

low pass filter and high pass filter may be used to enhance surface information

and edge information respectively. When we put kernels on image pixel

locations and add it enhances image and also facilitates interpolation. A

modulated wave filter is more effective in edge enhancement and texture than a

unipolar narrow filter at higher resolution (can also be called as high pass filter

as in a Gaussian wave). In regular lattice isotropic kernels are important, but for

unequal pixel distances directional and orthogonal direction filtering are

important. This explains the reason for combining the responses of the filter-

bank.

Three orientations Hex-Gabor kernel can be designed in two ways: (i)

By performing filtering in three distinct directions (00, 600 and 1200) on

99

hexagonal lattice and adding the responses (ii) The resultant kernel is obtained

by adding the responses at .120,60,0 000 and The first method can be

implemented using mimicking scheme such as half pixel shift method. Initially,

the author used half pixel method for image enhancement and interpolation

using the first method. The results with Hex-Gabor interpolation with the

second method dispute the fact that the half pixel method requires radial

distance corrections, and they defeat edge enhancement as part of the edges get

averaged out. It was found that only an optimum value of sigma (2/pi) made a

visually error-free image. It was also noted that minor corrections are required

at the receding end of the kernel by windowing. It is clearly evident that sigma

(σ in Equation 3.20) values have to be kept same in all directions. However,

only the specific value could yield a good result. In scattered images like X-

rays, the contribution to intra-pixel information is not very obvious, and hence

this was attempted. Different sigma values enhanced different depth information

in such images.

The features of Hex-Gabor are analyzed and summarized as the contribution.

• It is capturing intra-pixel information only because it is filtered through

a 2D hex base. Conventional Gabor filtering does not do this, because it

only considers pixels along a line whether it is horizontal, vertical or

along axes in hex-domain.

• Hex-Gabor is considering the intra-pixel distances and directions of

adjacent pixels in a regular lattice also.

• At sigma value 2/pi the kernel is isotropic, unipolar and nonlinear. This

enabled filtering that would enhance the image edges and would retain

smooth information in any regular lattice.

• The resulting kernel can be used in rectangular lattice and we could

obtain same results as that has been achieved with hexagonal image

processing. This shows the importance of considering components in

orthogonal directions.

100

• It also shows that if this is considered even a shift from regular

hexagonal lattice of the retinal elements could be suitably compensated.

The results in this work also show how finer details are visualized by

this sigma selection by the eye. For sigma values more than 2/pi the

kernel was not unipolar. However, it works suitable for edge detection

as a pre processing step to a limiting value of sigma up to 4/pi.

• The isotropy of the Hex-Gabor kernel is lost when sigma exceeds 2/pi,

and smooth filtering qualities are lost for values above this value. To

demonstrate this, the shapes of the kernels are displayed below (Figure

3.13) for sigma=1/pi, 2/pi and 3/pi respectively.

(a) Sigma=1/pi (b) Sigma=2/pi

( c ) Sigma=3/pi

Figure 3.13 Hex Gabor kernels at different sigma values

• Filtering quality reduces substantially for a slight increase/decrease in

the value of sigma after 2/pi.

• While using kernels for image enhancement it generates continuous

image surface enabling interpolation. In addition to the fact that

Gaussian kernels are suitable for this purpose, a compact support is also

ensured in this method.

101

• Hex-Gabor is doing a nonlinear filtering [44] operation using an isotropic

kernel which is shown in Figure 3.14.

Figure 3.14 Hex-Gabor kernel with details of absolute values at sigma = 2/pi.

Another interesting property of Hex-Gabor is, the data is closely fitting with cubic B-

spline data (Figure 3.15) at sigma=2/pi (as per its well known equation defined in [84,

91]) as

VW )W* ' * + , -- . X -' * - W , -- . '+' , --

0 (3.22)

Figure 3.15 Comparison of Hex-Gabor data and cubic B-spline data

Horizontal scale pixel distance X 10.

• Gradient is computed by taking difference between the absolute values of the

Hex-Gabor at sigma = 2/pi and is found to be maximum at 0.636 (2/pi) which

is shown in Figure 3.16.

0 10 20 30 40 50 60 70 800 50 100

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

102

Figure 3.16 Computation of gradient with respect to pixel distance

3.7.3 Hex-Gabor Interpolation and Enhancement

Image interpolation can be used in image enlargement and local image

zooming. Several commonly used interpolation algorithms on rectangular and

hexagonal lattices have been suggested by various authors and those techniques

were reviewed and analyzed in section 3.4. Training of images is done to obtain

specific orientation or circular features in an image for the purpose of super

resolution [56]. However, no such training is required in Hex-Gabor case as

edge enhancement simultaneously takes care of the smooth parts of the image.

Gabor filtering methods [34] also tackle the image piecewise along the edges,

which deteriorates information along orthogonal directions.

The combination kernel obtained through Hex-Gabor is isotropic and

nonlinear and this effect is elaborated in the section 3.10.1.

3.7.4 Implementation results of Hex-Gabor

From the results of Figure 3.17 below, it is evident that Hex-Gabor is

considering the intrapixel distances and directions of adjacent pixels in any

regular lattice. At sigma value 2/pi the kernel was isotropic, unipolar and

103

nonlinear (Figure 3.17(a) and (b)). This enabled filtering that would enhance the

image edges and would retain smooth information in any regular lattice. The

isotropic property is lost if the sigma value is greater than 2/pi (Figure 3.17(e)

and (f)).

(a) (b)

(a), (b) sigma=2/pi (isotropic)

( c ) (d)

(c), (d) sigma = 3/pi(quasi-isotropic)

0 10 20 30 40 50 60 70 800 50 100

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

104

(e) (f)

(e), (f) sigma= 10/pi (non-isotropic)

(g) sigma = 20/pi (h)sigma=100/pi

Figure 3.17 Plot of Hex-Gabor kernel at different sigma values

From the results of Hex-Gabor surface in square lattice, plot for sigma= 100/pi

and more (Figure 3.18) it is clear that it can produce continuous surface and

Gabor profiles represents hexagonal lattice.

(a)

105

(b)

Figure 3.18 Hex-Gabor kernel surface at (a) sigma = 100/pi (b) sigma = 500/pi

3.7.5 Enhancement of Reflected and X-Ray Images using Hex-Gabor

Diffusion type of image processing requires that the pixel information in

the nearby pixels to be correlated. This is possible in case of normal images. In

the X-ray image, intensities in adjacent pixels could be uncorrelated, as it might

be coming from an interfering object in the path of X-rays. This is due to the

source of illumination which is not absolutely normal to the photographic plate

for this pixel information. The edges and quality of the images can be enhanced

by Hex-Gabor for the reflected images and scattered images like X-ray images

considered in this work. It was possible to obtain more PSNR value at unique

sigma (2/pi) in the case of reflected images. But, it was unsuccessful in

enhancing X-ray images with one unique value of sigma. Hence, the range of

values from 0.1 to 3 was used for the analysis, because the isotropic property of

Hex-Gabor is preserved in this range.

When an image is generated from a physical process, its values are

proportional to the energy radiated by a physical source (e.g. electromagnetic

waves). The image function ),( yxf may be characterized by the two

components: illumination and reflectance. The range of reflectance varies from

0 and 1. X-rays deal with a transmission rather than reflection function. The

intensity of the X-rays is modified by absorption as they pass through a patient

and the resulting energy falling on the film develops it in the same way that

light develops in the photographic film. When X-rays encounter any form of

matter they are partly transmitted and partly absorbed. Typically, digital X-ray

106

images are corrupted by additive noise relatively higher with respect to

conventional X-ray films. The most important part of the corrupting noise is the

Gaussian noise whose variance may vary with the signal level (due to sensor

nonlinearity) and spatially depending on the instrumentation.

Various X-ray image enhancement operations were proposed for

rectangular lattice such as non-linear mapping functions [100], Discrete Wavelet

Transforms (DWT) [101], canonical correlation based Kernel Independent

Component Analysis (KICA) [103], and nonlinear deblurring algorithm [104]. In

this work, an attempt is made to process the enhancement of X-ray images using

Hex-Gabor. It works well and gives better results on mimicking hexagonal grid

and also on rectangular grid. Hanning window (Equation 3.24) is used to

regulate the signal which yields better results. It is seen that a hanning window

requirement is eight times the size of kernel which gives the best results. It is

selected in such a way that the magnitude of the kernel at the second pixel

location reduces to almost zero as in the case of B-Spline. The window is

expressed as [105],

( +YZ * +Z/ [\] L4^NS + , _ , ` (3.24)

3.7.6 Image enhancement results of Hex-Gabor on mimicking hexagonal

grid

Table 3.2 summarizes the results of Hex-Gabor on mimicking hexagonal

grid (Half-pixel shift) using windowing. Even though there is marginal

improvement in PSNR and MSE values, it was possible to obtain good visual

appeal on the enhanced image compared with the response without using

window. As Hex-Gabor is performing well on true rectangular lattice

(discussion in section 3.7.7), the author is not doing further analysis on this.

107

Table 3.2 Performance measures of Hex-Gabor kernel filter on mimicking

hexagonal grid (half pixel shift) with windowing technique on reflected

normal images

3.7.7 Image enhancement results of Hex-Gabor on true rectangular lattice

with windowing technique on reflected images.

The Hex-Gabor was tested on true rectangular lattice without using any

mimicking scheme and it yields better enhanced image (Figure 3.19). The

performance is better in terms of PSNR and MSE (Table 3.3).

Image

Type

Hex-Gabor response without using

window

Hex-Gabor response with Hanning

window

PSNR MSE Computation

time (in sec)


time (in sec)

Wood 52.89 0.3340 3.72 57.54 0.1082 10.27

Lena 54.75 0.2177 3.45 54.99 0.2053 13.33

barbara 54.98 0.2063 3.44 59.75 0.0654 12.22

Peppers 57.76 0.1087 3.36 58.01 0.1028 13.25

mri 63.29 0.0197 3.87 65.50 0.0183 5.25

Bacteria 53.92 0.2654 4.85 60.89 0.0424 6.81

Pout 52.95 0.3321 3.36 64.58 0.0143 8.21

Testpat 53.38 0.2981 4.52 59.44 0.0587 8.83

window 56.06 0.1654 3.42 61.88 0.0335 16.98

108

Figure 3.19 (a),(c),(e),(g) original images Lena, Barbara, Peppers and Cat

respectively. (b),(d),(f),(h) Enhanced images using Hex-Gabor.

109

Table 3.3 Performance measures of image enhancement using Hex-Gabor

on true square lattice with windowing technique on reflected images

Image Type PSNR MSE Computation

time (sec)

Wood 64.83 0.0214 12.27

Lena 65.31 0.0044 14.32

barbara 71.01 0.0064 15.22

Peppers 68.19 0.0013 13.28

mri 66.43 0.0176 5.36

Bacteria 62.78 0.0378 6.88

Pout 66.58 0.0143 7.21

Testpat 60.44 0.0587 7.83

window 62.88 0.0335 15.98

The response of the Gabor filter depends on the tuning of the parameter sigma

(Equation 3.21). To prove the validity of sigma at 2/pi, the response of the Hex-

Gabor was tested for the range of sigma values from 0.1 to 3. Figure 3.20 is the

plot of the details shown in the Table 3.4 (sigma values vs PSNR value - for

Barbara image). The parameter sigma = 2/pi yields maximum PSNR value

(Figure 3.20). The Hex-Gabor kernel at this value has also some interesting

properties which were mentioned in the section 3.7.2. The image quality at

other sigma values, both PSNR and visual quality deteriorated substantially. It

was noted that the effect of Hex - Gabor is pronounced in other pseudo-lattices

with this value of sigma, and suboptimal image quality is obtained for three

direction operations as in regular hexagonal lattice.

110

Table 3.4 Performance comparison of image enhancement results of Hex-

Gabor using half pixel shift and Hex-Gabor for the sigma values from 0.1

to 3 (Barbara Image)

Image Type : Barbara.png

Sigma 0.1 0.5 0.637 1 1.5 2 2.5 3

PSNR

(Half pixel shift)

26.61 56.32 59.71 59.072 56.41 54.92 54.37 52.16

PSNR

(Hex-Gabor) 31.23 69.91 71.03 70.15 68.21 66.51

61.35

58.59

Figure 3.20 Plot of sigma vs PSNR for the Barbara image using Hex-Gabor

3.8 Comparative study of Hex-spline and Hex - Gabor

B. Foster et.al.[106], proved that by defining B - splines with complex

exponents, it is possible to construct complex wavelet bases that are tunable in a

continuous fashion, and that can closely approximate Gabor functions. These

extended complex B - splines inherit most of the properties of their classical

polynomial counterparts, and that they can generate a variety of multiresolution

bases. The basis functions are tunable with respect to two parameters. The real

part of the complex exponent (degree) is responsible for the smoothness,

111

whereas the imaginary part induces a one-sided predominance in the frequency

domain. This latter property is especially interesting for signal analysis. K.N.

Chaudhury, M. Unser [107] constructed a family of complex spline wavelets

that resemble the directional Gabor functions proposed by Daugman. Thus, the

Hex-Gabor kernel proposed in this work is able to satisfy most of these

properties even on the rectangular lattice. Overview of hex-spline is given in

section 3.4.6. Spline interpolation is used through Hex-splines for the purpose

of shear interpolation by earlier workers [45]. Gabor filter is different from

splines as it is a modulated basis function. It gathers simultaneously low pass

information through its envelope and high frequency information from

modulating wave. This may be used to gather mutual information

simultaneously between three neighboring pixels in three directions if intra-

pixel information is derived.

A Hex-Gabor function is a directional filter function when used

separately, but when combined it is isotropic. Interpolation function may have

low-pass characteristic within the interpolating distances, but may have edge

enhancement characteristic beyond these distances. An interpolation function in

a Hex domain (Hex-Bspline – spline used in three directions) could be different

from that in square domain (Bspline itself used in two directions). Hex-Gabor

filters as such are suitable for enhancement and interpolation in regular lattices.

3.8.1 Implementation issues and algorithm computational complexity

For performance comparison, Hex-spline kernel is modeled using the

MATLAB software and applied for image enhancement. Figure 3.21 illustrates

the enhancement results of Hex-spline and Hex-Gabor for Lena image. From

the results, it is evident that Hex-Gabor enhancement yields better visual appeal

including edge enhancement. Hex-Gabor response is found to be better in terms

of more PSNR and less MSE (Table 3.5). But, computation time of Hex-Gabor

process is marginally increasing because of windowing operation compared

with Hex-spline process. Third order Hex-spline is considered for comparison

because of its better performance (Table 3.1).

112

Figure 3.21 Comparative study of Hex-spline and Hex-Gabor (a) Original

Image (b)Tiling using Hex-spline (c) Enhanced image using Hex-spline (d)

Enhanced image using Hex-Gabor

Table 3.5 Performance comparison of image enhancement results of Hex-

Gabor and third order Hex-spline.

Image Type Response of Third order Hex - spline Response of Hex-Gabor at sigma = 2/pi


time (in sec)

PSNR MSE

Computati

on time

(in sec)

Wood 55.54 0.2053 4.12 64.83 0.0214 12.27

Lena 53.88 0.2085 5.92 65.31 0.0044 14.32

barbara 57.53 0.1015 5.72 71.01 0.0064 15.22

Peppers 56.88 0.1810 4.25 68.19 0.0013 13.28

mri 63.56 0.1028 3.98 66.43 0.0176 5.36

Bacteria 59.75 0.0654 5.08 62.78 0.0378 6.88

Pout 64.44 0.0183 4.35 66.58 0.0143 7.21

Testpat 58.04 0.0654 5.37 60.44 0.0587 7.83

window 60.54 0.0567 4.73 62.88 0.0335 15.98

113

3.9 Enhancement of X – ray images using Hex-Gabor Filter

Gabor filter models may also be specially parameterized for X-ray images (Figure

3.22) considering the optical and interfering characteristics of objects. It was found that

different values of sigma gave enhancement of different depth information. The range

of sigma values for the scattered images like X-rays varies from 0.5 to 3. These values

can be mapped with the sigma values of simple cell of BCS using the mathematical

model [5] described in the Equation 2.2.

Figure 3.22 (a), (c), (e), (g) Original images of chestxray.tif, thumb.jpg, X-ray1.jpg

and X-ray2.jpg respectively (b),(d),(f) and (h) Enhanced images using Hex-Gabor.

114

Table 3.6 summarizes the performance comparison of the enhanced images

using Hex-Gabor on (i) Mimicking scheme using half pixel shift approach (ii)

true square lattice at sigma=2/pi. It would not be necessary to use methods like

half pixel shift method to get similar results. Because, Hex-Gabor is considering

the intra-pixel distances and directions of adjacent pixels in any regular lattice.

A set of original and processed images are shown in Figure 3.22.

Table 3.6 Performance comparison of the enhanced images using Hex-

Gabor with (i) Half pixel shift approach and (ii) Hex-Gabor on true square

lattice (sigma=2/pi)

Hex-Gabor response on

hexagonal lattice using

half pixel shift approach

Hex-Gabor response on true

square lattice

PSNR MSE PSNR MSE

Chest xray.tif 64.296 0.0242 72.7232 0.0035

Thumb.jpg 59.452 0.0737 73.4625 0.0029

X-ray1 54.5063 0.2304 65.5676 0.0180

X-ray2 54.3071 0.2412 75.4815 0.0018

3.10 Comparison of Hex-Gabor results with nonlinear diffusion process

and Kramer’s PDE model

3.10.1 Non linear aspect of Gabor filter

When designing two-dimensional nonlinear filters for image processing, one

usually imposes an additional requirement that the constructed operator is

isotropic with respect to changing the original image’s orientation. Nonlinear

diffusion processes can be found in many recent methods for image processing

and computer vision. In image processing and computer vision, anisotropic

diffusion also called Perona - Malik diffusion is a technique aiming at reducing

image noise without removing significant parts of the image content, namely

115

edges, lines or other details that are important for the interpretation of the

image. Anisotropic diffusion resembles the process that creates a scale - space

where an image generates a parameterized family of successively more and

more blurred images based on a diffusion process. Each of the resulting images

in this family is given as a convolution between the image and a 2D isotropic

Gaussian where the width of the filter increases with the parameter [44]. This

diffusion process is a linear and space-invariant transformation of the original

image. Anisotropic diffusion is a generalization of this diffusion process: it

produces a family of parameterized images, but each resulting image is a

combination between the original image and a filter that depends on the local

content of the original image. As a consequence, anisotropic diffusion is a non-

linear and space-variant transformation of the original image.

The combination kernel obtained through Hex-Gabor is isotropic and

nonlinear. This is a favorable condition for edge enhancement. A step further,

Kramer’s PDE model [99] utilizes both of these more effectively (A

comparative result is given in Figure 3.24). Similar and/or better results were

obtained by the use of Hex-Gabor method in square lattice processing. The

results with the Hex-Gabor also supports the results of Gabor filter [34] in

respect of solution by heat equation, using a negative sign for the aJ where u

denotes the image. Hence, the non-linear function of Hex-Gabor

2

21

1781.1

+x

is analyzed and plotted (Figure 3.23) where 1.1781 is the peak

absolute value of the Hex-Gabor kernel obtained.

Figure 3.24 shows the comparative results of Kramer’s PDE model [99]

and Hex-Gabor. The enhanced image obtained using Kramer’s PDE model is

like a negative image (Figure 3.24(b)). It was possible to get a better quality

image including the edges using Hex-Gabor (Figure 3.24(c)) and it can be

noticed that soft tissues are enhanced well.

116

Figure 3.23 Nonlinear windowed function ) and Hex-Gabor

kernel plotted on the same figure.

Figure 3.24 (a) Original image (b) Enhanced images using the approaches (b)

Kramer PDE model [99] (c) Hex-Gabor

3.11 Summary

Hex-Gabor filter proposed in this work is found to be suitable for

enhancement and interpolation. Assuming that all the operations required by

human visual system is by Gabor filters in hexagonal domain, it is now

established by the results of this work in terms of its perfection to view

minutest details and sigma is also tuned for detailed and/or global vision. For

normal reflected images it was possible to obtain better results at unique sigma.

As X-ray images contain light scattered pixels instead of unique sigma, the

117

range of sigma from 0.5 to 3 was found to satisfy most of the image

interpolation/enhancement requirements in terms of visual effects. From the

comparative study, the performance of hexagonal grid based operations using

Hex-Gabor was found to be better than the Hex-spline process. Yue Yang and

Baoxin [108] proposed non-linear image enhancement using Gabor filter for

digital TV applications in rectangular lattice and image based kernels are also

adopted for high definition TV. Hex-Gabor therefore would find much

application in high definition TV which can be considered as future work.

Isotropy is not destroyed using this method if we choose proper sigma value

and it is parallelized. Hex-Gabor is also used in this work for X-ray imaging. It

can also be used for the other medical image processing applications such as

mammogram image enhancement and MRI images.

chapter 3 - hexagonal processing based interpolation...

Documents