newer image segmentation methods for biomedical object detection

Newer Image Segmentation Methods for Biomedical Object Detection

Tarundeep Singh Dhot Dept of ECE

Concordia University Montreal, QC H3G 1M8

ABSTRACT The basic aim of the study is intended towards

the field of image segmentation of cells with the

aim that it can be further extended to detect cell

inclusions. The study can be divided into three

phases. The first phase deals with the

segmentation of individual cell from a cell

image. Three segmentation approaches have

been used for this purpose. This phase is

followed by image mapping of the cell into

various image intensity plots. It is then intended

to use a Genetic Algorithm to locate the global

optima of the plot.

Categories and Subject Descriptors I.4 [Image Processing and Computer

Vision]: Segmentation – pixel classification

General Terms Algorithms, Experimentation

Keywords

Image Segmentation, Genetic Algorithms

1. INTRODUCTION The field of Biomedical Image Processing is an

exciting and growing cross-disciplinary field.

An exponential increase in the power of digital

processors, a steady growth in the field of

Digital Image Processing principles has given a

much needed impetus for emerging as a mature

subject in its own rights. One of the most

upcoming and active areas of interest is the

analysis of cell images for detection of cell

inclusions. Cell image segmentation is a

necessary first step of many automated

Biomedical Image Processing procedures.

Image segmentation is the identification and

isolation of an image into regions that one

hopes, correspond to structural units. It is an

essentially important operation in biomedical

image processing since it is used to isolate

physiological and biological structures of

interest. General approaches to segmentation

can be grouped into three classes: pixel-based

methods, regional methods and edge-based

methods.

Traditionally, pixel-based methods are the

easiest to understand and to implement, but are

also the least powerful and, since they operate

on one element at a time, they are particularly

susceptible to noise. Continuity-based and edge-

based methods approach the segmentation

problem from opposing sides: edge-based

methods search for differences while continuity-

based methods search for similarities.

In this study, three segmentation approaches

have been used: pixel-based methods,

continuity-based multi-thresholding and the

third one is based on a recent research

publication “Automatic segmentation of Cells

from Microscopic Imagery using Ellipse

Detection”. It is also important to mention the

use of morphological operators in the last

approach. Morphological operations have to do

with processing shapes. In this sense they are

continuity-based techniques, but in some

applications they also operate on edges, making

them useful in edge-based approaches as well.

After loading the image, pre-processing of the

image was carried out in terms of reduction of

noise using various filters.

Once the image is pre-processed and

successfully segmented, mapping of the image

of the individual cell is done into various image

intensity plots. These intensity plots of the

image represent the segmented image into

various 3-Dimensional surface plots of their

matrix data. These intensity plots show exactly

the location of the cellular inclusion on the basis

of presence of a region (inclusion) having

similar pixel values. Thus, the concentration of

the pixel values is represented in the intensity

plot over the entire cellular region. Once the

segmented image is represented in as a 3-D

intensity plot, the last intended step is to use a

genetic algorithm to locate the global optima.

This phase is currently in the process of

formulation.

The project is developed on the MATLAB 7.0

platform.

2. METHODOLOGY

As stated above, the study has three phases:

1. Segmentation of individual cells from cell

image;

2. Mapping image of individual cell to an

intensity image; and

3. Use of GA to locate the global maxima

Let us start of with cell segmentation.

In segmentation, the inputs are images but the

outputs are attributes extracted from those

images i.e. segmentation subdivides an image

into its constituent regions or objects. In the

study, three approaches to cell segmentation are

used. We shall see them individually in detail.

The first step in cell segmentation is loading the

image and converting it to gray scale from the

usual RGB color format. The image is then

further pre-processed. This step is common to

all the methods used. This step can be referred

to as the Loading Subroutine.

Loading Subroutine:

a) Load the image

b) Perform grayscaling.

c) Improve contrast for the grayscaled

image.

d) Improve precision (of the image from

step c).

It may be remembered that the images are all in

the bitmap format (.bmp). One inclusion from

the slide was pasted on a black background.

This also reduces the effect of noise. All

segmentation techniques are implemented on

this image to show variance in effectiveness of

the three methods.

2.1 Approach 1: Pixel-Based

Segmentation Using Minimal

Variance Iterative Technique For

Thresholding

The most straightforward and common of the

pixel-based methods is thresholding in which all

pixels having intensity values above, or below,

some level are classified as part of the segment.

Thresholding is an integral part of converting an

intensity image to

a binary image. Thresholding is usually quite

fast and can be done in real time allowing for

interactive setting of the threshold. The basic

concept of thresholding can be extended to

include both upper and lower boundaries

(slicing).

A major concern in pixel-based methods is

setting the thresholding or slicing level(s)

appropriately. Usually these levels are set by the

program, although in some situations they can

be set interactively by the user. Finding an

appropriate threshold level can be aided by a

plot of pixel intensity distribution over the

whole image, regardless of whether you adjust

the pixel level interactively or automatically.

Such a plot is termed as the Intensity

Histogram. Intensity histograms can be very

helpful in selecting threshold levels, not only

for the original image, but for images produced

by various segmentation algorithms. Initially

histograms can be useful in evaluating the

efficacy of different processing schemes: as the

separation between structures improves,

histogram peaks should become more

distinctive. If the intensity histogram is, or can

be assumed as bimodal (or multi-modal), a

common strategy is to search for low points, or

minima, in the histogram.

Keeping this in mind, there are two approaches

based on which Approach 1 is designed and

implemented.

The first approach is to improve the

determination of histogram minima is based on

the observation that many boundary points carry

values intermediate to either side of the

boundary. These intermediate values will be

associated with the region between the actual

boundary values and may mask the optimal

threshold value. However, these intermediate

points have the highest gradient, and it should

be possible to identify them using a gradient-

sensitive filter, such as the Sobel or Canny

filter. After these boundary points are identified,

they can be eliminated from the image, and a

new histogram is computed with a distribution

that is possibly more definitive. Thus, this leads

to slightly better segmentation of the cell.

The second threshold strategy is one that does

not use the histogram is based on the concept of

minimizing the variance between presumed

foreground and the background elements.

Although the approach assumes two different

gray levels, it works well even when the

distribution is not bimodal. The approach uses

an iterative process to find a threshold that

minimizes the variance between the intensity

values on either side of the threshold level

(Outso’s method). This can be referred to as

Minimal Variance Iterative Technique.

The above two approaches are thus incorporated

and implemented for Approach 1 as follows:

1. Load the image of the cell and perform the

necessary pre-processing like gray scaling

on the image (Loading Subroutine)

2. Display images after pre-processing along

with the intensity histogram

3. Remove the edge pixels from the image and

display the histogram of this modified

image.

4. Determine thresholds using minimal

variance iterative technique described

above

5. Apply this approach to threshold both

images.

6. Display the resultant thresholded images.

7. Perform class conversion of the final

segmented image

8. Plot the image intensity plots

To remove the edge boundaries, first

identify these boundaries using an edge

detection scheme. While any of the edge

detection filters can be used, for our approach,

Canny filter is used as it is more robust to noise.

The implementation of this filter will produce a

binary image of the boundaries. This boundary

image is converted to a boundary mask by

inverting the image. After inversion, the edge

pixels will be zero while all other pixels will be

one. Multiplying the original image by the

boundary mask will produce an image in which

the boundary points are removed (i.e. set to zero

or black). Perform class conversion (to double -

double-precision floating-point number array)

on the final segmented image in order to plot

the intensity graphs.

Figure 1.1 displays broadly the various image

transformations of Pixel-based segmentation

approach used for Approach 1. The original

image as shown in Fig 1.1 a) is first converted

to an increased contrast, high precision gray

scale image as shown in Fig 1.1 b). Firstly, edge

pixels are removed and then the image is

thresholded to give the final segmented image

shown in Fig 1.1 c). Intensity plot are

determined on Fig 1.1 c) which gives rise to the

intensity plot as shown in Fig 1.1 d) which is a

meshed image intensity plot.

2.2 Approach 2: Image Segmentation

Using Multi-Thresholding Technique

(Using AND - OR Operators)

The results of several different segmentation

approaches can be combined either by adding

the images together or more commonly, by first

thresholding images into separate binary images

and then combining them using logical

operations. Either the AND or OR operator

would be used depending on the characteristics

of each segmentation procedure. If each

procedure identified all of the segments, but

also included non-desired areas, the AND

operator could be used to reduce the artifacts.

Alternatively, if each procedure identified some

portion of the segment(s), then the OR operator

could be used to combine the various portions.

This approach is used in Approach 2 where first

two, then three, thresholded images are

combined to improve segment identification. The structure of interest is a cell which is shown

on a gray background. Threshold levels above

and below the gray background are combined

(after one is inverted) to provide improved

isolation. Including a third binary image

obtained by thresholding a texture image further

improves the identification.

Implementation of Approach 2:




2. Display images after pre-processing

3. Perform linear filtering of the image using

Low Pass Filter

4. Threshold the image its complement

5. Threshold texture image

Fig 1.1 a) Original Image

Fig 1.1 b) High Precision Gray

Scaled Image

Fig 1.1 c) Segmented Image (Thresholded

Edge Removed)

Fig 1.1 d) Image Intensity Plot (Mesh)

Figure 1.1: Image transformations using

Pixel –Based Segmentation

6. Combine the thresholded image and its

complement

7. Combine the above image with the

threshold texture image

8. Display thresholded and combined images


segmented image


It is possible to isolate some portions of the cell

by thresholding above and below the

background level. After inversion of the

thresholded image, the images are combined

using a logical OR. Since the cell also shows

some textural features, a texture image is

constructed by taking the regional standard

deviation. After thresholding, this texture-based

image is also combined with the other two

images.

The figure on the left displays the main image

transformation of the cell using the Multi-

Thresholding technique used in Approach 2.

Again, here too, first the original image shown

in Fig 1.2 a) is pre-processed into an increased

contrast, high precision gray scale image shown

in Fig 1.2 b). The image shown in Fig 1.2 c) is

obtained after use of logical OR operator on the

thresholded and inverse thresholded image. Fig

1.2 d) shows the logical OR operation of image

obtained in Figure 1.2 c) with the textural

image.

Fig 1.2 e) and 1.2 f) show the intensity plots for

Fig 1.2 c) and d).

2.3 Approach 3: Automatic

Segmentation of Cells from

Microscopic Imagery using Ellipse

Detection:

In this approach, a two-phase algorithm that

combines iterative thresholding with ellipse

detection is used for potent cell segmentation

mechanism. In this approach, a combination of

Morphological Operators with adaptive

thresholding is used in order to simultaneously

eliminate noise and locate inclusion regions,

hence, computing the best possible local

threshold levels for the inclusion extraction.

The main reason why this approach of also

included in this study was due to the use of

morphological operators. As stated earlier,

Fig 1.2 a) Original Image

Fig 1.2 b) High Precision Gray

Scaled Image

Fig 1.2 c) Combined Thresholded Image

Fig 1.2 d) Combined Thresholded and

Textural Image

Fig 1.2 e) Intensity Plot

Fig 1.2 f) Intensity Plot

Figure 1.2: Image transformations using

Multi –Thresholding

morphological operations have to do with

processing shapes. The two most common

morphological operations are dilation and

erosion. In dilation the rich get richer and in

erosion the poor get poorer. Specifically, in

dilation the centre or active pixel is set to the

maximum of its

neighbors and in erosion it is set to the

minimum of its neighbors. Since these

operations are often performed on binary

images, dilation tends to expand edges, borders

or regions, while erosion tends to decrease or

even eliminate small regions. The two processes

can be done in tandem, over the same area.

Since erosion and dilation are nonlinear

operations, they are not invertible

transformations i.e. one followed by the other

will not generally result in the original image. If

erosion is followed by dilation, the operation is

termed as opening. If image is binary, this

combined operation will tend to remove small

objects without changing the shape and size of

large objects. Basically, the initial erosion tends

to reduce all objects, but some of the smaller

objects will disappear altogether. The

subsequent dilation will restore those objects

that were not eliminated by erosion. If the order

is order is reversed and dilation is performed

first followed by erosion, the combined process

is called closing. Closing connects objects that

are close to each other, tends to fill up small

holes, and smooths an object outline by filling

small gaps. As with the more fundamental

operations of dilation and erosion, the size of

objects removed by opening and closing

depends on the size and shape of the

neighborhood that is selected.

Implementation of Approach 3:




2. Display images after pre-processing

3. Perform linear filtering of the image using

Low Pass Filter

4. Application of Open Morphological

Operation:

Erode Dilate.

5. Global Thresholding.

6. Multi-Region Adaptive Thresholding.

7. Second application of Open Morphological

Operation

8. Display thresholded images


segmented image


3. RESULTS

The results for the study are described in the

form on the images obtained from the

experiment. Image output obtained from all the

three approaches are displayed in a sequential

flow chart format for easier understanding. The

study was carried out on an Intel Pentium 4,

3.20GHz, 512 MB DDR RAM. Experiments

were performed on MATLAB 7.0.

As shown in Fig 2 for Pixel-based edge

removed thresholding, after the initial pre-

processing of the images, using a Canny filter,

the edges are removed (Fig 2 g). This image is

then thresholded to give the final segmented

image (Fig 2 h). The intensity plot (mesh) of

this image is shown in Fig 2 (i).

Fig 2.1 deals with the various intensity plots

depending on the function against which the

image is plotted. Three such functions are

shown: mesh, ribbon and surf.

Also displayed are two pairs of histograms (Fig

2.2 and 2.3), the first one comparing a simple

gray scale image to a high precision image, and

the second one showing improved separability

when boundaries are eliminated.

Similarly, the output images of Approach 2 i.e.

Multi-Thresholding are displayed in Fig 3. One

thing to note here is that Fig 3 (e) shows the

Thresholded image. This image is then inversed

(complemented) which is shown in Fig 3 (f) and

these both images are then logically ORed to

produce the Combined Threshold Image of Fig

3 (g). This image is again ORed with a textural

image shown in Fig 3 (h) to give rise to

Combined Threshold Textural Image shown in

Fig 3 (i). Fig 3 (j) shows the intensity plot of

only image Fig 3.1 (g) while Fig 3 (k) is the

intensity plot of Fig 3 (j).

Fig 3.1 represents the various intensity plots.

Fig 4 represents the output images of Approach

3: Auto-segmentation of cells. Fig 4 (e) and (f)

represent the first operation of the

morphological operators: Erode followed by

Dilation i.e. opening. Fig 4 (g) represents image

of cell after first thresholding. Similarly, Fig 4

(h) and (i) represent the “second opening”

operation while Fig 4 (j) represents the second

thresholding on the cell.

Fig 4.1 show the intensity plots after each

opening operation. Fig 4.2 shows the various

intensity plots for the final thresholded image in

Fig 4 (j).

4. DISCUSSION

In this project, an effort was made to study three

different segmentation approaches even though

no broad comparison study was made. As it can

be seen, the output segmented images of all the

three methods are completely different. This

shows how segmentation results from different

approaches can yield to completely different

that pixel based methods are particularly

Fig 1.3a) Original Image

Fig 1.3b) High Precision Gray

Scaled image

Fig 1.3c) First Opening

Fig 1.3 d) First Thresholding

Fig 1.3 e) Second Opening

Fig 1.3 f) Final Thresholding

Fig 1.3 g) Intensity Plot

Fig 1.3: Image transformation using

Automatic Segmentation Approach

susceptible to noise, since they operate on one

element at a time. Filtering can reduce the effect

of noise, thus is used in all the three approaches.

Also with Multi-Thresholding, we can logically

AND or OR images according to our

requirement which can be really helpful. Use of

textured image also increases the options for

use. Lastly, morphological operators allow for

manipulation of maxima and minima of an

image. Also with intensity plots, visualization

of cell inclusions improves as it shows

magnitude of relative intensity to its location.

5. FIGURES

5.1 Pixel Based Methods

5.1.1 Phases in Image Transition

a) Original Image b) Gray Scaled c) Increased Contrast

d) High Precision

e) Gaussian Low

Pass Filterf) 3-D Convolution

Filter (Canny)

g) Edge Removed

h) Segmented Image:

Thresholded Edge

Removedi) Intensity Plot (Mesh)

Fig 2: Pixel-Based Segmentation Using Minimal Variance Iterative Technique For

Thresholding

5.1.2 Intensity Plots

a) 3-D Mesh Plot

b) 3-D Ribbon Graph of Matrix

c) 3-D Surface Plots

Fig 2.1 Various Plots of Image Intensity for Pixel Based Edge-Removed Thresholding

5.1.3 Histograms

Fig 2.2: Histograms of Gray Scaled Image and High Precision Image. There is

significant improvement in the histogram of High precision Image

Fig 2.3: Images of cell with (upper) and without (lower) intermediate boundaries

removed. The associated histograms (right side) show improved separability when

boundaries are eliminated

5.2 Multi-Thresholding

5.2.1 Phases In Image Transition

a) Original Image b) Gray Scaled c) Increased Contrast d) High Precision

e) Thresholded

Image

f) Complement

of Threshold

g) Combined Threshold Image

h) Textural Image

i) Combined Threshold

Textural Image

j) Intensity Plot: Combined Threshold Image

j) Intensity Plot: Combined Threshold Textural Image

Fig 3: Image Segmentation Using Multi-Thresholding Technique (Using AND

- OR Operators)


(a) Intensity Plots of Combined Thresholded and Threshold Textural Image

(b) Intensity Represented by Various Functions

Fig 3.1: Intensity Plots: Multi-thresholding Technique

5.3 Auto-Segmentation Of Cells

5.3.1 Phases In Image Transition

a) Original Image b) Gray Scaled c) Increased Contrast d) High Precision

j) Intensity Plot (Mesh and Ribbon)

Fig 4: Automatic Segmentation of Cells

e) Erode - I

f) Dilate - I

g) Threshold - Ih) Erode - IIi) Dilate - IIk) Threshold - II

5.3.2 Intensity Plots Of Morphological Operations

a) Mesh Plot After Erosion - I

c) Mesh Plot After Erosion - II

b) Mesh Plot After Dilation - I

d) Mesh Plot After Dilation - II

Fig 4.1 Intensity Plots After Opening Operations

The above figure shows various intensity plots after the use of “Opening Operation” before and after

thresholding of the image. The marked change in the plots can be seen. The plots before thresholding suffer

from presence of noise in the image. After thresholding, presence of noise seen is minimal.


(a) 3-D Mesh Plot of Image (b) 3-D Surface Plot of

Image

(c) 3-D Contour Plot of Image

Fig 4.2: Intensity Plots for Automatic Cell Segmentation Method

newer image segmentation methods for biomedical object detection

Documents