advanced (mid-level) dip -- morphological image...

1

1

Advanced (Mid-Level) DIP-- Morphological Image Processing

Xiaojun Qi

2

Morphology

• Morphology: It is a tool for extracting image components that are useful in the representation and description of region shape, such as boundaries, skeletons, and the convex hull.

• Morphological operations preserve the basic properties of object while removing irrelevant features.

• Applications: segmentation, enhancement, restoration, edge detection, texture analysis, shape analysis, thinning, and curve filling.

3

• The binary images are normally produced by simple segmentation techniques such as thresholding. They may contain numerous imperfections caused by noise, texture or the inaccurate specification of a threshold. Morphological image processing techniques can remove the imperfections and provide us with information on the form and structureof the image.

• Morphological techniques are also applicable to grayscale images, where they can be used for non-linear smoothing and feature enhancement. 4

• The term “morphological image processing” describes a range of non-linear image processing techniques that deal with the shape of features in an image.

• Morphological techniques typically probe an image with a small shape or template known as a structuring element (SE). The SE is positioned at all possible locations in the image and it is compared with the corresponding neighborhood of pixels.

• Morphological operations differ in how they carry out the comparison. Some test whether the structuring element ‘fits’ within the neighborhood; others test whether it ‘hits’ or ‘intersects’ the neighborhood.

5

C

A

B

o

Probe an image with a structuring element at A, B, and C positions.

Fit or Hit?

6

11111

11111

11111

11111

11111

00100

01110

11111

01110

00100

00100

00100

11111

00100

00100

The dimensions of the matrix determine the overall size of the structuring element, and its shape is determined by the pattern of ones and zeros.

1) What are the sizes of the above three SEs?2) What are the shapes of the above three SEs?

2

7

Fit or Hit?• A structuring element (SE) is said to fit an

image if for each of its pixels that is set to 1, the corresponding image pixel is also 1. The SE pixels that are 0 define points where the corresponding image value is irrelevant.

• The SE is said to hit an image if for any of its pixels that is set to 1, the corresponding image pixel is also 1. We also ignore image pixels for which the corresponding SE pixel is 0. 8

000000000000011111100000011111111100000111111100000011111100000011111110000011111110000001111100000000011000000000000000

Suppose s1 is a 3*3 square SE and s2 is a 3*3 cross-shaped SE, determine whether they are hit or fit in three positions above.

9

Basic Idea of Morphology• Probe an image with a SE and determine

the manner in which the SE fits, hits, (or does not fit or hit) within the image.

• The SE will be small compared to the image.

• By marking the locations where the SE fits or hits within the image, one obtains information about the structure of the image which depends upon the size and shape of the SE.

10

Set Theory and Notation• Translation:• Reflection:• Complement:• Intersection:• Union:• Difference:• SubSet:• Disjoint (Mutually Exclusive)

{ }AaaccA ∈−== ,|ˆ{ }AaxaccA x ∈+== ,)(

{ }AccA c ∉={ })()( BcAccBA ∈∧∈=∩

{ })()( BcAccBA ∈∨∈=∪

{ })()( BcAccBA ∉∧∈=−},|{ BcthenAcifcBA ∈∈=⊆

φ=∩ BAThe elements of the sets are the coordinates of pixels (i.e., members of the 2D integer space Z2) representing objects or other features of interest in an image.

11 12

Logical Operations

These three operations are functionally complete in the sense that they can be combined to form any other logic operation.

3

13 14

Binary Dilation• The dilation of A by B is defined as:

• Set B is commonly referred to as the structuring element in dilation, as well as in other morphological operations.

• Dilation expands the object if the origin is in B.

{ }.})ˆ(|{}|}{{

,,})ˆ(|{

φ

φ

≠∩+=∈+∪=

∈∈+==≠∩=⊕

ApBpBbbA

BbAabaccABpBA p

15

Binary Dilation Implementation View 1

• Position a SE s such that its origin is at image pixel coordinates (x, y) and apply the rule:

repeating for all x and y.• Dilation creates a new image that

shows all the locations of a SE’s origin at which it hits the input image.

=otherwise. 0

. hits if 1),(

fsyxg

16We used a 3*3 square SE to do the dilation

Binary Dilation Example 1

17

Binary Dilation Implementation View 2

Think of the SE as a template, and move it over the image. When the origin of the SE aligns with a black pixel in the image, all of the image pixels that correspond to black pixels in the SE are marked, and will later be changed to black. After the entireimage has been swept by the SE, the dilation calculation is complete. 18

Binary Dilation Example 2-- Origin is part of the SE

Final Dilation Result

4

19

Binary Dilation Example 3-- Origin isn’t part of the SE

20

Binary Dilation Example 4-- Origin is part of the SE

(a) (b) (c) dilation by one pixel each time

1) What kind of SE can achieve (a) (b) and (b) (c)?

2) What kind of SE can achieve (a) (c)?

21

Binary Dilation Example 5-- Effects of the SE size

Here we used square SE

What is the effect after applying the dilation operation? -- Inner and outer boundaries, holes, and gaps

22

Binary Dilation Example 6-- Effects of the SE shape

Here we used a square SE of size 6*6 and a circle SE of radius 3

What is the effect after applying the dilation operation? -- Inner and outer boundaries, holes, gaps and corner

23

Binary Dilate Example 7-- Effects of the SE shape

5*5 Square SE Circle SE with radius of 2

What is the effect after applying the dilation operation? -- Boundaries, holes, and corner

24


We used 3*3 square SE to do the dilation

5

25

Assume the maximum length of the breaks is known to be two pixels.


26

Binary Dilation Summary

27

Binary Erosion• The erosion of A by B is defined as:

• If the translated set is contained in set A, the p is in the erosion of A with B.

• Erosion results in a shrink of the image.• Dilation and erosion are duals of each other with

respect to set complementation and reflection.

( ){ } { }}|)({

|

BbbA

ABppABpBA p

∈−∩=

⊆+=⊆=Θ

BABA cc ˆ)( ⊕=Θ28

Binary Erosion Implementation View 1

• Position a SE s such that its origin is at image pixel coordinates (x, y) and apply the rule:

repeating for all x and y.• Erosion creates a new image that marks all

the locations of a SE’s origin at which it fits the input image.

=otherwise. 0

. fits if 1),(

fsyxg

29

Binary Erosion Example 1

We used 3*3 square SE to do the erosion

30

Binary Erosion Implementation View 2

6

31


(c) (b) (a) erosion by one pixel each time

1) What kind of SE can achieve (c) (b) and (b) (a)?

2) What kind of SE can achieve (c) (a)?32

Binary Erosion Example 3-- Effects of the SE size

Here we used square SE

What is the effect after applying the erosion operation? -- Inner and outer boundaries, holes, and gaps

33

Binary Erosion Example 4-- Effects of the structuring element size

5*5 Square SE Circle SE with radius of 2

What is the effect after applying the erosion operation? -- Boundaries, holes, and corner

34


We used 3*3 square SE to do the Erosion

35

Finding a Region Boundary

Source Code to Generate the BoundaryA = imread('char_a.jpg') ;se1 = strel('square', 3) ; SquareErodeA1 = imerode(A, se1) ;Boundary1 = imsubtract(A, SquareErodeA1) ;SquareDilateA1 = imdilate(A, se1) ;Boundary2 = imsubtract(SquareDilateA1, A) ;

What is the difference between these two

boundaries?

36

Binary Erosion Summary

7

37

• The result of dilation/erosion depends on SE size and shape, with larger SEs having a more pronounced effect.

• The result of dilation/erosion with a large SE is similar to the result obtained by iterated dilation/erosion using a smaller SE of the same shape.

• Dilation by a disc enlarges the object and smoothes its convex corner; Erosion by a disc shrinks the object and smoothes its concave corner.

Summary-- Binary Dilation and Binary Erosion

38

1. What is the structuring element?2. Does erosion followed by dilation fully restore the eroded

objects in this case?3. Can we expand the conclusion from 2 to a general case?

The “active” elements of structuring elements assume the same binary values as the objects of interest.

39

Binary Opening• The opening of A by B is defined as:

• The opening of A by B is the erosion of A by B, followed by a dilation of the result by B.

• Opening generally smoothes the contour of an object, breaks narrow isthmuses, and eliminates thin protrusions. “open” small gaps or spaces between touching objects in an image.

• Remove much of the black pixel noise in the background of the image. (Pepper Noise)

BBABA ⊕Θ= )(o

40

BAoThe boundary of the is established by the points in B that reach the farthest into the boundary of A as B is rolled around the inside of this boundary

})(|){( ABBBA zz ⊆∪=o

41

Binary Opening Example 1-- Eliminate Thin Protrusions

Original Image A

BSE

Opening A by B

42

Binary Opening Example 2-- Break Narrow Connections

Original Image A

BSE

Opening A by B

8

43

Binary Opening Example 3-- Remove Pepper Noise

• Pepper noise consists of collections of small dark objects.

• Open operator can remove pepper noise.• The trick is to find a structuring element which

passes the image but eliminates the pepper noise.• One often uses a symmetric structuring element

so that results do not depend on the orientation of the picture.

• One often starts with small structuring elements B and increases the size systematically.

timesnBBBnBnBnBSnBS

⊕⊕⊕=⊕Θ=

......)(o

44

Binary Opening Example 3-- Remove Pepper Noise (Cont.)

A

B Opening A by B

Can we use a square SE to remove pepper noise by using the opening operation?

45

Binary Opening Example 4-- Remove Noise

(a) A binary image (b) Opening of (a) with a disc SE46

Binary Closing• The closing of A by B is defined as:

• The closing of A by B is simply the dilation of A by B, followed by the erosion of the result by B.

• Closing also tends to smooth sections of contours, but it generally fuses narrow breaks and long thin gulfs, eliminates small holes, and fills gaps in the contour.

• Remove much of the white pixel noise in the foreground (objects) of the image, giving a fairly clean image. (Salt Noise)

BBABA Θ⊕=• )(

47

Roll B on the outside of the boundary

})(|){( φ≠∩∪=• ABBBA zz

48

Binary Closing Example 1-- Eliminate Small Holes

Original Image A

BSE

Closing A by B

9

49

Binary Closing Example 2-- Fuse Narrow Breaks and Long Thin Gulfs

Original Image A

BSE

Closing A by B

50

Binary Closing Example 3-- Remove Salt Noise

• Salt noise consists of small white objects.• The close operator will remove salt noise

in a manner similar to the way the open operator removes pepper noise.

A

B

Closing A by B

51

Binary Closing Example 4

(a)A binary image with five clusters of points. Points within each cluster satisfy the partition property with the distance ρ0, and the clusters are farther from each other than 2ρ0 pixels.

(b)The image of a closed by a disk with a radius just greater than 2ρ0 52

Binary Opening vs. Binary Closing-- Original Image

111111111111111111111111111111

11111111111111111111111111111

1111111111111111111111111111

53

Binary Opening vs. Binary Closing-- Opening (Break Narrow Connections)

1111111111111111111111111111111111111111111111111111111111111111111

54

Binary Opening vs. Binary Closing-- Closing (Fuse Narrow Breaks and Long Thin Gulfs)

11111111111111111111111111111111

111111111111111111111111111111111111

11111111111111111111111111111111

10

55

Properties of Binary Opening and Closing• The opening operation satisfies the following

properties:

• The closing operation satisfies the following properties:

BABBABDBCDC

ABA

ooo

oo

o

=))(3 ofsubset a is then , ofsubset a is If )2

. of (subimage)subset a is )1

BABBABDBCDC

BAA

•=••••

•

))(3 ofsubset a is then , ofsubset a is If )2

. of (subimage)subset a is )1

56

Application of Opening and Closing

(a) An image having many connected objects, (b) Objects can be isolated by opening using the simple structuring element, (c) An image that has been subjected to noise, (d) The noisy image after opening showing that the black noise pixels have been removed.

57

Application of Opening and Closing (Cont.)

The result of closing Figure (d) using the simple structuring

element

Multiple closings for outline smoothing. (a)

after a depth 2 closing, (b) after a depth 3 closing. 58

59

Morphological filtering: (a) original, noisy image; (b) result of erosion; (c) opening of A; (d) result or performing dilation on the opening; (e) final result showing the closing of the opening. 60

Hit-or-Miss Transform• The hit-or-miss transform is a basic tool for

shape detection or object recognition.

• The hit-and-miss transform is a general binary morphological operation that can be used to look for particular patterns of foreground and background pixels in an image.

• The hit-or-miss transform is defined as:Let B ={B1, B2}, where B1 is the set formed from elements of B associated with an object and B2 is the set of elements of B associated with the corresponding background, where B1 and B2 are disjoint.

)2ˆ()1(

)2()1(

BABA

BABABA c

⊕−Θ=

Θ∩Θ=∗

11

61

• A pixel belonging to an object is preserved by the hit-or-miss transform if and only if B1 translated to that pixel fits inside the object and B2 translated to that pixel fits outside the object.

• B1 and B2 cannot intersect, otherwise it would be impossible for both fits to occur simultaneously.

62

Four SEs used for corner finding in binary images using the hit-and-miss transform.

If the foreground and background pixels in the structuring element exactly matchforeground and background pixels in the image, then the pixel underneath the origin of the structuring element is set to the foreground color. If it doesn't match, then that pixel is set to the background color.

63

Four SEs used for corner finding in binary images using the hit-and-miss transform.

64

The Graphical Description of the Hit-or-Miss

Transform

65

Shape Detection-- Find the character ‘n’

B1 =

1 1 0 1 1 1 1 0 0

1 1 1 1 1 1 1 1 0

1 1 1 0 0 0 1 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

1 1 0 0 0 0 0 1 1

B2 =

1 1 1 1 1 1 1 1 1 1 1

1 0 0 1 0 0 0 0 1 1 1

1 0 0 0 0 0 0 0 0 1 1

1 0 0 0 1 1 1 0 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 0 0 1 1 1 1 1 0 0 1

1 1 1 1 1 1 1 1 1 1 166

} and )(|)()(max{))((

bf DxDxsxbxsfsbf

∈∈−+−=⊕

1D Gray-Scale DilationChoose the maximum value of f+b in the interval defined by the shape of the SE, which intersects with the domain of Df.

The range of b intersects with the range of the displaced f.

12

67

1D Gray-Scale Dilation Ex 1

699858974-30-3

-30-3-30-3

-30-3-30-3-30-3-30-3

-30-3-30-39983897

Dilation Result

Original Intensity

68


551111155

SE

Original Intensity

12321

•What is the size of the dilated result?•What is the final dilated result?

69

At each position of the SE, the value of dilation at that point is the maximum of the sum of f and b in the interval spanned by b.

70

2D Gray-Scale Dilation

• Gray-scale dilation is defined by:

• Where

},;)(),(|),(),(max{),)((

bf DyxDytxsyxbytxsftsbf

∈∈−−+−−=⊕

ly.respective b, and f of domains theare and bf DD

.morphologybinary in case in the

sets of instead functions are and bf DD

71


Original Image Dilated ImageThe general effect of performing dilation on a gray-scale image is twofold:1) If all the values of the SE are positive, the output image tends to be brighter than the input.2) Dark details either are reduced or eliminated, depending on how their values and shapes relate to the SE used for dilation. 72

1D Gray-Scale Erosion

} and )(|)()(min{))((

bf DxDxsxbxsfsbf

∈∈+−+=Θ

The range of b is completely contained within the range of the displaced f.

Choose the minimum value of f-b in the interval defined by the shape of the SE within the domain of Df.

13

73

1D Gray-Scale Erosion Ex 1

*96369*-30-3

-30-3-30-3

-30-3-30-3

9983897

•What is the result after using the SE = (0 0 0)?•Applying which filter will achieve the same result? 74

2D Gray-Scale Erosion• Gray-scale erosion is defined by:

• Where},;)(),(|),(),(min{

),)((

bf DyxDytxsyxbytxsftsbf

∈∈++−++=Θ

ly.respective b, and f of domains theare and bf DD

.morphologybinary in case in the

sets of instead functions are and bf DD

75

2D Gray-Scale Erosion Ex 1

Original Image Eroded ImageThe general effect of performing erosion on a gray-scale image is twofold:1) If all the elements of the SE are positive, the output image tends to be darker than the input image.2) The effect of bright details in the input image that are smaller in area than the SE is reduced, with the degree of reduction being determined by the gray-level values surrounding the bright detail and by the shape and amplitude values of the SE itself.

76

77

2D Gray-Scale Opening

• The opening of image f by SE b is defined as:

• Opening operations usually are applied to remove small (with respect to the size of the SE) light details, while leaving the overall gray levels and larger bright features relatively undisturbed.

bbfbf ⊕Θ= )(o

78

2D Gray-Scale Closing

• The closing of image f by SE b is defined as:

• Closing operations are generally used to remove dark details from an image, while leaving bright features relatively undisturbed.

bbfbf Θ⊕=• )(

14

79 80

5888468838882778

111121111

Original Image SE

•What is the dilation result?•What is the erosion result?•What is the opening result?•What is the closing results?

81

2D Gray-Scale Opening Ex 1

Original Image Opened Image82

2D Gray-Scale Closing Ex 1

Original Image Closed Image

83 84

The Applications of Binary Morphology

• When dealing with binary images, the principal application of morphology is extracting image components that are useful in the representation and description of shape.– Extract boundaries– Extract connected components– Extract convex hull– Extract the skeleton of a region– Others (Used in pre- or post- processing steps):

Region filling, thinning, thickening, and pruning.

15

85

Basic Binary Morphological Algorithms-- Boundary Extraction

(1) Internal Boundary:(2) External Boundary:(3) Morphological Gradient:

ABAA −⊕= )()(β)()( BAAA Θ−=β

)()()( BABAA Θ−⊕=β

(1) (2) (3)

where B is a 3x3 square SE and A is the solid rectangle

What is the boundary and the thickness of the boundary for each case? 86

Boundary Extraction Example

What is the thickness of the boundary?

87

Basic Binary Morphological Algorithms-- Region Filling

• Consider filling a region surrounded by a curve. Let A be the curve and p a point in the interior of the region A. Then

• The algorithm terminates at iteration step k if Xk = Xk-1.

• The set union of Xk and A contains the filled set and its boundary.

......,3,2,1)(

;

1

0

=∩⊕=

=

− kABXXandpX

ckk

Conditional Dilation

88

Region Filling Example

The goal is to eliminate the black inner spots in each white circle.

How to find the starting points which are contained within the spheres (i.e., black regions completely contained within white region)?

89

Region Filling Example-- Connected Regions in Maze

90

Basic Binary Morphological Algorithms-- Extraction of Connected Components

• Let A be a connected component and p a point on the connected component A. Then

• The algorithm terminates at iteration step k if Xk= Xk-1. The algorithm converges to A = Xk.

......,3,2,1)(;

1

0

=∩⊕==

− kABXXandpX

kk

What is the difference between this formula and the region filling formula?

16

91

Extraction of Connected Components Ex 1

92

Extraction of Connected Components Ex 2

Shown below is an original image, a color-coded image showing the connected components, and an index color palette.

93

Basic Binary Morphological Algorithms-- Thinning

• Thinning removes pixels from a set until only a narrow set remains. It is used to reveal set structure in recognition applications.

• The thinning of a set A by a SE B is defined in terms of the hit-or-miss transform:

• A more useful expression for thinning A symmetrically is based on a sequence of SEs:

where Bi is a rotated version of Bi-1.• The entire process is repeated until no further changes

occur.

cBAABAABA )()( ∗∩=∗−=⊗

}...,,,,{}{ 321 nBBBBB =).)...))((((...}{ 21 nBBBABA ⊗⊗⊗=⊗

94

Thinning Ex 1

Reduce all lines to single pixel thickness.

95

Thinning Ex 2

Original Image Thinned Image

There are broken lines in the thinned image. How to get a thinned image without any broken line? 96

Original Image Thinned Image without broken lines

Solution Preprocess the original image. That is,

dilate the image to connect the lines before thinning it.

17

97

Thinning Ex 3The basic structure is captured by the thinned objects (red). The small ends could be removed by further processing to refine the result.

98

Basic Binary Morphological Algorithms-- Thickening

• The thickening of a set A by a SE B is defined in terms of the hit-or-miss transform:

• A more useful expression for thickening Asymmetrically is based on a sequence of SEs:

where Bi is a rotated version of Bi-1 and all 1’s and 0’s interchanged with the B’s in thinning operation.

}...,,,,{}{ 321 nBBBBB =

)( BAABA ∗∪=•

).)...))((((...}{ 21 nBBBABA •••=•

99

Basic Binary Morphological Algorithms-- Convex Hull

• A set A is said to be convex if the straight line segment joining any two points in A lies entirely within A. That is, any two points can be joined by a straight line that does not go outside the set.

• The convex hull H of an arbitrary set S is the smallest convex set containing S, i.e.,

• The set difference H-S is called the convex deficiency of S.

• The convex set and convex deficiency are needed in many applications such as object description.

.HS ⊆

100

Basic Binary Morphological Algorithms-- Convex Hull

• Let be a set of SEs, then

with

• The convergence result is

• The convex hull of A is:

}...,,,,{}{ 321 nBBBBB =

ik

ik

ik

i XXXD 1where −==

2,...1,k andn ...,2,1,i)( 1 ==∪∗= − forABXX ik

ik

AX i =0

in

iDAC ∪

=

=1

)(

1. The procedure consists of iteratively applying the hit-or-miss transform to A with a series of SEs.

101

Convex Hull Ex 1

(a) (b) (c)

Original Image A sequence of SEs Convex Hull

Origin

What is the minimum convex hull for the original image? 102

Convex Hull Ex 1 (Cont.)

• One obvious shortcoming of the procedure is that the convex hull can grow beyond the minimum dimensions required to guarantee convexity.

• Solution:

Limit growth so that it does not extend past the vertical and horizontal dimensions of the original set of points. That is, Trim C(A) to the bounding box of the component.

What is the final trimming result of the convex hull for the previous example?

18

103

Basic Binary Morphological Algorithms-- Skeletons

• The skeleton of A can be expressed in terms of erosion and openings. That is:

with

K is the last iterative step before A erodes to an empty set.

UK

kk ASAS

0

)()(=

=

BkBAkBAASk o)()()( Θ−Θ=

104

Skeletons Ex 1

105

Skeletons Ex 2

Original Image Skeletons

106

Basic Binary Morphological Algorithms-- Pruning

• Pruning methods are an essential complement to thinning and skeletonizing algorithms because these procedures tend to leave parasitic components that need to be “cleaned up” by post-processing.

• Pruning is a four step procedure. [Refer to section 9.5.8 for details]

107

Pruning Example

Origin

108

Some Applications of Gray-Scale Morphology

• Morphology Smoothing:

The effect is to remove or attenuate both bright and dark artifacts or noise.

.)( bbfs •= o

19

109


• Morphology Gradient: ).()( bfbfg Θ−⊕=

It is a good strategy to use symmetrical SEs since they tend to depend less on edge directionality.

110


• Top-hat Transformation: ).( bffh o−=

It is useful for enhancing detail in the presence of shading.

111


• Textual Segmentation:

1. Use a SE of the size of the small blob to do a closing operation.

2. Use a SE of the size of the large blob to do a opening operation.

3. Thresholding.112


• Granulometry:

1. where b are SEs with increasing size.2. These differences are normalized and then used to

construct a histogram of particle-size distribution.• It is based on the idea that opening operations of a

particular size have the most effect on regions of the input image that contain particles of similar size.

).( bffh o−=

113

Useful Matlab Commands

• strel, getsequence• imdilate, imerode, imopen, imclose• bwhitmiss• bwmorph• bwlabel• imreconstruct, imfill, imclearborder

advanced (mid-level) dip -- morphological image...

Documents