978-1-4244-9074-5/10/$26.00 ©2010 IEEE 2010 Annual IEEE India Conference (INDICON)

Phase Extraction and Boundary Removal in Dual Phase Steel Micrographs

Omprita Chatterjee#1, Kaustav Das*2, Saptarshi Dutta*3 , Subhabrata Datta$4 , Sanjoy Kumar Saha*5 # Computer Application Dept., Pailan College of Mgmt. & Tech., Kolkata India

* CSE Department, Jadavpur University, Kolkata, India 5sks_ju@yahoo.co.in

$ Birla Institute of Technology, Deoghar Campus, Jharkhand, India

Abstract— In order to study the material characteristics, it is essential to obtain the quantitative description of micro-structure of the materials. In this context, digital image processing is used to analyze the microscopic images of the materials. Extraction of grains/phases present in the material is the fundamental step to achieve the description of the micro-structure. In this work, we present an automated scheme for segmenting the phases present in the microscopic image of high strength low alloy steel. The challenge posed by the presence of revealed grain boundaries bearing striking similarity with one of the phases also has been addressed successfully. Experiment shows that proposed scheme has the inherent capability to cope up with the factors like magnification, imaging condition affecting the image characteristics. Keywords— Phase segmentation, grain boundary removal, micro-structure analysis, material characterization

I. INTRODUCTION This In many engineering application, it is very essential to

study the behaviour of a material under various conditions like static or dynamic forces, loadings, operating temperature, stress, corrosion. It is also known that properties of materials are related to their micro structure [11,17,21]. Thus, it is essential to obtain a quantitative description of the micro-structure of the materials.

As it has been outlined in [23], there are two different approaches namely, indirect and direct techniques, to obtain such description. X-ray diffraction based measurement [14] is an indirect technique where structural parameters are estimated by measuring lattice parameter. On the contrary, in direct technique, the structural parameters are directly measured. Moreover, indirect approach can not give correct result in presence of stress and texture. Direct technique relies on the microscopic investigations. Such analysis includes steps like microscopic observation and image collection, image processing and analysis [15]. Many variations of microscopes like optical microscope, scanning electron microscope and transmission electron microscope, are used with great success allowing us to peer into spaces small enough to be seen with the unaided eye. Thus these instruments have become an indispensable tool for all scientific and technological activities, especially in materials research.

The images/micrographs produced by a microscope are easily converted into digital form for subsequent storage, analysis, or processing prior to display and/or interpretation [24, 20, 19, 12,16]. Digital Image processing greatly enhances the process of extracting information about the specimen from a micrograph and has become an integral part of microscopy related experimentation in metallurgy and materials engineering [3,8]. The paper is organized as follows. The brief introduction presented in this section is followed by past work in section 2 which reveals that segmentation is the crucial step for quantitative analysis of the micrographs. In section 3, we concentrate on the proposed methodology for segmenting dual phase steel micrograph. Experimental results are presented in section 4 and it is finally concluded in section 5.

II. PAST WORK An Major steps in image analysis based measurement

techniques are image acquisition, extraction/segmentation of area of interest and measurement of properties. Once the images are converted into digital form, segmentation appears as the major challenge. Micro-structure contains various grains/phases which are to be extracted. Such grains may be discriminated using the features like graylevel intensity, textural pattern, edge orientation. Thus, depending on the material under study, a suitable feature has to be chosen as the discriminator. Presence of noise or the impurities introduced by the image acquiring system or environment (i.e. contrast, brightness etc.) may cause hindrance in the process. But, presence of revealed grain boundary with features similar to those of another grain offers the major challenge.

Gauthier et al. [9] has presented an automated scheme for segmenting WC grains in the Cobalt matrix. They have adopted a two stage algorithm. Based on graylevel threshold and morphological gradient filter, first level of segmentation has been done and in the next stage, they have gone through a series of involved processing to get rid of grain boundaries. Classification based approach was tried in [18]. In [13], a scheme has been presented where an image classifier has been integrated with contextvision [5]. It facilitates contextual analysis (i.e. spatial dependencies among the regions) for extracting the areas of interest. But, such analysis incurs computational cost. Moreover, the classification accuracy

2010 Annual IEEE India Conference (INDICON)

heavily depends on proper training. Neural network also have been tried to classify the phases of an alloy [7]. Once the grains/phases/regions haven been extracted, properties/features are measured for those. In the area of materials engineering, quantitative metallography deals with the features of the micro-structure in a two dimensional plane, which can be correlated with the other properties of the material. The analysis of metallurgical samples includes grain size analysis, inclusion rating, volume fraction, porosity, particle size, morphology etc. Thus the results obtained from such analysis have immense importance not only for material characterization, but also for industrial production for effective quality control[6, 10, 1]. The stereo logical analysis method is successfully applied for quantification of chunky graphite [2] and volume fraction phases in titanium alloys [22]. It has already been emphasized that the most important aspect of automatic image analysis is to properly classify different phases in dual-phase or multiphase metallography samples. The accuracy of subsequent measures relies on the performance of the underlying segmentation scheme. The extent of etching or magnification have significant role, as it changes the character of the images of different phases to a certain extent. On the other hand presence of grain boundary also disturbs proper quantification of phases, as the volume or surface fraction of the grain boundaries is included in the quantified image as a part of a particular phase, thus increasing the calculated volume of the phase to some extent compared to its actual existence. The problem is more severe in case of images with higher magnification or in case of samples where the phase, with which the pixel values of the grain boundaries matches, is present in small amount. Considering all these issues, it is evident that a robust and elegant segmentation scheme is still in demand. It has motivated us to focus on segmentation.

Fig 1 A microscopic image of dual phase steel with revealed phase

boundary

III. PROPOSED METHODOLOGY In this work, we have dealt with scanning electron

microscopic images (in secondary electron mode) of high strength low alloy (HSLA) steels consisting of two distinct

phases. These dual phase (DP) steels comprises of soft polygonal ferrite along with the distribution of hard phases like martensite and/or bainite. The ferrite grains/phase with black appearance cover the major part whereas the sparsely distributed second phase appear as white blobs (see Fig.1). Thus, the task can be mapped onto the classical problem of segmenting the background and foreground. As it has been discussed in section 2, the grain/phase boundary also possesses the similar property as one of the phase and exclusion of those from the colliding phase is the challenge. In case of the dual-phase steel under consideration, such boundaries appear as white i.e. it becomes the part of second phase. Thus, in the context of segmentation algorithm, the task boils down to segment the image into 3-categories -- the two phases and phase/grain boundary. Hence, it is not the case of mere binarization of a grayscale image, it has to go beyond that. On successful identification of such regions, subsequent analysis proceeds with regions representing the phases only.

The proposed segmentation scheme follows a sequence of steps to classify the pixels into 3-categories. Primarily, binarization of the image is done and thereby one phase (black) is obtained and the other category needs refinement to exclude the phase boundary. The broad steps are as follows.

Binarize the image and consider black regions as first

phase.

Initial segmentation on white region to identify the candidates for second phase and area of confusion.

Final segmentation on area of confusion for possible inclusion in the second phase.

The details of the steps are elaborated in the subsequent subsections.

Fig 2 Binarized version of the image shown in fig 1.

A. Binarization It has been observed that the intensity values of the pixels

belonging to the two phases predominantly falls into the black and white region of the grayscale respectively. Hence, colour

2010 Annual IEEE India Conference (INDICON)

image is converted into grayscale version and with that we go for binarization to categorize the pixels into two classes. The steps are as follows.

Preprocessing.

Thresholding.

If the contrast between the two phases is strong enough then it may be sufficient to go for thresholding directly. But, due to the condition of image acquisition environment, it cannot be ensured. Electronic noise also may be present. Hence, before thresholding, a low pass filtering is preferred. In preprocessing, we apply a median filter on the grayscale version of the image. As a result, noise in the smooth regions are removed. On the other hand, blurring of edges (i.e. phase boundaries) is avoided and those are preserved. With the filtered image, we go for thresholding.

We have adopted intensity based thresholding. Hence, the main issue is to decide the intensity value called threshold which will act as the discriminator for two classes of pixels. Pixels with intensity value higher than the threshold are taken as white and the rest are taken as black. The phases present in the images are not uniformly black or white. Due to non uniform illumination, surface roughness etc., considerable variation may exist within the phases. Keeping the issues in mind, we have selected the threshold in the following way.

• Prepare intensity histogram corresponding to the image F(x,y).

• Il= Intensity of the highest peak within intensity range [0 …….K).

• Ih= Intensity of the highest peak within intensity range [K …….256).

• th = (Il + Ih) /2

• Thresholded image, Ft(x,y)=255 if F(x,y)>th. Ft(x,y)=0 otherwise.

In our experiment, we have considered K as 128. The output of binarization applied on the image in Fig. 1 has been shown in Fig. 2. It appears that black region more or less corresponds to ferrite grain. But, the white region includes the phase boundaries along with the second grain. We go for further refinement of white region in subsequent steps.

B. Initial Segmentation At this stage, we focus on the white region obtained after thresholding operation. In an endeavor to isolate the phase boundary and actual phase embedded in this region, we first try to extract the area which can be confidently considered as part of desired phase. We refer such area as a candidate for second phase. In order to find out the candidates, we rely on the observation that the second phase

appears as distributed white patch/blob. On the other hand, phase boundaries are thin lines with arbitrary orientation. The overlapping of the desired phase areas and boundaries makes the problem difficult. In the segmented white region, we find out the disjoint components using component labeling algorithm [4]. Based on its area within the enclosing rectangle, if a component is adjudged as a patch/blob then it is taken as a candidate for the desired phase else it belongs to area of confusion. The steps are as follows.

• Initialize segmented image, Fs(x,y)=Ft(x,y). • Do component labeling. • perform open-close operation. • For each component, Ci

Determine rectangular bounding box (BB). Compute occupancy ratio,

or = )()(

BBareaCiarea

if (or>thI) then Ci is a candidate. Otherwise, Ci Є area of confusion and Fs(x,y)=127 for (x,y) Є Ci.

As it appears in Fig.2, number of very small blob like components are present. These may be attributed to broken phase boundary lines, residual noise present after filtering and limitations introduced by thresholding. But they may qualify for a candidate. In order to get rid of those we rely on morphological open-close operation [4]. Structuring element(SE) of size K X K is used to perform these operations. First of all, opening is done on the components taken as candidates. It removes the components whose size is smaller than that of SE. Particularly, thin boundary lines are likely to be removed. Retained candidates also lose pixels around their border due to open operations. To bring them in proper shape, close operation is carried out on them. Selection of the size of SE also demands attention. The images that we are dealing with are the magnified one. As the degree of magnification may vary from image to image, concept of small/large component is also related to that. Hence, size of SE also has to be chosen dynamically in lieu of the common practice of considering a fixed one. Median of the area of candidate components is taken as the area of SE. We have introduced the candidate selection criteria based on the occupancy ratio as defined. A component that qualifies to be a candidate will occupy sizable area in its bounding box. Thus, isolated patch/blobs are more likely to get selected as candidates. As the boundary lines are generally of irregular orientation, the component consisting of such lines of considerable length are likely to fail in complying the criteria and thereby gets rejected. A component comprising of desired phase and a considerable part of phase boundary may get disqualified as the inclusion of thin boundary increases the area of its BB without adding much to the component area. As a result, we may lose part of desired phase also. Even, a component with only the desired phase may also fail to

2010 Annual IEEE India Conference (INDICON)

qualify if thI is high enough. In our experiment, we have considered thI as 0.6 keeping in mind that a circular blob attains the occupancy around 0.8. The output of initial segmentation for the image in Fig. 1 has been shown in Fig. 3. It shows three categories -- black, white and gray. The gray region denotes the area of confusion where we apply our final phase of the segmentation scheme.

Fig 3. Output of initial segmentation corresponding to the image in Fig 1.

C. Final Segmentation Initial segmentation is quite strict in accepting a white component as desired grains. In classification, it considers the whole component as a unit and categorizes accordingly. It is more focussed in excluding boundary even at the cost of desired phase. As the components in the area of confusion may also contain part of desired phase, we look into those for possible inclusion in the phase area. The basic approach is that instead of applying the selection criteria on the whole component, we verify the same within a mask of M × M dimension in the bounding box (BB)of the component. Gradually, shifting the mask, we test the whole component in parts. In choosing the size of mask, we face the similar issues as in case of SE in subsection 3.2 and we resolve it also in the same manner. For a component Ci with bounding box BBi in the area of confusion, the refinement process is carried out as follows.

• Place the mask on the top left position of BBi • Repeat till BBi has been tested completely

Determine oa, the overlapped region of mask and ci

Occupancy ratio, or = maskarea

oaarea )(

If or > thF then include oa as desired phase Shift the mask to next position

In final segmentation, as we are looking for occupancy in a localized area. Hence, we can not afford to be relaxed. Hence, thF has been taken as .75 in our experiment. Mask shifting demands attention. BBi is scanned horizontally in left to right direction. Mask is normally shifted by one position. But in case, a masked area is selected for inclusion then shifting by

mere one position may make the neighbouring undesired region eligible for inclusion by the virtue of induction. To combat it, in such cases, we shift the mask by M/2 position. Similar concept is also adopted in deciding the vertical shift on reaching the end of horizontal scan line.

Fig 4. Output of final segmentation corresponding to the image in Fig 1. Final segmented out corresponding to the image shown in Fig 1 has been shown in Fig. 4. Looking through the images in Fig. 2, 3 and 4, subsequent refinement is visible.

IV. EXPERIMENTAL RESULTS In order to carry out the experiment, we have considered 12 dual-phase HSLA steel micrographs. They vary in terms of grain size and concentration, magnification and illumination level. Such collection has enabled us to judge the robustness of proposed scheme. Sample outputs are shown in Fig 5. To measure the segmentation performance, we have compared the result with ground-truth information. As white grains are interfered by the boundaries, analysis is also focused on it. It has been observed that the proposed scheme successfully discards the grain boundaries with high precision -- it discards more than 95% of the grain boundary pixels. Thus, incorrect inclusion of boundary pixels in white grains is as low as 5%. It also retains around 92% pixels of desired white grains.

V. CONCLUSION We have presented a novel scheme for automatic extraction of the phases present in the micro scoping image of high strength low alloy steel. In spite of strong similarity between the grain boundary and a phase, the scheme successfully discriminates them. Moreover, the strength of the scheme lies in the fact that it does not rely on any assumption about the issues like magnification factor, contrast level which affects the image characteristics. Experimental result indicates that performance of the scheme is satisfactory for a wide variety of cases.

.

2010 Annual IEEE India Conference (INDICON)

Fig 5. Sample Result: each row shows an image and corresponding segmented output.

REFERENCES [1] Materials Handbook, Metallography and Microscopy, volume 9.

ASTM Publication, Ohio, 1988. [2] K. Breitkreutz, H. Frenz, and P. D. Portella. On the quantitative

determination of chunky graphite in cast iron. Practical Metallography, 28:532.541, 1991.

[3] K. R. Castleman. Digital Image Processing. Prentice-Hall, 1996. [4] B. Chanda and D. D. Majumder. Digital Image Processing and

Analysis. Prentice-Hall, 2001. [5] Contextvision. Users's Guide, MicroGOP2000/S Software v. 3.1. 1999. [6] G. Das. Image analysis in quantitative metallography. In Proc.

Workshop on Materials Charaterization Techniques . Principles and Application, pages 135.150, NML, Jamshedpur, India, 1999.

[7] V. H. C. deAlbuquerque, A. R. Alexandria, P. C. Cortez, and J. M. R. S. Tavares. Evaluation of multilayer perceptron and self-organizing map neural network topologies applied on microstructure segmentation from metallographic images. NDT and E International, 42:644. 651, 2009.

[8] A. Diaspro. Confocal and Two-photon Microscopy. Wiley-Liss, 2001. [9] G. Gauthier, M. Coster, L. Chermant, and J. Chermant. Morphological

segmentation of cutting tools. Microsc. Microanal. Microstruct., 7:339.344, 1996.

[10] M. Goken and H. Vehoff. Quantitative metallography of structural materials with the atomic force microscope. Scripta Materialia, 35:983.989, 1996.

[11] E. O. Hall. The deformation and ageing of mild steel: Discussion of results. Proc. of Royal Society, B64(9):747.753, 1951.

[12] S. Inoue and K. R. Spring. Video Microscopy. Springer, 1997. [13] J. Komenda. Automatic recognition of complex microstructures using

the image classi_er. Materials Characterization, 46:87.92, 2001. [14] C. E. Kril and R. Birringer. Estimating grain-size distributions in

nanocrystalline materials from x-ray diffraction pro_le analysis. Philosophical Magazine A, 77(3):621.640, 2001.

[15] K. J. Kurzydlowski and B. Ralph. The Quantitative Description of the Microstructure of Materials. CRC Press, London, 1995.

[16] D. B. Murphy. Fundamentals of Light Microscopy and Electronic Imaging. Wiley-Liss, 2001.

[17] N. J. Petch. The cleavage strength of polycrystals. J. of the Iron and Steel Institute, 174:25.28, 1953.

[18] H. V. Saebo, K. Braten, N. L. Hjort, B. Llewellyn, and E. Mohn. Contextual classi_cation of remotely sensed data: Statistical methods and development of a system. Technical Report, Norwegian Computing Center, 1985.

[19] G. Sluder and D. E. Wolf. Digital Microscopy. Academic Press, UK, 2003.

[20] D. L. Spector and R. D. Goldman. Basic Methods in Microscopy. Cold Spring Harbor Laboratory Press, 2005.

[21] W. L. Spychalski, K. J. Kurzydlowski, and B. Ralph. Computer study of inter- and intraangular surface cracks in brittle polycrystals. Materials Characterization, 49(9):45.53, 2002.

[22] K. Wang, W. Zeng, Y. Shao, Z. Zhao, and Y. Zhou. Quanti_cation of microstructural features in titanium alloys based on stereology. Rare Metal Materials and Engineering, 38:398.403, 2009.

[23] T. Wejrzanowski, W. L. Spychalski, K. Rozniatowski, and K. J. Kurzydlowski. Image based analysis of microstructures of engineering materials. Int. J. Appl. Math. Comp. Sc., 18(1):33.39, 2008.

[24] Q.Wu, F. S. Merchant, and K. R. Catleman. Microscope Image Processing. Academic Press, UK, 2008.