novel segmentation method for optical braille character identification

3rd International Conference on Electrical, Electronics, Engineering Trends, Communication, Optimization and Sciences (EEECOS)-2016

Novel Segmentation Method for Optical Braille Character Identification

W.D. Chamalee N. De Silva, N.G. Chandana Srilal, Harsha B. Athapaththu and L. Ranathunga

Department of Information Technology, University of Moratuwa, Sri Lanka [email protected], [email protected], [email protected], [email protected]

Keywords: Optical Braille recognition, Braille, image processing, dilation, Segmentation.

Abstract

Braille is a tactile writing system used by the blind. Optical Braille recognition fills the communication gap between the blind and sighted people as well as the Braille literate and illiterate people. Most of the optical Braille recognition algorithms include segmentation methods to extract the characters inside the scanned images of Braille documents. But most of the systems involve high computations and standard resolutions. This paper presents a new algorithm for Braille text segmentation in Optical Braille Recognition. The proposing algorithm involves a stripping method which is used to identify each of the Braille characters separately in an embossed printed paper using image processing. This method uses techniques such as dilation, standard distance ratios and a vertical filling algorithm according to that, to extracts the Braille characters. The algorithm has successfully tested for the embossed printed papers written in Braille for Sinhalese language and has shown a 99.2% of average accuracy.

1 Introduction

The world is evolving with the new technologies and has enhanced the power of communication among the people as well as between people and machines. Anyone anywhere around the world has the capability to communicate with each other using oral, written, symbolical and graphical methods using various new technologies. But there is one party in the world that cannot see the world. They need special support to do their communication in reading and writing. Those are the blind people who are 39 million of world’s population. And there are 285 million people in the world who are visually impaired due to various reasons [1]. 90% of these blind and visually impaired people are seen in developing countries [1]. For the tasks of education and communication blind people use “Braille” which is a tactile writing system. Braille was introduced in 1824 by Louise Braille for French and by then it was developed for various other languages including English, Japanese, Korean, Bharthi, Arabic, Greek, Sinhala, etc. The Braille character is represented using six positions of dots which are organized in the rectangular manner including 3 rows and two columns.

Primarily slate and stylus are used to hand write in Braille and today there are sophisticated printers to print Braille. Separate type of papers are use to take the Braille writing in printed format which shows characters in an embossed manner. The blind person read it by touching on top of the Braille characters and by understanding how many dots are there in the character, the character size is nearly equal to the thumb size. There are standard distances between the dots in the printed format. Appearance of the number of dots from the six and the position that they are situated in, the character that it represents may vary. The dimensions considering one character are as shown in Fig. 1.

Fig. 1. Braille cell with dimensions

However to use Braille to read and write the user should be Braille literate. But the problem involved that is the amounts of people who are Braille literate are very few considering both national and international level. Therefore the Optical Braille recognition mechanisms were introduced to electronically recognize the Braille characters written in a printed Braille paper. In optical Braille recognition the typical steps followed are acquiring the image; preprocess the image; dot localization and segmentation; cell recognition and finally convert the cell in to the desired language [3]. Segmentation and cell extraction from the embossed Braille text paper is the major and very important part in Optical Braille Recognition (OBR). This paper describes a novel segmentation method that can be used with less computation.

2 Literature Review

Braille cell segmentation in Optical Braille recognition involves the extraction of the number of rectangular constructs from the embossed printed Braille paper. Before the segmentation acquiring a correct scanned image of the embossed printed paper and preprocess the image is essential.

760


Preprocessing includes Grayscale conversion, contrast improvement, binarisation, inverting, noise removal and skew angle detection [4]. Some researchers have given the circle fitting the points as a preprocessing task to identify the points excluding the noise involved there. For the segmentation task various researchers has introduced several methodologies. Basically there are two categories. First one is distance measurement segmentation and the second one is grid based segmentation. The grid based method is performing by creating a grid by joining the dots vertically and horizontally. The points where the horizontal points cross the vertical points are identified as check points and if there is a Braille dot detected then it considered as a part of a character. Since the Braille document is standard and includes two columns and three rows for each character the cells are extracted according to that. In the second method the standard distances between the dots are taken in to the consideration. All the distances between each of the two points are calculated and according to the distances it decides what are the other neighboring points of the same Braille cell and the points which are not belong in to a same cell [4]. Mennens et al. in [7] used a method based on several assumptions. As an example a single Braille dot of a Braille character is represented using two grey level intensities. Due to the scanning this effect is expected in the optical Braille recognition. They have implemented their system to recognize the Braille dots which are printed in either sides of the paper. But this method produces false characters if some two dots are vertically situated in neighboring region. Another approach has taken by Hermedia et al. That process uses a similar approach by taking the couples of black and white dots which denoted one single dot of Braille text [8]. The drawback involves in this method is that if some required points are not there in the image the method produces false points. Most of other methodologies used are resolution dependent. The Arabic Braille Recognition research conducted by AbdulMalik Al-Salman has considered the dot size as 8 pixels [5]. It is about 1.5 mm in actual format. With that and the other standard distances they have given standard pixel sizes for the characters and segmented the Braille document which is printed two sided.

2.1 Comparison with the proposed segmentation method

The proposing segmentation method has several specifications than the existing approaches in Braille.

• It is not resolution dependent and it used the ratios of the standard distances between the dots in the scanned image of the embossed Braille text.

• The characters that have dots in both columns has identified as full characters and the characters which have dots in one column has identified as half characters. Those half characters are then filled as a full character using s stripping algorithm.

• In this method first the horizontal segmentation is performed to divide the document into rows and then row by row is considered and vertical segmentation is performed.

• Image dilation techniques are used in the algorithms to segment the document horizontally and vertically.

• The method involves less computations and a high accuracy rate in terms of extracting the Braille characters.

3 Methodology

The development was done in the Java development environment with Opencv image processing library. The system involves several stages in Braille character extraction such as image acquisition, preprocessing, horizontal segmentation, vertical segmentation and character extraction. Both horizontal and vertical segmentation part two dilation techniques are used. A stripping algorithm is also performed in both segmentations to distinguish the Braille character for the extraction.

3.1 Image acquisition

The printed Braille document is a paper which has located the characters using dots that has elevated from the paper. That paper is scanned using a proper scanner that can detect those dots distinguishably with the background. Fig. 2 shows a sample of a scanned image of Braille text acquired as such. The image is saved in JPEG file format.

Fig. 2. Scanned image of Braille text

For this research we used Braille documented written in Sinhala language. This image is acquired and then taken for the preprocessing tasks.

3.2 Image Preprocessing

In image preprocessing we first do the grayscale conversion of the image using COLOR_RGB2GREY specifications in opencv. Then the grayscale image is converted into binary. This binary image contains black dots which are laciniate. To avoid that and also to remove unwanted small amounts of noise the image is filtered. For filtering Gaussian filter with sigma value 3 is used. Then the image is binarized again with the intention of having more clear dots. Then the final step of preprocessing is inverting the image. Fig. 3 shows the samples of resulting image in the preprocessing steps.

(a) (b)

761


(c) (d)

Fig. 3. (a) Grayscale image (b) binary image (c) second binary image after Gaussian blur (d) inverted image

The inverted image is the result of the preprocessing stage and that image is used in the segmentation.

3.3 Horizontal segmentation

In the horizontal segmentation the rows that have the Braille characters are extracted. First the resulting image of the preprocessing is dilated using Imageproc.dilate( ) function in opencv. This dilation is performed starting with a constant dilation of kernel size 2(dilation size) +1 vertically and 20(dilation size) +1 as the horizontal dilation. The dilation should ensure not to connect neighboring rows but to connect the most of the dots in the same row. The resulting image of this horizontal dilation is shown in Fig. 4.

Fig. 4. Dilated image in horizontal segmentation Then an algorithm is performed to identify the rows from the image. The horizontal stripping algorithm works as follows: Algorithm 1:

a) A threshold is defined as follows. Threshold (T) = K/2 (1)

b) Where K = kernel width of the horizontal dilation c) After getting the horizontally dilated image traverse

the image pixel by pixel, row by row. d) If the number of white pixels of one row > T then

make the whole row of pixels of value 255 (white color).

e) Repeat the steps 2 and 3 until the end pixel of the image.

After performing this algorithm, an image is constructed with the size of the Braille document image where the rows that have the characters will be all white and other space will be all black. Small noises will not affect to the recognition due to the use of a threshold. According to that the rows are extracted using the submat function in opencv from the inverted image. Then the rows

will be divided into separate Mat structures. Fig. 5 shows the resulting image of the above algorithm and the separated rows.

(a)

(b)

Fig. 5. (a) Resulting image of horizontal stripping algorithm

(b) separate images of rows of Braille characters The separated rows of the Braille cells are then considered individually one by one in vertical segmentation.

3.3 Vertical stripping

The rows of the Braille cells are separated in the horizontal segmentation. Then each of those rows is then stripped using white stripping algorithm after performing dilation. The dilation for the vertical segmentation is equally affected to the Braille dots of the inverted image horizontally and vertically. In dilation in vertical segmentation the dilation size should be carefully selected. The dilation size should be able to merge the dots of the same Braille cell which are in the two columns together without merging the neighboring dots together. For that, the system adjusts the dilation size by checking the stripping widths in further steps. Sometimes the dilation cannot be able to merge the two columns of the character dots together and result in a space of several pixels. After performing the dilation the stripping algorithm described above is performed considering column by column and draw the white strips. Fig. 6 shows the dilation of one row and the resulting image of the white stripping of the dilated image. (a)

(b)

Fig. 6. (a) Dilated image for the vertical segmentation (b) Resulting image of white stripping algorithm

In the white striped image as shown in Fig. 6(b), there are various types of black distances between the white rectangles that has constructed. Mainly there are three types of white rectangles in the resulting image. Those are,

762


a) Rectangles that cover the whole cell (high width rectangles) – type 1

b) Rectangles that cover the whole cell by with very little black space within the cell – type 2

c) Rectangles that cover one column of the cell due to absence of dots in the other column – type 3

From the above three types of rectangles first type of white rectangles are error free. The other types of rectangles should be constructed as full cells to do the character extraction.

3.4 Full and half cell identification

From the resulting image of the white stripping algorithm the type-1 rectangles are considered as full cells. Type-2 and type-3 rectangles are considered as half cells. The half cells are then to be filled to create them full cells.

3.5 Half cell filling

The half cells identified in type-2 and type-3 has the width of half of the width of full cells identified in type-1. Then these half cells are considered separately and the width of those half cells is expanded to the left side or the right side to make them full cells. To expand the half characters the black space of either sides of the half character is considered with the standard distance ratios of the Braille printing. The standard distance between two characters is 3.5 mm and the standard distance between two immediate characters in the same character is 2.5mm. Due to the dilation the 3.5 mm distance has decreased and the standard distance of the characters between two characters considering two immediate white strips are recognized as nearly 1/6 of the full character width and no more than ¼ of the full character width.

Fig. 7. Possible spaces between the Braille cells. Other than that the small spaces occurred between the full characters due to the insufficiency of the dilation to merge the

full character are recognized as less than or equal to 1/9 of the full cell width. Other than that there are 5 other different types of spaces. Fig. 7 shows various types of those black spaces. If the width of a full character is taken as M then the distance (x) of the space characters can be shown as Fig. 7. The red colored space represents the spaces. First of all the small black spaces between the full characters are replaced with white in each of the images of white stripped segments. Then the half characters of the either sides of that space become full characters. Then the other half characters are considered. The left side and the right side of the half character can have several combinations of that distances identified in Fig. 7. If the spaces represented in the Fig.7 are considered as 1, 2, 3, 4, 5, 6, 7 and 8 respectively the combinations that can be occurred are as follows: (Situation 1 is already considered.) (2, 3), (2, 4), (2, 6), (2, 7), (2, 8), (3, 4), (3, 5), (3, 6), (3, 8), (4, 5), (4, 6), (4, 8), (5, 6), (5, 7), (5, 8) (6, 7), (6, 8) (7, 8) (3, 2), (4, 2), (6, 2), (7, 2), (8, 2), (4, 3), (5, 3), (6, 3), (8, 3), (5, 4), (6, 4), (8, 4), (6, 5), (7, 5), (8, 5) (7, 6), (8, 6) (8, 7) As an example (2, 3) means that left side of the half cell has a black space of width of 2nd situation (M/9 <x < M/4) and the right side of the half cell has a adjoining black space that has a width of 3rd situation (M/2 < x <M). Following vertical filling algorithm catch those combinations and expand the half character according to the situation of the width of the black space. Symbolic indicators indicated in the algorithm are: LW = width of the left side black space of half character RW = width of the right side black space of half character M = width of a full character WS = Starting column number of the half character WE = Ending column number of the half character HC = half character Space ranges: S1, S2, S3, S4, S5, S6, S7, S8 (respectively according to the diagram indicated in Fig. 7.) Algorithm 2: If LW < RW If LW = S2 Expand width of HC from WE to (WE+ M/2) Elseif LW = S3

0 < x < = M/9 M/9 < x < M/4

M/2 < x < M

M < x < 3M/2

3M/2 < x < 2M 2M < x < 3M x > 3M

763


If RW = S4 Expand width of HC from WE to (WE+ M/2) If RW = S5 Expand width of HC from WS to (WS - M/2) If RW = S6 or RW = S8 If WS – M/2 column is white Expand width of HC from WE to (WE+ M/2) Else Expand width of HC from WS to (WS - M/2) Endif Elseif LW = S4 Expand width of HC from WS to (WS - M/2) Elseif LW = S5 Expand width of HC from WE to (WE+ M/2) Elseif LW = S6 If RW = S7 Expand width of HC from WE to (WE+ M/2) Elseif RW = S8 If WS-3M/2 is white Expand width of HC from WE to (WE+ M/2) Else Expand width of HC from WS to (WS - M/2) Elseif LW = S7 Expand width of HC from WS to (WS - M/2) Endif Else // right side is the smallest now If RW = S2 Expand width of HC from WS to (WS - M/2) Elseif RW = S3 If LW = S4 Expand width of HC from WS to (WS - M/2) Elseif LW = S5 Expand width of HC from WE to (WE+ M/2) Elseif LW = S6 If WS – 3M/2 column is white Expand width of HC from WE to (WE+ M/2)

Else Expand width of HC from WS to (WS - M/2) Endif Elseif LW = S8 Check next character width If next is a full character Expand width of HC from WE to (WE+ M/2) Else Expand width of HC from WS to (WS - M/2) Endif Elseif RW = S4 Expand width of HC from WE to (WE+ M/2) Elseif RW = S5 Expand width of HC from WS to (WS - M/2) Elseif RW = S6 If LW = S7 Expand width of HC from WS to (WS - M/2) Elseif LW = S8 Check next character width If next is a full character Expand width of HC from WE to (WE+ M/2) Else

Expand width of HC from WS to (WS - M/2) Endif Elseif RW = S7 Expand width of HC from WE to (WE+ M/2) Endif Endif The above algorithm fills the other half of the cell for the half characters that is found in the stripping of vertical segmentation. The resulting images after performing the vertical filling algorithm are horizontal segments which has a set of white rectangles representing only the full characters. All the half characters have then become equal size full characters. Fig. 8 (a) indicates that resulting image and the extra filling by the filling algorithm are represented in grey. (a)

(b)

Fig. 8 (a). Filled half characters by filling algorithm (b). Final row of Braille cell including space characters

3.6 Character Space detection

After created all the Braille characters as similar size cells, the space characters are detected. For that black widths which are more than the width of a full Braille cell are detected and they are filled with a white rectangle with the same size of full Braille cell in the middle of that black space using the Algorithm 1 mentioned above. Fig. 8(b) shows the final image including the space characters.

3.7 Character extraction

Fig. 9. Characters extracted from the segmentation As the final step the sets of pixels of the character rows are extracted from the previous inverted image corresponding to the positions that denotes a white rectangle in each row of Braille document. The pixels extracted can be stored in a Mat

764


structure for recognition of the letter/part of the letter that it denotes. Fig. 9 shows the characters and space characters extracted from the rows. The characters extracted using these segmentation methods are more accurate and it denotes the Braille cell symmetrically. Therefore it can be used as a clear input for the recognition of the character which is the next step in Optical Braille recognition.

4 Results and Discussion

The developed segmentation algorithm was tested with 100 A4 scanned images of various Braille text documents embossed printed in Sinhala Braille. The documents were taken from The School of Blind in Ceylon School of Deaf and Blind in Sri Lanka. The testing was done in java opencv environment and the image processing was performed in an Intel Core i3 PC with 2GB RAM. The average duration taken by the system to segment and save each of the Braille characters as separate images was 22s. According to the testing results retrieved overall precision and recall were calculated. It showed 99.22% amount of F1 score for the 100 documents tested. The overall Precision and recall for the number of characters extracted are shown in the following Table 1.

Total number of Documents 100

Total no. of Characters 49,905

False characters 652

Precision 98.81%

Recall 99.65%

F1 Score 99.22% a. Average result for 100 documents

Table 1: Precision and Recall Summary The images were taken for testing in 3 categories. Those are least noisy images, normal images and noisy images. The average result shown for the Braille text segmentation is denoted in the table below.

Image Type Least Noisy Normal Noisy image

99.6% 98.27% 85.2% b. Average result for 100 documents

Table 2: Testing Results

5 Conclusion

In the process of Optical Braille character identification segmentation takes a major role. If segmentation is not done in the accurate way the identified results get deviated from the correct result. Therefore employing efficient segmentation algorithm is very important. Though there are several segmentation methods existing such as grid based

segmentation, distance measurement segmentation and neighborhood checking there exist several limitations such as resolution dependency and other factors. The segmentation method proposed in this paper solve most of the problems exists there. In this algorithm the segmentation is happening in horizontally and vertically respectively. According to the results shown in the testing of new algorithm developed, this segmentation methodology can be used to extract the Braille characters in an efficient way. The output of the image of each character is almost symmetric so that, the extraction of the character is easier in Optical Braille Character identification. Other than that with adjustable dilation size this method can be used for various resolutions. The noise of the image is removed in several stages of the segmentation in addition to preprocessing so that the result has sufficient accuracy level. Since the segmentation is done first horizontally and then vertically, the characters in line by line can be obtained separately and the space character also can be detected easily. Distance measurement computations for all the dots in the Braille text is not involved in this image. Traversals through the image are less so that the processing power needed for the execution is less with this segmentation method.

References

[1] “Visual impaired and blindness”, fact sheet N282, World health Organization.

[2] A. Marwa Abdelmonem, B.M.H.El-Hoseiny, C. Asmaa Ali, D.Karim Emara, E.Habiba Abdel Hafez and F. Asmaa Gamal, “Dynamic Optical Braille Recognition (OBR) System”, International Conference on Image Processing, Computer Vision, & Pattern Recognition, IPCV 2009, volume 2 (2009).

[3] S.D. Al-Shamma and S. Fathi, “Arabic Braille Recognition and Transcription into Text and voice”, “5th Calro International Biomedical Conference”, Calro, Egypt, December 16-18, (2010).

[4] M.Y. Babadi and S. Jafari, “Novel grid-based optical Braille conversion: from scanning to wording”, International Journal of Electronics, volume 98, No. 12,, pp. 1659-1671, (2011)

[5] Y.A. AbdulMalik Al-Salman, “Arabic Optical Braille Recognition System”, ICTA’07, Hammamet, Tunisia, (2007).

[6] T. Acharya and A.K. Ray, “Image Processing Principles and Application”, Wiley Publication, ISBN-13 978-0-471-71998-4.

[7] J. Mennens, et al, “Optical Recognition of Braille Writing Using Standard Equipment”, IEEE transactions of rehabilitation engineering, volume 2, No. 4, (1994).

[8] X.F. Hermida, et al, “A Braille O.C.R. for Blind People”, Proceedings of The Seventh International Conference on Signal Processing Applications & Technology, ICSPAT-96. Boston (U.S.A.), (1996).

765