Distance and Color Invariant Automatic License PlateRecognition System

Nabeel Younus Khan II], Ali Shariq Imran['], Naveed Ali I']I'] Computer Science Department, National University of Science & Technology, Rawalpindi

Abstract - This paper proposes a novel technique to extract andrecognize the vehicle number plates from the still images/videos.Numbers from the plates are automatically extracted and thenmatched with the database to identify the cars owner. System hasbeen designed to work on the standard Islamabad ComputerizedNumber Plates (ICT). The method is simple and extremely fast,making it suitable for many real time applications. This method hasno distance, color and single plate limitations. The system has twomain modules (1) Plate Extraction Module (2) OCR Module. Wehave developed our own SCAN_NP algorithm for the first module.For the second module, we have used Template matching andneural networks to choose the effective method from them. We haveused this system on different ICT plates and we have got above90% accuracy with 2 Mega Pixel. With a higher quality camera thisaccuracy can be increased further.

Index Terms -Image Enhancement, Number Plate Extraction,OCR module, Template Matching, Neural Network.

I. INTRODUCTION

Automatic license plate recognition system is an importantarea of research due to its many applications. Examples are:Border crossing control, Identification of stole vehicles,Automated parking attendant, Petrol station forecourtsurveillance, Red light camera, Speed enforcement, Securityetc. There is a growing demand for traffic data concerning,traffic flow and automatic vehicle identification. Increasinglevel of the road traffic has led researchers around the world toadopt advanced electronic and advanced computer visiontechnology to monitor and to control traffic. Among thecommercial license plate recognition systems availableworldwide are Car Plate Recognition by J.A.G. Nijhuiset.al.[3], Car Plate Reader (CPR) by Rafael et.al.[4], OpticalCar by Emiris and Koulouriotis [7] and Automatic NumberPlate Recognition (ANPR) by Shyang- Lih Chang [5] andMehmwet Sabih Aksoy et. al. [6].

This paper describes a novel system which can be used forthe automatic verification ofnumber plates. Extraction of platefrom the image is a difficult task as there are many rectangularregions in the image along with the plate. Moreover lightingproblem also makes the extraction of plate much difficult. Incase of colored plates the technique which works on peakvalley mechanism fails as color of the background changes. Soautomatic adjustment is needed. The characters in the imagemay become little bit distorted and proper mechanism is

required to recognize them. Our system almost handles mostof these issues.The use of fingerprints for human identification has been

used from a long period of time. Archaeological evidencesreveal that fingerprints have been used as a form ofidentification since 7000 BC [1]. Fingerprint matching hasbecome an important way for accurate personal identification.Fingerprint is a pattern of ridges and furrows on the surface offingers. These patterns are formed by a set of ridgelines,which sometimes terminates (ridge ending) or intersects (ridgebifurcation). Such ridge-endings and bifurcations form a set offeatures know as minutiae. Various approaches of automaticfingerprint matching have been proposed in the literature.However fingerprint identification systems rely on one of thetwo classical methods i.e. Minutiae based methods [2]-[4] andImage based methods [5]-[7]. Minutiae-based techniques firstlocate the minutiae points, and then match their relativeplacement in a given finger image and the template. Thisapproach depends heavily on pre-processing and post-processing operations in order to extract the reliable minutiaefeatures from the fingerprint image. So these techniques arecomputationally complex. The main drawback of theseapproaches is that errors can propagate from minutiaeextraction down to the decision stage [6].

II. PREVIOUS WORK

Number plate recognition is a well know problem of ImageProcessing. Different work has been done in this field.Nowadays, most of the plate regions are extracted by thecolored image processing techniques. These methods aremainly as follow: (i) first, segmenting the colors in image byneural network. Second, the horizontal and vertical projectionsof plate's background color are calculated. Then the plate'sregion is located with the ratio of plate's height and width [2].(ii) Using color edge detection combined vehicle plate'sbackground in different provinces. Otherwise, the color of theplate would fade when the plate has been used for a long time.The same color's RGB will be not same in differentluminance, either. But all these works have some limitations.Distance of the car from the camera is a major issue in most ofthe approaches. Most of these approaches only used one fontand their accuracy was not good. In most of these approaches,the problem was the extraction of the plate. Some of themused the Hough Transform while others use the plates flag

1-4244-1494-6/07/$25.00 C 2007 IEEE

color information for the plate verification. But suchtechniques are error prone and expensive too.

This paper is organized as follows. In the third section theSystem model is presented. Plate extraction module isdiscussed in section 4. Optical character recognition (OCR)module will be described in section 5. Finally results will bediscussed in Section 6.

III. SYSTEM MODEL

Our proposed system is divided into two importantmodules:

1. Plate Extraction Module2. OCR Module

Each module has a number of phases which are executedone by one to authenticate the number plate of the car. Thebasic proposed system model is being shown in Figure 1.

Image forfani gacl~'

~~~>w*

o"..mmu "POehv-pro i7 llv"i

nwe vAh r::* _.- - _ __.__.-._.

lgridu

Me rvgthW n

.>clerlen W_rIIUfIb- R_ gisd JR~gsru

Figure 1- Proposed System

IV. PLATE EXTRACTION MODULE

The extraction module is further subdivided into ImageAcquisition, Image Enhancement & Plate Segmentationmodules respectively. Details about each sub-module arespecified as follows.A. Image Acquisition

images simply they can be loaded from the menu. In case ofmovie, frames are picked one by one and treated as stillimages. So basically we have made an AVI player whichextracts frames form the movie. For frame extraction we hadused the vfw32 library. Simple captured image is shownbelow in Figure 2.

B. Image Enhancement

In this module numbers of operations are performed on theimage to convert it into such form suitable for the OpticalCharacter Recognition module. All these operations arespecified as follows.1. Grayscale Image

First image is being converted into grayscale i.e. each pixelwill now have one value ranging from 0 to 255 via the (1).

I(i,j) =0.2989*R+0.5870*G+0.1140*B (1)

Where I (ij) represents the grayscale image.

2. Edge DetectionIn edge detection we try to find out the edges in an image.

Basically it indicates the change in contrast. At all placeswhere change in contrast occurs, edges will be drawn. Foredge detection we have used sobel edge detector due to itsbetter results. Sobel maks used by us are shown in Figure 3.

0 +1

-2 0O |.2

-1 0 +1

ex

±41 42 41

0

-1 -2 -1

Gy

Figure 3 - Sobel Masks

This is the first module in the NPRS system. There arebasically following three ways of acquiring an image [8].

1. Using a conventional camera and a scanner.2. Using a digital camera3. Using a video camera.

We have used the digital camera up to 2 Mega pixels. Oursystem can work on still images and videos as well. In case of

3. Brightness and BinarizationThe system automatically sets the brightness of the

image as image may be taken at evening or during the daytime. For brightness we take power of each pixels value asdepicted by (2). Resulting image will have now brighter edges.Now we can binarize the image without fear of losing anyedges. Image is then binarized by using a suitable thresholdvalue. So the resulting image will only contain black andwhite pixels.

I(i,j) = I(i,j)P (2) 6. Binarization

4. Morphological ProcessingMathematical morphology is a part of digital image

processing which is concerned with image filtering andgeometric analysis by using structuring elements (SE).Dilation and erosion are two fundamental operations in thealgebra of mathematical morphology. Dilation is a

morphological transformation that combines two sets by usingvector addition of set structural element. On the other handErosion can be obtained by dilating the complement of theblack pixels and the taking the complement of the resultingpoint set. Let B denote a structural element which is a 3x3square. The dilation and erosion of the image A is shown inFigure 4 and Figure 5.

Figure 4 - Illustration of Dilation

......

.HH R1.....

|I

Figure 5 - Illustration of Erosion

After binarization, edges which we get are broken. Theedges of the plate are normally broken and not continuous. Sowe have used dilation with suitable small square mask to fillup the broken edges. Following formula [9] is being followed:

A 8 B ={z [(B)z rZ]c A} (3)

5. Gaussian Mask

We further applied the Gaussian mask to thicken the edgeswhich may be lost and could not be recovered via dilation.Following is the Gaussian mask which has been used as

indicated in Figure 6.

Now image is finally binarized by choosing a specificthreshold. Now it is ready for the plate extraction. The pre-

processed image is shown in Figure 7.

Figure 7 - Pre-Processed Image

7. Plate SegmentationFor plate segmentation we have developed our own flexible

algorithm i.e. SCAN MP which can find out multiple objectswhich are or just like number plates. Beauty of the algorithmis that during the scanning it tries to eliminate the noisyobjects thus reducing the processing time. The algorithmfinds out only those objects which have horizontal and verticallines. It checks for multiple rows and columns to ensure thecorrect extraction ofthe object. Only objects having four sideswill be extracted as shown in Figure 8 (found object is shownby filling it with green color). So most of the noisy objectswill be rejected. Found objects are subject to the recognitionmodule.

1

1 5

Figure 6 - Gaussian Mask

Gaussian mask can also be generated via (4):

_ (x*x)+(y*y)

e 2*var iance 2

Figure 8 - Object Recognition

V. OPTICAL CHARACTER RECOGNITION MODULE

This is the final crucial module of the system whichidentifies the number plate of the car. It performs a number ofoperations which are stated as follows. Not only number isrecognized but plate is also validated i.e. whether it is a noisyobject or not.

A. Plate Validation

ICT plates have certain characteristics. These plates have(4) specific width/ height ratio and specific ratio of black pixels.

2 4 5 4 2

4 9 12 9 4

5 1 2 1 5 1 2 5

4 9 12 9 4

2 4 5 4 2

Infect every plate has certain attributes. In this phase, thesefeatures are used to validate that the region is plate or not. Ifobject passes through these tests successfully it means that itcan be a plate. The region is then subjected to isolation.B. Characters Segmentation

In isolation, we have used the modified peak to valleytechnique First step is to identify the region in which we canapply the technique to isolate the characters. The wholenumber in the plate has some distance from the boundary ofthe plate in all directions. We had determined the distanceratios which can be used in images of any size e.g. let's saythat numbers are at 20% ratio from the top. So we took 20% ofthe height of the plate and add it to the starting value (i.e. 0) toget the starting row. Similarly via subtracting and adding,other perimeters will be found and a boundary will be drawnaround the number as shown in Figure 9. Boundary has beenshown via green lines.

the required character/ digit. We have designed separatetemplates for digits and characters.

n'l m-l

T*8I(X, Y) = E = E T(i, j)*8I(X +i, Y +j) (5)i=o j=O

Template matching is faster but it didn't provide betterresults. So we had also used neural networks [ 1] (perceptron)in order to get better accuracy. We have trained the neuralnetwork with all numbers (i.e. characters and digits) in their 4different styles (i.e. bit noisy, oblique) with theircorresponding weights leading to a total of 144 trainingsamples. In the process of recognizing characters from anynumber plate, each character on the number plate isrecognized individually. So the characters on plate are treatedindividually. Each character is taken one by one and then it ismatched against all the training patterns. In case of match,corresponding output is being stored and process takes thenext character. The process continues till last character onnumber plate is being matched with the training pattern. Atypical Perceptron diagram which is used for the classificationis shown in Figure 11.

After finding the boundary in the plate, we have used thepeak to valley technique to isolate the characters. We havemodified the actual algorithm. We first scanned columns toget the starting and ending columns. We then moved row wiseto get the starting and ending rows for each number. Theisolated characters are shown in Figure 10.

Figure 11 - Perceptron for Recognition

C. Validation

We further checked that the region is a plate or not. If fiveobjects were found in the plate after the isolation it means thatthe region is the plate. Then recognition will be performed elseregion will be rejected and the next found region will beisolated once again. This process will continue until no otherregion (plate) is left. This thing aids in improving theefficiency of the systemD. Character Recognition

For character recognition, we have used two techniques i.e.neural networks and template matching.

In template matching, we had stored the templates of all thecharacters. Since distance is not an issue the extractednumbers can be of any size. So according to the size of thecharacters, stored templates are stretched (normalized) [10].The characters are then matched with the normalizedtemplates and a matching score is calculated as depicted by(5). Template giving the maximum score is considered to be

In our system we had recognized not only the numberof the care but also the district. So these recognitions areperformed in the way described as below:1. Car Number Recognition

In case of Templates, the numbers of the car are matchedwith all possible templates. The template which gives the bestscore is considered to be the required character representation.Digit templates are used for the digits while character templateis used for characters only.

In case of Perceptron, every determined number is fed intothe Perceptron and output is obtained. Training data was inform of O's and l's. Testing data will also be in form of O's andl's. So whole number will be figured out.

Finally the recognized number obtained by any of the abovemethods is matched in the database to display the informationabout the cars owner.

2. District RecognitionThe district is also isolated from the plate as shown in

Figure 10. For district boundary determination we had alsoused the determined distance ratios. All district characters arevery closely connected to each other so it's not possible to

isolate them. So we had decided to use template matching forthe districts. We stored some of the complete templates of thedistricts in our system and had used them for recognition. Wealso trained the system with other district templates such asdistrict templates from NWFP plates to check the recognitionaccuracy. The results are good. District is recognized and thenit is printed on the screen.3. District RecognitionThe district is also isolated from the plate as shown in

Figure 10. For district boundary determination we had alsoused the determined distance ratios. All district characters arevery closely connected to each other so it's not possible toisolate them. So we had decided to use template matching forthe districts. We stored some of the complete templates of thedistricts in our system and had used them for recognition. Wealso trained the system with other district templates such asdistrict templates from NWFP plates to check the recognitionaccuracy. The results are good. District is recognized and thenit is printed on the screen.

VI. SYSTEM FLEXIBILITY

Our proposed system is much flexible due to followingreasons:

1. First we have provided adjustment of each and everything in the form of dialog boxes. Any frame orimage can be preprocessed manually as well.

2. Secondly our system can detect and recognizemultiple plates as well as shown in Figure 12.

3. Colored plates can be recognized easily.4. Different sized plates can be found and recognized.5. Can detect plates in darker and brighter images!6. We have run our system on some NWFP number

plates. The results were very good. It recognized thenumber and district of those cars also with a goodaccuracy.

So the developed system is easily extensible and can bemodified by adding new modules.

VII. EXPERIMENTAL SETTINGS

We initially checked out plate extraction module on 200images/ scenes. Module showed very good accuracy of above90%. We then tested our system on these 200 different imagesfor recognition via two matching techniques i.e. template

matching and neural network. We then modified the color of100 plates of the cars. Even then the results were promising.We also put some noise on the number plates of about 100images. Our system also handled these images fairly andperformed recognition effectively. We then checked imagestaken during the night. Plate color was little bit changed in thenight and it became darker. Since our system is color invariantso it also provided good accuracy in such images i.e. above85%. The higher accuracy and good speed has been achievedvia our proposed system which is indicated in the Table 1.

TABLE 1 EXPERIMENTAL RESULTS

Number Percentage TotalImages

Detected Plate 187 93.5 200

Number Recognition 172 86 200(Template Matching)

Number Recognition 182 91 200(Neural Network)

Modified Color Plates 94 94 100Detection

Number Recognition in 91 91 100Noisy ImagesNumber Recognition in 87 87 100Dark InagesDistrict Recognition 91 91 100

So neural networks have provided better matching for therecognition as compared to the template matching. So we willrecommend using neural networks in the future.We also checked our system on some NWFP number plates.

On these plates, number plate was extracted correctly.Because these standardized plates have nearly the sameattributes which ICT plates have. Recognition rate was alsogood with the neural networks which gave us accuracy ratioup to 85%. With template matching, accuracy went down to76% as shown in Table 2.

TABLE 2- EXPERIMENTAL RESULTS

Number Percentage Total Inages

Detected Plate 69 93.24 74

Number Recognition 56 76 74(Template Matching)

Number Recognition 63 85 74(Neural Network)

District Recognition 66 89 74

Each NWFP plate has different districts as compared to ICTplates where every plate has only one district. So accuracy ofdistrict recognition is properly checked with NWFP plates

which come very good. The accuracy of the system can beincreased by providing more training samples.

VIII. CONCLUSION

This paper has presented a novel technique to identify thenumber plates of the moving vehicles. Proposed technique isnot only faster but is also efficient. Proposed system is veryflexible. New modules can be added and removed. Platerecognition module works fairly well. In order to improve theaccuracy of the proposed system in the future followingmeasures should be taken:

1. Focus should be made to use the suitable featureextraction technique which can extract importantfeatures from the characters. Those feature vectorsthen can be matched by any of the classifiers.

2. Attention should also be given to handle the occludedobjects in number plate. Sometimes the numberpixels are lost while capturing. So during thematching, recognition rate goes down. So if we canredraw that character up to some extent, accuracy ofthe system can be improved further.

3. With efficient and complex neural networks,recognition rate can still be increased.

So if above measures are taken then the proposed systemaccuracy will increase up to more subsequent level.

REFERENCES

[1] Ji-Lin. Li, Jia-Tao. Song, Li-Ya. Ding, "Vehicle LicensePlate Recognition System with High Performance",ACTA AUTOMATICA SINICA, Vol.29, No.3, p. 457-465, May.2003.

[2] Wei. W, X. Huang, M. Wang, "An automatic system ofvehicle number-plate recognition based on neuralnetworks", Journal of Systems Engineering andElectronics, v 12, No. 2, p. 63-72, 2001.

[3] J.A.G. Nijhuis, M.H ter Brugge, and K.A. Helmholt," Carlicense plate recognition with neural network and fuzzylogic", In IEEE Proceedings of International Conferenceon Neural Networks., volume 5, pages 2232-2236. IEEE,27 Nov-1 Dec 1995.

[4] J.Barosso, E.L Dagless, A.Rafel, and J. Bulas- Cruz,"Number plate reading using computer vision", InProceedings of IEEE International symposium onIndustrial Electronics., volume 3, pages 761-766, 1997.

[5] Shyang-Lih Chang, Li shien Chen, Yun-Chung Chung,and Sei-Wan Chen," Automatic license platerecognition", IEEE Transaction Intelligent transportationsystem, 5:42-53, 2004.

[6] Mehmwet Sabih Aksoy, Gultekin Cagil, and AhmetKursat Turker," Number-plate recognition using inductivelearning", Robotics and Autonomous systems, Elsevvierscience B. V., 33:149-153, 2000.

[7] D.M.Emiris and D.E.Koulouriotis, "Automated opticrecognition of alphanumeric content in car license platesis semi structured environment", In Proceedings of

International Conference on Image Processing., volume 3,pages 50-53, 7-10 Oct 2001.

[8] J. Cowell, and F. Hussain, "A fast recognition system furisolated Arabic characters ", Proceedings SixthInternational Conference on Information andVisualization, IEEE Compute rSocieg London, England,pp. fi50-654,2002.

[9] Cheokman Wu, Lei Chan On, Chan Hon Weng, Tong SioKuan, Kengchung, "A Macao License PPlate RecognitionSystem", Proceedings of the Fourth InternationalConference on Machine Learning and Cybernetics,Guangzhou, 18-21 August 2005

[10] M J. Ahmed, M Sarfaz, A. Zidouri, and K G. AI-Khatib,"License Plate Recognition System", 2003 IEEE.

[11] Handbook of "Fundamentals of Neural Networks" ByLaurene Fausete, 2003.