Number plate recognition for use in different countries using an improved segmentation

NCETACS 2011

ANPR – Automatic Number Plate Recognition

Transborder TrafficControl AuthoritiesTransborder TrafficControl Authorities

Car Log inParking areasCar Log inParking areas

Road Security (Check on notoriousDrivers in black list)

Road Security (Check on notoriousDrivers in black list)

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 2

Primarily developed to cater to the needs of the law enforcement agencies.

An important figure worth mentioning in this regard is that Britain itself has 10,502 ANPR and most of their locations are kept secret. Thames Valley police, which has released details of spending but not locations, has put nearly £2m into 47 fixed cameras, 31 in road vehicles, 11 portable kits and one in a helicopter.

Data courtesy Guardian.co.uk

Input image Pre-processor

Analyzer

Recognizer

Segmentation

UnitOutput

PercentageAccuracy

The approach do handles the entire ANPR module addressing each of the steps but the novelty lies in the segmentation scheme adopted

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 3

Software part

Pre-processing

Image denoising

A statistical Median filter is used to remove salt and peppernoise from the image in grayscale before binarizing. we have used a 3 × 3 masking sub window for this purpose.

Adaptive Thresholding

Both Otsu method and Ni back’s method were tested. Otsu method was finally used as it is globally adaptive which would increase processing speed as compared to Niback’s threshold scheme.

Without Filtration

After Filtration

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 4

Segmentation

Clustering of white pixel zones

Component labeling of the clusters

Sorting the component clusters

Directional region growing of the clusters

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 5

MOTIVATIONAlphanumeric characters of the License

Plate are the ones that have the

higher pixel count among

the pixel clusters

Clustering of white pixel zones

Clustering

The clustering of the pixels are done on the basis of an eight connected neighborhood of the white pixels.

Since wiener filtration was used previously so Impulse Noise was largely eliminated, hence Algorithm works more on relevant data having less noise

Brings downProcessing time

Test Image from Jerome Coninx database

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 6

Component labeling of the clusters

Each white cluster is labeled with a particular component tag

Component tag : Algorithm scans the entire image and assigns a number to each cluster that it

faces. The number is initialised by 0 and incremented by one when it jumps to the next

cluster

Number of pixels in each pixel cluster is recorded against the component tag and the position of

each cluster (corner co-ordinates) are noted

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 7

Sorting the component clusters

Pixel count in each cluster is then sorted in a descending manner

Number of characters (n) specified by the Law Enforcement Agencies is taken as the input and a buffer

of (2n-2) is set

Graph showing number of pixels in each cluster against the order in which they appear after sorting

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 8

The value of (2n-2) is determined empirically to cope up with the

over segmented characters

Directional region growing of the clusters

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 9

A problem still persists that many over segmented characters that have entered the calculation because of the buffer value

(2n-2) set. Now the need is to associate these glyphs into relevant characters

What we presently haveA sorted matrix of the

pixel clusters which has (2n-2) number of

members

A matrix containing the positional information of

the clusters

Directional Region Growing is used based on the observation that distance within glyphs of the same character is less than that within glyphs of different

characters

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 10

Directional region growing of the clusters

Use this pixel as seed , join the region between the two pixel horizontally

Dilation of the joined line

Re-label the entire image using 8-connected neighbourhood

Sort the pixel count and check the condition again

The entire algorithm is repeated again using (y-axis) distance check and

comparing distances between lower most point of upper cluster and uppermost

point of lower cluster . The process stops when minimization of the number of

characters is not further possible

Using the positional information (x-axis) check the dist between the rightmost pixel of a cluster and the leftmost pixel of the cluster next to it. If dist<dcritical

T in the upper row isapproximated by horizontalRegion growing and 7 by Vertical region growing

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 11

Segmentation and Normalization

Segmentation

The individual characters are then segmented using

bounding box

Now the glyphs do vary Greatly in shape so ….

NormalizationThis normalization is done on the basis of size of the extracted

images. All of them are scaled to [15x15] pixels

Segmented and normalized arranged according to positional information

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 12

Recognition Module

Automatic Neural Network (ANN) based recognition scheme

It consisted of 225 input nodes

36 output nodes (26 uppercase letters and the 10 digits)

1 hidden layer with 300 neurons

The activation function

Weight update function(α is the learning term β is the momentumParameter E is the error term)

( slope parameter in the sigmoid function is set to 1)

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 13

Database and train set

The database comprised of 150 different images of license plates used in 58 different countries of the world.

Entire Test Dataset Availableat www.ankushroy.webs.com

75 images were used for training and the rest used as test set

The individual pixel values were used as the input of the 15x15 binary imageof individual characters segmented

Here the module has the option of allowing the end user to select the appropriateImages (75). Just name the countries and the network selects them from the pool of images

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 14

Experimentation and error analysis

The percentage accuracy here is based on the character wise reconstruction of the license plate after passing through the Recognizer.

The skewness of the number-plate and improper lighting condition in many cases are the main limiting factors that affect the recognition percentage adversely

Calculated over the entire set a accuracy

of 91.59 % was reached

Prof . Anjan Kr. Rakshit, Department of Elect Engg. Jadavpur University, Kolkata,

[1] Vehicle Registration Plates of India. Available: http://en.wikipedia.org/wiki/Vehicle_registration_plates_of_India

[2] Ward Nicholson, “License Plate Fonts of the Western World”,Available:http://www.leewardpro.com/articles/licplatefonts/licplate-fonts-intro.html

[3] Parking and Traffic Technologies Ltd, Smartreg ANPR, Available:http://www.parkingandtraffic.co.uk/ANPR/smartreg-anpr

[4] J.A.G. Nijhuis, M.H ter Brugge, and K.A. Helmolt, “Car License Plate recognition with network and fuzzy logic”, in Proc. Of IEEE International Conference on Neural Networks., volume 5, pp 2232-2236, Dec 1995

[5] Shyang-Lih Chang, Li Shein Chen, Yun-Chung Chung, and Sei-Wan Chen, “ Automatic license plate recognition” IEEE Transaction Intelligent Transportation System, 5:42-53,2004

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 15

Thank You

Any Questions??

Ankush Roy & Debarshi Patanjali Ghoshal Jadavpur University 16