Computers and Electrical Engineering 39 (2013) 863–872

Contents lists available at SciVerse ScienceDirect

Computers and Electrical Engineering

journal homepage: www.elsevier .com/ locate/compeleceng

Rapid vehicle logo region detection based on informationtheory q

0045-7906/$ - see front matter � 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.compeleceng.2013.03.004

q Reviews processed and approved for publication by Editor-in-Chief Dr. Manu Malek.⇑ Corresponding author.

E-mail address: maosongan@gmail.com (S. Mao).

Songan Mao a,⇑, Mao Ye a, Xue Li b, Feng Pang a, Jinglei Zhou a

a School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, PR Chinab School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Queensland 4072, Australia

a r t i c l e i n f o

Article history:Available online 6 April 2013

a b s t r a c t

Vehicle logo detection is an important task in intelligent transportation systems. In thispaper, a novel method is proposed for detecting the vehicle logo in an image. Our methodconsists of three main steps. First, horizontal and vertical direction filters are applied to theoriginal image to produce two new images. Then, a saliency map is generated from eachimage. Second, two clusters in the corresponding saliency map are formed to create a bin-ary image. Finally, the vehicle logo is localized by searching the regions with the maximumuseful information. Our method has two main contributions. One is that the vehicle logocan be detected rapidly without learning. The other is that our method is adaptable to dif-ferent situations without adjusting the parameters. A series of experiments are performedon 970 images, which are captured from different real-time situations. Experimentalresults show that our method is also very fast and can achieve a high detection rate, whichis suitable for real-time applications.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction

In recent years, with the increasing number of forms of transportation and complex modern transportation networks,intelligent transportation systems play an important role in our daily life. The systems’ function is to monitor traffic action,to record vehicle information, and to report abnormal traffic events. License plate (LP) recognition, an important function ofthis system, is an essential part of our daily life. However, LP information is not always available. When it is missing, coveredor forged, the system cannot get the appropriate vehicle information. Therefore, the manufacturer’s iconic logo [1] and vehi-cle logo [2] detection and recognition offer another way to record and track automobile information. Because of the differentvehicle models and designs, the location of the vehicle logo appears in different places, as shown in Fig. 1. If we can localizethe vehicle logo, the accuracy of vehicle logo recognition will be guaranteed.

Vehicle logo detection is a difficult problem in the area of computer vision. Because of the numerous variances in logos,they are difficult to classify. Recently, some methods have been proposed to solve the problem of vehicle logo detection. Onemethod is that a cascade classifier is trained based on wavelets and Adaboost [3]. However, this method mainly relies ontraining samples. Motivated by the observation that a vehicle logo has sufficient edge and corner information, a methodusing edge detection, projection and morphological filter is presented in [4,5]. But this method needs an ideal coarse local-ization of a vehicle logo with little background and noise information. Therefore, the results are not consistent in differentillumination conditions. By matching the samples with vehicle headlights, grille and texture information, a coarse-to-fine

Fig. 1. The first row represents vehicle logos appearing in the middle of the radiator grille. The second row represents vehicle logos appearing on the hoodof the vehicles. The third row represents vehicle logos appearing on complex grilles. Also, the colors differ. (For interpretation of the references to color inthis figure legend, the reader is referred to the web version of this article.)

864 S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872

vehicle logo detection method is proposed in [6,7]. All the methods mentioned above are sensitive to the light reflections anddifferent view points, and complicated parameter settings are needed.

In our work, we combine bottom-up and top-down approaches to detect vehicle logos. In the bottom-up process, twoimages are created applying the horizontal and vertical Sobel operator filters respectively. Then a saliency map [8] is gen-erated from each image. In the end, two clusters in the corresponding saliency map are formed to generate a binary image.The top-down process uses the theory of information entropy. First, a minimal criterion of the image’s information entropyvalue is used to choose a suitable binary image. Then, the vehicle logo region is located with a maximum criterion of im-proved information entropy theory.

Our method is capable of searching for the vehicle logo in an image without learning. For changing environments, themethod is capable of detecting vehicle logos without adjusting the parameters. Thus, our method is very suitable for prac-tical applications.

The remainder of this paper is organized as follows. In Section 2, we first introduce the coarse localization process, andthen show the flow chart of our method. The subsequent three sections present the details of our method. In Section 6, theexperimental results are presented to evaluate our method. We conclude this paper in Section 7.

2. Preliminary

Since environments are constantly changing, detecting a vehicle logo directly from an image is not reliable. By observingand analyzing the vehicle logo location, we find that most logos appear in the area above the LP region. Therefore, we need toextract this coarse region first.

Fortunately, many LP detection methods have shown good results [9–12]. Suppose we obtained the original widthand height of the location of the license plate, x and y respectively, then adjust them to the same defined standardscale as x0 and y0. For different vehicle types, in order to get a coarse region that includes the vehicle logo, we choosetrucks and other large scale vehicles (the distance between the vehicle logo and license plate is farther than on smallvehicle types). Then we measure the distance L (a fixed parameter in our method) between the vehicle’s front wind-screen and LP. Therefore, the new width and height of vehicle logo coarse region become x0 and L respectively. Forsome vehicle types, such as a private car or Jeep, the coarse region may include some extra noise information, suchas the vehicle hood, front windshield and windshield wipers. The vehicle logo coarse localizing process is illustratedin Fig. 2.

The process of our method is illustrated in Fig. 3. The coarse localized vehicle logo region is the input image. Then, we uselinear direction filter methods to generate the horizontal and vertical filtered images respectively. Based on the visual atten-tion model [8], we get the corresponding saliency maps. After image pixel value clustering, the saliency maps are convertedto a binary image. And then we choose the binary image with the information entropy. Finally, we use the improvedinformation entropy to detect the vehicle logo region.

Fig. 2. The process of vehicle logo coarse localization.

Fig. 3. The flow chart of our method.

S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872 865

3. Direction filters

The basic principle of the efficient coding theory [13] is to suppress the redundant information and retain the useful infor-mation. Therefore, in our work, the image information H(Image) can be decomposed into the vehicle logo information H(Ob-ject) and the rest of image information H(Background), which is illustrated as follows:

HðImageÞ ¼ HðObjectÞ þ HðBackgroundÞ ð1Þ

In order to find the distinct features, we separate 970 vehicle images into three groups. The three groups are the wholeimage, the vehicle logo region and the region without the vehicle logo, respectively. Then we calculate the histogram of gra-dient direction. The results are shown in Figs. 4–6. In the figures, the horizontal axis represents the degree of direction andthe vertical axis represents the number of pixel points in respect to each direction.

It can be observed that the whole image mainly consists of the horizontal and vertical direction information. The regionwithout the vehicle logo also mainly consists of the horizontal and vertical direction information. The vehicle logo region

0 30 60 90 120 150 1800

100

200

300

400

500

600

Direction degree

The

num

ber o

f pix

el p

oint

s

Fig. 4. The histogram of gradient direction of the whole image.

0 30 60 90 120 150 1800

5

10

15

20

25

30

35

40

Direction degree

The

num

ber o

f pix

el p

oint

s

Fig. 5. The histogram of gradient direction of the vehicle logo region.

0 30 60 90 120 150 1800

50

100

150

200

250

300

350

400

450

Direction degree

The

num

ber o

f pix

el p

oint

s

Fig. 6. The histogram of gradient direction of the region without vehicle logo.

866 S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872

Fig. 7. The first column shows the original images. The second and third columns represent the filtered images after using the Gabor filter and Sobeloperator filter, respectively.

S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872 867

does not have specific direction information. This property indicates that the vehicle logo region is distinguishable. There-fore, we can use direction filters to remove the unwanted areas using either the Gabor or Sobel operators.

The Gabor filter has been widely used in many image processing problems, such as texture segmentation, edge detectionand image representation [14]. In the method, we set the scale factor r 2 [2–4], and the orientation factor h 2 {0� ,90�} torepresent the horizontal and vertical directions respectively; then we get the image O(r,h) from the image I [8]. After nor-malization, we can obtain a horizontal filtered image OH and a vertical filtered image OV; the results are drawn in Fig. 7.

The Sobel operator is also a popular method used in image processing problems [15]. The principle is convolving the im-age with a small, separable, and integer valued filter in both the horizontal and vertical directions. The direction Sobel tem-plates are constructed with 3 � 3 operator masks, I represents the original image, EH and EV are two images which at eachpoint contain the horizontal and vertical derivative approximations, the computations are as follows:

EH ¼�1 �2 �10 0 01 2 1

264

375 � I ð2Þ

EV ¼�1 0 1�2 0 2�1 0 1

264

375 � I ð3Þ

where ⁄ here denotes the 2-dimensional convolution operation. The results using a Sobel orientation filter on the originalimage are also drawn in Fig. 7.

By comparing the methods of the Gabor filter and Sobel operator filter, we find that the performance and execution speedof the Sobel operator filter are much better than that of the Gabor filter. Thus, we choose the Sobel operator filter as an idealdirection filter.

4. Generating the binary image

In order to detect the vehicle logo region, we need to generate a binary image that will separate the object from the back-ground. So we apply Itti’s saliency map and k-means clustering algorithm to generate this binary image.

4.1. Vehicle logo saliency map

Itti et al. [8] proposed a visual attention model based on a biologically plausible architecture. This model can detect a tar-get object that differs from surrounding destructors by its unique size, intensity and color. Therefore, we use Itti’s model toroughly highlight the vehicle logo region.

When the filtered images EH and EV are obtained, nine spatial scale images E(r) are created using a dyadic Gaussian pyr-amid [16], where r = {0, 1, . . . , 8}. Then we apply the algorithm of center-surround differences (defined as H) to images with

868 S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872

different scales. Suppose the center fine scale is defined as c, and a surround coarser scale is defined as s; the relations andequations are illustrated as follows:

s ¼ c þ d ð4ÞEðc; sÞ ¼ jEðcÞHEðsÞj ð5Þ

where c 2 {2,3,4} and d 2 {3,4}.From this principle, six different maps are created. First, we normalize these maps to the same size as EH and EV, and then

they are combined and normalized to create the saliency maps SH and SV.

4.2. Vehicle logo binary image

Usually, the saliency map only represents the object’s coarse salient regions, so many of the state-of-the-art object detec-tion methods are window-based search algorithms. In order to locate the object’s region precisely and quickly, we need toobtain the specific object-based binary image. In our case, the foreground information (vehicle logo and noise) in the saliencymap is distinguishable, so we can easily cluster the image pixel values into two groups. The pixel values of one group are setto zero in the background while the pixel values of another group are set to 255 in the foreground.

The method of K-means clustering aims to separate n observations into k clusters and then put these observations intotheir nearest cluster [17]. In our work, we need to cluster the saliency map into two groups, for background and foregroundimage information. So the k-means algorithm is applied to form the binary image.

However, because the binary image is generated based on a saliency map, some false clustered pixel points may lead todisconnection in the foreground regions. In our work, the closing operation, dilation followed by erosion, is used to fill in theholes and gaps between the foreground regions.

5. Vehicle logo detection

In this section, we will introduce how to choose a suitable binary image and then detect the vehicle logo region from it.

5.1. Choosing one suitable binary image

By analyzing the binary images, we find that both of the horizontal and vertical filtered binary images contain the vehiclelogo region, but the highlighted regions from these two images contain noise and unwanted blobs, such as the vehicle hoodand windshield wiper, etc. To improve the detection rate, we need to choose one binary image that contains less foregroundinformation. Therefore, information entropy theory can be used to choose the image.

According to the principle, the information entropy value increases with the degree of disorder and complexity of the im-age. The image with the smallest information entropy value means that it contains less noise and unwanted information. Wedenote the entropy En of candidate images Bj with possible pixel values {x1, . . . , xn} and probability mass function p(x) as:

EnðBjÞ ¼ �Xn

i¼1

pðxiÞ log pðxiÞ ð6Þ

where j 2 {Horizontal,Vertical}, x is the pixel value range from 0 to 255.

5.2. Detecting the vehicle logo region

After a suitable binary image is chosen, the highlighted regions are changed back to the original gray image. Generally, thevehicle logo region is the area that contains the most information. Given these regions (x1, x2, . . . , xk) from the image, weadjust them to the same scale Ri, N denotes the number of different pixel values in this region. The region with the largestinformation entropy value is assigned as the vehicle logo region. The improved information entropy theory is defined asfollows,

En0ðRiÞ ¼ N � EnðRiÞ ð7Þ

6. Experiments

Our method is implemented by programming language C++ on an Intel Pentium Dual-Core E5700 PC with 2.0 GB mem-ory. The details of our methods steps are illustrated in Fig. 8.

First, we convert the vehicle logo coarse localized images to gray images, then create their horizontal and vertical direc-tion filtered images. Second, we use the visual attention model to create a corresponding saliency map and then form thebinary image. Third, we use the information entropy theory to choose one of these binary images. Finally, we change thehighlighted regions back to the gray image, then locate the vehicle logo region from it.

Fig. 8. Steps for detecting vehicle logos: (a) the initial images, (b) the gray images, (c) the direction filter results, (d) the saliency maps, (e) the binaryimages, (f) the binary images after the mathematical morphology operation, (g) the chosen binary images, and (h) the detection results.

S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872 869

6.1. The experimental dataset

Our dataset consists of 970 digital images captured from traffic surveillance videos in areas, such as highways, roads andparking lots. These images can be roughly categorized into three groups. Each of the groups’ examples are shown inFigs. 9–11.

In the first group, the vehicle appears in different sizes and scales under the same illumination conditions. The sec-ond group of images are taken from different view points, including left, right and tilted. In the third group, the vehi-cle logos are exposed to different illumination conditions, such as in sunshine, on cloudy day, at twilight andnighttime, etc.

6.2. Experiment results and analysis

Our experimental results are summarized in Table 1. For the first test group of vehicle scale differences, due to first resiz-ing them to the standard vehicle scale, we have a high detection rate. In the second test group, because both the horizontaland vertical direction filters are effective for different

Fig. 9. Vehicle logos in different scales: (a) vehicle shows up in small scale, (b) vehicle shows up in medium scale, (c) vehicle shows up in large scale, and (d)vehicle shows up in huge scale.

Fig. 10. Vehicle logos in different view points: (a and b) vehicles captured from the left and right view points, respectively, (c and d) vehicles withacclivitous view point.

Fig. 11. Vehicle logos in different illuminations: (a) vehicle in sunshine, (b) vehicle on cloudy day, (c) vehicle at twilight, and (d) vehicle at nighttime.

Table 1Results of vehicle logo detection.

Vehicle showing up in different situations Image numbers Detected numbers Detection rate (%)

In small scale 20 20 100In medium scale 50 47 94In large scale 60 58 96.7In huge scale 20 20 100From right view point 180 175 97.2From left view point 180 176 97.8From left tilt view point 120 115 95.8From right tilt view point 120 118 98.3In sunshine 100 86 86On cloudy day 40 37 92.5At twilight 40 25 62.5At nighttime 40 22 55Total 970 899 92.6

870 S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872

view points, the results show that our detection rate is also high. As for the third test group, since our image binarizationprocess is based on the self-adapting threshold setting, it is robust for adaptability to changing illumination conditions. Someof the correct vehicle logo detection results are drawn in Fig. 12. In general, the average image processing time and the detec-tion rates are 20 ms and 92.6% respectively.

Fig. 12. Some correct results of our method.

Fig. 13. The first, second and third rows represent the cases of false detection in sunshine, at twilight and nighttime, and the correct detection withunwanted background information.

S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872 871

Analyzing some false detection results, we find that some drawbacks exist in our method. In the sunshine environment,because of the light reflection affect, sometimes the vehicle hood and grill are misidentified. Samples are illustrated in Fig. 13(1). If the light is too dim or weak, for example, at twilight and nighttime, our method may fail to locate the vehicle logoregions, as shown in Fig. 13 (2). And also, if the vehicle logo region exists in surroundings with complex texture, after theprocess of mathematical morphology, the detected vehicle logo region may contain some additional unwanted information,illustrated in Fig. 13 (3).

7. Conclusion

This paper proposed a rapid and robust method for vehicle logo detection. We apply direction filters and saliency map tohighlight the vehicle logo region. Then, the information entropy is used to choose the suitable binary image and precisely

872 S. Mao et al. / Computers and Electrical Engineering 39 (2013) 863–872

locate the vehicle logo region. Our method has two obvious advantages. One is that the vehicle logo can be detected rapidlywithout learning. The other one is that this method is adaptable to different situations without adjusting the parameters. Theexperiment results show that our method has excellent performance in both detection rate and computation time. In futurework, we will focus on the accurate recognition of vehicle logos.

Acknowledgments

This work was supported in part by the 973 National Basic Research Program of China (2010CB732501), Fundation ofSichuan Excellent Young Talents (09ZQ026-035) and the Fundamental Research Funds for the Central University.

References

[1] Burkhard T, Minich AJ, Li C. Vehicle logo recognition and classification: feature descriptors vs. shape descriptors. EE368 digital image processing,spring. Stanford University; 2011.

[2] Psyllos AP, Anagnostopoulos CN, Kayafas E. Vehicle logo recognition using a SIFT-based enhanced matching scheme. IEEE Trans Intell Transport Syst2010:322–8.

[3] Pan X, Huang X, Zhang S. Wavelet moment and improved Adaboost application to vehicle-logo location. In: IEEE international conference proceedingson systems, man, and cybernetics, San Antonio, TX; 11–14 October, 2009. p. 5174–7.

[4] Liu Y, Li S. A vehicle-logo location approach based on edge detection and projection. IEEE Int Conf Veh Electron Safety 2011:165–8.[5] Li W, Li L. A novel approach for vehicle-logo location based on edge detection and morphological filter. In: Second international symposium on

electronic commerce and security; May 2009. p. 343–5.[6] Wang Y, Liu Z, Xiao F. A fast coarse-to-fine vehicle logo detection and recognition method. IEEE Int Conf Robot Biomimetics 2007:691–6.[7] Lu W, Zhang H, Lan K, Guo J. Detection of vehicle manufacture logos using contextual information. In: ACCV 2009; 2010. p. 546–55.[8] Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 1998;20(11):1254–9.[9] Deb K, Vavilin A, Kim J, Jo K. Vehicle license plate tilt correction based on the straight line fitting method and minimizing variance of coordinates of

projection points. Int J Control Automat Syst 2010;8(5):975–84.[10] Kim KI, Jung K, Kim JH. Color texture-based object detection: an application to license plate localization. Pattern Recognit Support Vector Mach

2002:321–35.[11] Chang S, Chen L, Chung Y, Chen S. Automatic license plate recognition. IEEE Trans Intell Transport Syst 2004;5(1):42–53.[12] Dlagnekov L. License plate detection using AdaBoost. San Diego, CA: Comput. Sci. Eng. Dept., Univ. California, San Diego; March 2004.[13] Barlow H. Possible principles underlying the transformation of sensory messages. Sens Commun 1961:217–34.[14] Jones J, Palmer L. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J Neurophys 1987:1233–58.[15] N.Kanopoulos, Vasanthavada N, Baker RL. Design of an image edge detection filter using the Sobel operator. IEEE J Solid-State Circ 1988;23:358–67.[16] Greenspan H, Belongie S, Goodman R, Perona P, Rakshit S, Anderson CH. Overcomplete steerable pyramid filters and rotation invariance. Proc IEEE

Comput Vision Pattern Recogn 1994:222–8.[17] Hartigan JA, Wong MA. Algorithm as 136: a K-means clustering algorithm. J Roy Statist Soc Ser C 1979;28:100–8.

Songan Mao received an M.S. degree from the University of Electronic Science and Technology of China, Chengdu, PR China. His research interests includemachine learning and computer vision.

Mao Ye received a Ph.D. degree in mathematics from Chinese University of Hong Kong, in 2002. He is currently a professor and Director of CVLab at theUniversity of Electronic Science and Technology of China. His current research interests include machine learning and computer vision. In these areas, hehas published over 70 papers in leading international journals and conference proceedings.

Xue Li is an associate professor in the School of Information Technology and Electrical Engineering at the University of Queensland. His research interestsinclude data mining, multimedia data security, database systems, and intelligent Web information systems. In these areas, he has published over 80 papersin leading international journals or conference proceedings.

Feng Pang received an M.S. degree from the University of Electronic Science and Technology of China, Chengdu, PR China. His current research interestsinclude object detection and image registration.

Jinglei Zhou received an M.S. degree from Southwest Jiaotong University, Chengdu, PR China in 2011. His current research interests include visual eventanalysis and recognition in computer vision.