Rear Lights Vehicle Detection for Collision Avoidance

Evangelos Skodras1, George Siogkas

2, Evangelos Dermatas

2 and Nikolaos Fakotakis

1

1 Artificial Intelligence Group,

2 Pattern Recognition Group

Wire Communications Laboratory, Department of Electrical and Computer Engineering

University of Patras, Patras, Greece {evskodras, fakotaki}@upatras.gr, {siogkas, dermatas}@wcl2.ee.upatras.gr

Abstract— Vehicle detection based on on-board mounted

cameras is an integral component of many driver assistance

systems aiming at alerting the driver about impending

collisions. In this paper an automated algorithm for detection

of preceding vehicles is proposed, based on the detection of

rear vehicle lights. Unlike many systems which make use of

static threshold boundaries for the red color segmentation of

rear lights, our method combines color and radial symmetry

cues while the threshold is dynamically adapted. The extracted

candidate rear lights are morphologically paired in order to

define possible areas where vehicles are present. The

verification of vehicle presence is then carried out through

axial symmetry check. Experimental results that demonstrate

the system’s high detection rates and robustness even in

adverse illumination and weather conditions are finally

presented.

Index Terms— Vehicle detection, rear lights detection, driver

assistance, collision avoidance, computer vision.

I. INTRODUCTION

Over the last decades, many efforts are directed towards

enhancing driving safety, following the dire statistics of

vehicle crashes in terms of expenses and human casualties.

Although vehicle safety improvement has significantly

reduced the death toll in vehicle crashes, accident prediction

and prevention would be the ultimate solution for

maximizing driving safety [1]. On these grounds, pre-crash

sensing has become an area of active research among

automotive manufacturers and universities. The

development of a vehicle-mounted driver assistance system

aiming at alerting the driver about an impending collision

requires reliable detection of preceding vehicles. Up to now,

existing state-of-the-art systems rely on active sensors for

this challenging task [2]. However, the exponential growth

in processing power and the availability of advanced

computer vision techniques render a lot of low-cost methods

for vehicle detection feasible.

Most techniques employed for vehicle detection are using

visual features, motion or appearance. An extensive review

of the most commonly used techniques can be found in [3].

Rear lights have been widely used as a cue for vehicle

detection, especially at night conditions, where other

features are impossible to detect. The most common

approaches use the RGB color space or its components in

order to identify rear vehicle lights. In [4] rear lights are

recognized based on grayscale information and a red

component resulting from the normalized difference of the

R and B channels. In [5] a “red level” of every pixel is

computed in small regions, based on a proportion between

RGB channels, whereas in [6] only the red channel is

utilized. Brake lights are detected in [7] using thresholds in

all RGB channels. However, the RGB color space is not

ideal for color thresholding, as channels are highly

correlated with each other, making it difficult to define and

manipulate color parameters. To overcome this difficulty,

other color spaces have been chosen for this task. Rear

lights are segmented using the YCbCr color space in [8],

using a subjectively chosen threshold in the Cr channel.

O‟Malley et al. in [9] defined thresholds in HSV color space

components derived from the color distribution of rear-lamp

pixels under real-world conditions. Finally, in [10] the

L*a*b* color space is used to detect rear lights, using two

specific thresholds for a* and b* channels.

In this paper we present a vehicle detection algorithm

which can be integrated in driver assistance systems for

forward collision avoidance. It is based upon a combination

of multiple cues present on vehicles, such as the red color of

rear lights, horizontal edges and symmetry. Many existing

systems utilize static thresholds for red color segmentation

of vehicle lights, thus being prone to illumination changes

and environmental conditions. The main feature of our

method lies in the absence of such static thresholds for the

detection of rear lights, making our system resilient to any

changes in illumination conditions, time of day, camera

settings and sensor characteristics. Moreover, the use of

multiple cues is a viable means for improving the reliability

of our system.

The paper is organized as follows. In Section 2 the

proposed algorithm is presented and Section 3 discusses its

performance. Finally, in Section 4, conclusions and future

work are outlined.

II. PROPOSED VEHICLE DETECTION SYSTEM

The proposed vehicle detection system is developed upon

knowledge-based methods and uses some of the vehicle‟s

most prominent features. At first, we extract a binary image

containing candidate rear vehicle lights using color

segmentation and radial symmetry.

Rear lights represent a conspicuous cue for vehicle

detection. Apart from being a common feature among all

vehicles according to legislation, they are also visible under

different illumination, weather conditions and time of day.

Moreover, they can be used to give an advanced warning of

a potential danger, as illuminated rear brake lights indicate

that a vehicle is beginning to slow down.

The second stage of our system involves morphological

pairing of candidate lights and horizontal edge detection, in

order to define possible vehicle areas. Subsequently, a

symmetry check along the vertical bisector is utilized for

vehicle presence verification. For the successfully detected

vehicles, an estimation of their distance is performed. The

architecture of the proposed system is presented in Figure 1.

A. Red Light Detection

In the first processing stage of our system, we seek the

areas with high red chromaticity in the color image. Among

various color spaces, L*a*b* was the most suitable for our

application, as it possesses a number of important features.

L*a*b* color space has the advantage of being a

perceptually uniform color space, mapping equally the

perceived color difference into qualitative distance in the

color space. In L*a*b*, luminance information (L

component) is separated from the chrominance information

(a*, b* components), which we utilize for the red color

segmentation. As a result, illumination changes have a

minimal effect on color information.

For the detection of candidate rear light areas, we utilize the

a* component (red - green) of L*a*b*, and split the positive

from the negative part, in order to acquire only the red

subspace. This subspace image is then scanned for

symmetrical shapes, using the fast radial symmetry

transform presented in [11]. Although there is no constraint

in the shape of rear lights, they generally follow a

symmetrical pattern. A judicious choice of a low radial-

strictness parameter (a=1) gives emphasis to non-radially

symmetric features [11], thus presenting great values at the

positions of rear lights (Figure 2).

The symmetry detection scans for shapes in one or more

ranges N. Normally, in order to blindly detect shapes of any

size, a large size of ranges must be used; however, the

computations are greatly accelerated by choosing a small

sparse set of ranges N, spanning between the extreme sizes

of possible rear lights. The result constitutes a very good

approximation to the output obtained if all the possible

ranges were examined. Using the fast radial transform

approach, the “blooming effect”, caused by the saturation of

bright pixels in CCD cameras with low dynamic range, is

very effectively handled. This is attributed to the fact that

saturated lights appear as bright spots with a red halo

around, thus yielding large radial symmetry values. This

phenomenon is illustrated in Figure 3.

For the final binarization of the image, we utilize the fast

and efficient Otsu‟s thresholding algorithm, which suggests

minimizing the weighted sum of variances of the objects‟

and background pixels to set an optimum threshold,

especially in the case of bimodal images. The resulting

binary image contains the candidate rear vehicle lights.

B. Morphological Lights Pairing

In this stage, a morphological rear lights pairing scheme

is applied to the binary image to determine vehicle

candidates. After connected component labeling, we

compute for each region an ellipse with similar second

moments as the region, in order to calculate its features. The

parameters of the ellipse, i.e., the center coordinates, the

major and minor axis lengths as well as the area are

computed. In order to find pairs of possible lights we

consider all the possible 𝑁!

2!∙ 𝑁−2 ! two-combinations.

However, from all these potential pairs only a few meet the

prerequisites that we impose, regarding the angle between

them and a similarity measure based on their geometrical

properties: Assuming that the target vehicle is in the same

tilt as the observing vehicle, the candidate pair of lights

must be aligned in the horizontal axis (with a permissible

Fig. 1. Overview of the proposed vehicle detection system.

Fig. 3. (a) Original image, (b) pseudo-colored red subspace of the L*a*b* color space where the “blooming effect” is visible and (c)

pseudo-colored fast radial symmetry transform of the red subspace.

Fig. 2. (a) Original image, (b) pseudo-colored red subspace of L*a*b* and

(c) pseudo-colored fast radial symmetry transform of the red subspace.

(c) (b) (a)

(a) (b) (c)

inclination of ±5 degrees). The morphological similarity

measure is based on the normalized difference of their major

axis length, minor axis length and area.

Confining the maximum and minimum allowable

distance of the candidate lights can further narrow down the

number of possible pairs. Even though this can be proven

very useful for speeding up the calculations, we omit it for

purposes of generality.

C. Horizontal Edge Boundaries

Given the candidate rear light pairs, we seek the

horizontal boundaries of the candidate vehicle (the vertical

boundaries are defined by the extreme points of the rear

lights). First, we determine a search region for the upper and

lower horizontal boundaries that is proportional to the width

of the vehicle, which is assigned as the distance between the

extreme points of the rear lights. Figure 4a illustrates a

search region on the original image. The „Canny‟ edge

detector is used to detect the edges in the grayscale image of

the search region (Fig. 4b). The horizontal projection of the

edge map is then computed (Fig. 4c), while the peak values

indicate pronounced horizontal edges. The upper and lower

boundaries of the car are defined as the first and last peak in

the projection graph, with value at least equal to the half of

the largest value. The outcome of this stage is bounding

boxes containing candidate vehicles.

D. Symmetry Check

As one of the main signatures of man-made objects,

symmetry represents a very interesting cue for vehicle

verification. Images of vehicles observed from the rear view

are in general symmetrical in the vertical direction [3]. The

symmetry check is performed by splitting each bounding

box image into two sub-images along the vertical bisector

and comparing them. The comparison of the sub-images is

carried out by utilizing two measures, namely the Mean

Absolute Error (MAE) and the Structural SIMilarity (SSIM)

measure [12]. MAE constitutes a straightforward and

efficient measure, formulated as follows:

𝑀𝐴𝐸 =1

𝑀∙𝑁 𝑥𝑖𝑗 − 𝑦𝑖𝑗

𝑁𝑗=1

𝑀𝑖=1

where M, N are the dimensions of the sub-images x and y

that we compare. The SSIM measure, originally used as an

image quality measure, can be effectively applied in our

system. It searches for similarity using three comparisons,

regarding luminance, contrast and structure. More details

about SSIM measure can be found in [12].

For the verification of vehicle presence, the result from

both measures must lie below thresholds, defined

heuristically, through extensive experimental tests.

E. Distance Estimation

Once the preceding vehicle is successfully detected, the

relative distance is calculated. A precise calculation of the

distance is not feasible, as a single frame cannot contain

enough information. However, a sufficient approximation

for typical sized cars can be achieved: Assuming an average

vehicle width of ~1.7m and given the width of the target

vehicle in the image (as a proportion of the vehicle‟s width

in pixels to the image‟s width in pixels) we are able to

estimate the desired distance. If the camera characteristics

are well known in advance, a more precise estimation can be

computed as in [6].

III. EXPERIMENTAL RESULTS

The performance of the proposed algorithm was tested in

two publicly available databases containing images of cars

from the rear [13] and a publicly available video sequence

of driving in an urban environment [14]. These test sets

contain images of many different cars, under various

illumination conditions, shot with different cameras. We

should clarify that in Caltech DB (Cars 2001) from the 526

images, 22 images were excluded as their red rear lights

were modified beyond legislation [9], or one of the brake

lights was blown. For the video sequence of [14] only

frames that contain a whole visible, preceding vehicle at the

same lane and in distance less than 15m were considered

(2716 out of 11179 frames). Red vehicles, recognized as

large regions in the binary image, were also detected using

the same method. The recognition results are summarized in

Table 1.

Our system scores high detection rates in all test sets (up

to 93.6%), and performs outstandingly in cases when the

preceding vehicle is braking, as can be observed from Table

1. This can be attributed to the intensive, highly

distinguishable color of illuminated brake lights and the

ability of our system to handle the “blooming effect” very

effectively. This specific feature is of key importance, as

accurate recognition at the stage when the preceding vehicle

Fig. 4. (a) Search region on the original image, (b) edge map of the

search region, (c) its horizontal projection and (d) bounding box

containing the candidate vehicle

(1)

TABLE I

DETECTION RATES

Database

Number of

images or

frames

Detection Rate

Detection

Rate when

Braking

Caltech DB

(Cars 1999) 126 92.1% -

Caltech DB

(Cars 2001) 504 93.6% 99.2%

Lara Urban

Sequence 1 2716 92.6% 96.3%

(a) (b) (c) (d)

is braking is very crucial for avoiding an impending

collision. A fruitful comparison can be made with the

system of [15], reporting results at the same databases

(Caltech DBs). Our approach performs better (93.3% versus

92%) on both databases, with the additional advantage of

requiring no training. Some representative detection results

from all data sets used are illustrated in Figure 5. Our

system was also tested on images acquired under adverse

weather conditions, downloaded from the internet. For these

images, although we cannot obtain quantitative results, we

can observe that our system performs sufficiently well,

yielding promising results (Figure 6).

Regarding the cases where vehicles are falsely recognized

(for example, there exist 7 and 46 false positives (FP) for the

Caltech 1999 and 2001, databases respectively), these can

be eliminated in various manners. The FP rate can be

significantly reduced if we impose certain restrictions in the

admissible distance between rear lights, by narrowing down

the region of interest or, most importantly, by taking

advantage of the temporal continuity of the data, as FPs are

not persistent in time (most FPs rarely appear in more than

one frame).

Examining the failed cases of our algorithm, the most

common cause is the presence of other red artifacts near rear

lights, recognized as a unity.

IV. CONCLUSIONS

The development of a robust and reliable vision-based

vehicle detection method is a crucial task for driver

assistance systems. In this paper we have presented an

automatic, resilient to illumination conditions algorithm for

vehicle detection. It makes use of color and radial symmetry

information for the segmentation of rear vehicle lights. After

morphological lights pairing and edge boundaries detection,

symmetry check is performed in the candidate bounding

boxes, in order to verify vehicle presence. Experimental

results report high detection rates even in challenging cases.

The proposed algorithm can be easily extended for vehicle

detection at night, because of its approach of using rear

lights for detection. Future efforts are directed towards

vehicle tracking and combining vehicle detection (and

braking recognition) with driver‟s gaze detection. In this

way, the level of attention of the driver can be correlated

with the potential danger of an impending collision.

REFERENCES

[1] W. Jones, “Keeping Cars from Crashing”, IEEE Spectrum, vol. 38, no

9, pp. 40-45, Sep 2001.

[2] R. Stevenson, “A Driver‟s Sixth Sense”, IEEE Spectrum, vol. 48, no

10, pp. 50-55, Oct 2011.

[3] Z. Sun, G. Bebis, and R. Miller, “On-road Vehicle Detection: A

Review”, IEEE Transactions on Pattern Analysis and Machine

Intelligence, Vol. 28, No. 5, pp. 694–711, May 2006.

[4] Y.-C. Lin, C.-C. Lin, L.-T. Chen and C.-K. Chen, “Adaptive IPM-

Based Lane Filtering for Night Forward Vehicle Detection”, IEEE Conference on Industrial Electronics and Applications (ICIEA),

Beijing, China, pp.1568-1573, June 2011.

[5] L. Gao, C. Li, T. Fang, and Z. Xiong, “Vehicle Detection Based on

Color and Edge Information”, Image Analysis and Recognition, vol.

5112, Lecture Notes in Computer Science , pp. 142–150, 2008.

[6] M. Betke, E. Haritaglu, and L. Davis, “Real-Time Multiple Vehicle

Detection and Tracking from a Moving Vehicle”, Machine Vision and Applications, vol. 12, no. 2, Feb 2000.

[7] P. Thammakaroon and P. Tangamchit, “Predictive Brake Warning at

Night using Taillight Characteristic”, IEEE International Symposium on Industrial Electronics (ISIE), Seoul, Korea, pp. 217–221, Jul 2009.

[8] S. Nagumo, H. Hasegawa and N. Okamoto, “Extraction of Forward Vehicles by Front-mounted Camera Using Brightness Information”,

IEEE Canadian Conference on Electrical and Computer Engineering,

vol. 2, pp. 1243–1246, May 2003.

[9] R. O‟Malley, E. Jones, and M. Glavin, “Rear-Lamp Vehicle

Detection and Tracking in Low-Exposure Color Video for Night Conditions”, IEEE Transactions on Intelligent Transportation

Systems, pp. 453–456, June 2010.

[10] I. Cabani, G. Toulminet, and A. Bensrhair, “Color-based Detection of

Vehicle Lights”, IEEE Intelligent Vehicles Symosium, pp. 278–283,

June 2005.

[11] G. Loy and A. Zelinsky, “Fast Radial Symmetry for Detecting Points

of Interest”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 8, pp. 959-973, Aug 2003.

[12] Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity”,

IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612,

April 2004.

[13] Caltech Image Database, http://www.vision.caltech.edu/html-

files/archive.html [14] Lara Urban Sequence 1, Robotics Centre of Mines ParisTech, 2010,

http://www.lara.prd.fr/benchmarks/trafficlightsrecognition [15] C.-C. Wang and J.-J. Lien, “Automatic Vehicle Detection Using

Local Features–A Statistical Approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 9, no. 1, pp. 83-96, Mar 2008.

Fig. 6. Detected vehicles in adverse weather conditions

Fig. 5. Detection results for the data sets used, (a) Caltech DB (cars

1999), (b) Caltech DB (cars 2001) and (c) Lara Urban Sequence

(a) (b) (c)