ROBUST ZEBRA-CROSSING DETECTION USING BIPOLARITY AND PROJECTIVEINVARIANT

Mohammad Shorif Uddin and Tadayoshi Shioyama

Department of Mechanical and System EngineeringKyoto Institute of Technology, Sakyo-ku, Kyoto 606-8585, Japan

E-mail: shorif@yahoo.com

ABSTRACT

Crossing roads is a great challenge for visually impairedpeople. A robust technique for detection of zebra cross-ings is described. This is achieved through bipolarity-based segmentation and projective invariant-based recog-nition. The technique shows good performances undervarious illuminations such as sunny, cloudy and rainy sit-uations in the day time and also evening and night envi-ronments.

1. INTRODUCTION

Independent walk is the main barrier for the visually im-paired people. Usually, they use a white cane. The rangeof detection of special patterns or obstacles using a cane isvery narrow. To improve the versatility of the white cane,various electronic devices [1] have been developed. But,these devices are unable to assist them in safely crossinga road. Some traffic lights have beepers, which promptthe visually impaired to cross the road, when it is safe todo so. However, such equipment is not available at everycrossing; perhaps, it would take too long for such equip-ment to be installed and maintained at every crossing.

Blind people obviously can not see, but can hear. Theprincipal objective of this research is to develop a com-puter vision-based wearable device that uses vocal com-mands to allow blind people to cross roads with total safety.For a safe road crossing, at first a blind person needs theinformation about the location of a crossing i.e. whetherhis frontal area is a crossing or not, then an idea aboutits length and the state of traffic lights. We have alreadydeveloped a crossing length measurement technique [2]assuming that the image contains a crossing. Therefore,crossing location detection becomes a prerequisite. Re-cently, we reported a computer vision-based zebra cross-ing detection system [3]. lighting environment affects theperformance of any computer vision application. How-ever, in that paper we did not investigate the lighting ef-fects. This paper tries to fill this gap.

From the view point of a computer vision-based ze-bra crossing detection, Stephen Se [4] proposed a tech-nique by grouping lines and checking for concurrency us-

Thanks to the Japan Society for the Promotion of Science for thesupport under Grants-in-Aid for Scientific Research (No. 16500110 andNo. 03232).

ing the vanishing point constraint. But, a thorough evalu-ation of this technique is not yet performed and it worksslow and is far from real time. On the other hand, thepresent method works very fast.

To confirm the robustness of the proposed method, ex-periment is performed using real road scenes with andwithout crossings under various illumination conditionssuch as sunny, cloudy and rainy situations in the day timeand also evening and night environments.

2. DETECTION PRINCIPLE

A zebra crossing is characterized by evenly spaced whitestripes on a usual black road surface. In Japan, the widthof each white or black stripe is 45 cm. Some experimentalimages of real road scenes are shown in Figs. 1 and 2 . Thesize of images is (width × height) = (640 × 480) pixels.The crossing pattern can be treated as a bipolar region.

The proposed technique includes four major steps. Tocope with the various lighting situations, first, we use his-togram equalization if the average intensity of the imageis less than a threshold. Second, we extract the candidatecrossing area based on bipolarity feature. Then checkingthe candidate area for the appropriateness of its positionin the image as well as crossing direction, as the tech-nique is searching for a frontal crossing. Third, we extractthe feature points on the central vertical line of the bina-rized candidate area. Fourth, taking the final decision ofa crossing or not a crossing using projective invariant. Asbipolarity and projective invariant features are used as themain tools in the proposed algorithm, we describe brieflythese two in the following subsections.

2.1. Bipolarity

We denote the intensity distribution of an image block asp0(x). If the block contains only black and white pix-els, then p0(x) can be written as p0(x) = α p1(x) + (1−α)p2(x), where 0≤ α ≤ 1, p1(x) is the intensity distribu-tion of black pixels and p2(x) is the intensity distributionof white pixels. Let define some variables as:

µi =∫ ∞

−∞xpi(x)dx, σ2

i =∫ ∞

−∞(x−µi)

2 pi(x)dx, (i = 0,1,2),

(1)

0-7803-9243-4/05/$20.00 ©2005 IEEE 571

where µi and σi2 represents mean and variance, respec-

tively. Using the above relations, we can write σ02 as

σ20 = ασ2

1 +(1−α)σ22 +α(1−α)(µ1−µ2)

2. (2)

Equation (2) shows that the total variance consists of theweighed sum of variances and the difference of means. Ifσ2

0 ≈ α(1−α)(µ1− µ2)2, then p0(x) can be said to be

almost bipolar. So, we define the bipolarity γ as

γ ≡ 1σ2

0

{

α(1−α)(µ1−µ2)2} . (3)

Equation (3) implies that 0 ≤ γ ≤ 1. If γ = 1, there areα , p1 and p2 such that σ1 = σ2 = 0. This means p1(x) =δ (x− µ1) and p2(x) = δ (x− µ2). So, γ = 1 correspondsto perfect bipolarity and γ = 0 represents the absence ofbipolarity.

2.2. Projective Invariant

Under projective transformation [5], [6], the cross ratio offour collinear points’ Euclidean distances

I ≡ l12l34

l13l24, (4)

is invariant, where li j is the Euclidean distance betweentwo points i and j.

Earlier we mentioned that a pedestrian crossing is char-acterized by equal width periodic white and black stripes.Let denote the width of each crossing band is b. Con-sider feature points, which are edge points of white bands,then using (4) we can write the projective invariant forfour consecutive feature points of a pedestrian crossing as

I ≡ l12l34

l13l24=

b ·b2b ·2b

= 0.25. (5)

If there are n feature points, we can find (n− 3) projec-tive invariants using (4). For each projective invariantI(k), k = 1,2, · · · ,n−3, we check whether I(k) is within10% of 0.25. That means,

∣

∣

∣

∣

I(k)0.25

−1.00

∣

∣

∣

∣

< 0.10. (6)

We have used this 10% tolerance to cope with the variousnoises.

3. DETECTION METHOD

At first, the color image is converted to a gray scale im-age. To cope with the various lighting situations, estimatethe average intensity of the image. If the average inten-sity value is less than a threshold (40 is used here on thebasis of experimental data) then use histogram equaliza-tion. The image is partitioned into equal-size rectangularblocks sized (16× 16) pixels to find the crossing regioncandidates.

1. This step identifies homogenous bipolar regions. Seg-mentation is carried out by merging neighboring blocksof similar pattern on the basis of intensity distribu-tions.

2. Only keep regions that are sufficiently bipolar andwide. Calculate the bipolarity of each segmentedregion. First, determine the largest region that hasbipolarity higher than 0.80. Then extract the can-didate regions, which have bipolarity greater than0.80 and areas more than 50% of the largest region’sarea.

Then, we follow the following steps in checkingeach candidate region.

3. This section refines the segmentation. First, checkthe largest area candidate region. If there exists asmall region of different label within this candidateregion, then fill it with same label if its area is lessthan 5% of the candidate area. Then refine the bot-tom boundary region of the crossing area. The re-finement is done by assigning same label to the pix-els on a different label horizontal line if its leftmostand rightmost pixels lie in the candidate region.

4. Make sure the region is centered in the image. Checkthe position (either it is in appropriate position ortoo left or too right or too far w. r. t. the observer)of the candidate region. If the candidate region isnot in an appropriate location then decide that thereis no crossing for this candidate and go to examinethe next candidate region (if any).

5. As a blind person is interested in detecting the frontalcrossing not the sided one, therefore, estimation ofcrossing direction is important. Calculate the cross-ing direction [2] from the power spectrum of thecandidate region with the help of the two-dimensionalfast Fourier transform. For a true frontal crossing,there will be only one prominent peak in the powerspectrum, however, due to a sided crossing or a toosteep crossing or other disturbances there will bemore than one prominent peaks in the power spec-trum. To determine a true frontal crossing, we usea Gaussian threshold at the maximum value of thepower spectrum. If there is any other peak, whichovershoot the Gaussian threshold then decide thatthere is no crossing for this candidate and go to ex-amine the next candidate region (if any). The cut-off and variance of the Gaussian threshold are taken33% of the maximum value of the power spectrumand 10◦, respectively.

6. This section extract feature points and then take thedecision on the basis of projective invariants. Bi-narize the original image content at the location ofthe candidate region using the mean. Use medianfiltering of window size (5× 5) pixels on the bina-rized image to eliminate sporadic noise. Determinethe center position of the candidate region. Then ex-tract the feature points (which are the edge points of

572

the white bands) on the vertical line passing throughthe center point of the candidate region. If thereare at least 7 feature points, then go for checkingthe projective invariant criterion. This condition en-sures that there are more than 3 white bands existin the crossing region. Some road markings (not acrossing) consist of three painted stripes. An im-age of this type is shown in Fig. 1(e). Hence, theabove condition will safeguard against false posi-tive detection result. If there is at least one invari-ant satisfying the invariant condition (6) then decidethat there is a crossing, otherwise go to examine thenext candidate region (if any).

4. EXPERIMENTAL RESULTS

To evaluate the performance of the proposed method weused 118 real images (81 images include crossing andthe rest 37 are of no crossing) with different backgroundsand illumination conditions taken by a commercial digitalcamera. Among them, 49 images are of sunny condition,55 are of cloudy condition, 3 are of rainy condition, 4 areof evening and the rest 7 are of non-rainy night. The im-ages of Fig. 1 are sufficiently illuminated, no equalizationis needed. However, Fig. 2 shows two sample images ofevening and non-rainy night environments, which are notsufficiently illuminated (i.e. low contrast). Therefore, weuse a preprocessing histogram equalization.

Figs. 3(b) and (d) show the power spectrum versus di-rection for the crossing candidates (shown in Figs. 3(a)and (c), respectively) of the images of Figs. 1(a) and (d),respectively. There is only one prominent peak (i.e. nopeak is above the Gaussian threshold) for the image ofFig. 1(a). The angle represented by the peak position in-dicates the crossing direction. But, there are two promi-nent peaks and one peak is above the Gaussian thresholddue to too steep sided crossing candidate of the image ofFig. 1(d). Hence, this image does not contain any frontalcrossing.

Fig. 4 presents the extracted binarized candidate areasand the feature points for the images of Figs. 1(h) and 2(d)in detecting the existence of a crossing.

Table 1. Crossing Detection Result Summary.Image with Image without

Decision crossing crossingcrossing 77 0

(Ok) (False positive)No crossing 4 37

(False negative) (Ok)

Total number 81 37of images

The complete result of the detection of the existenceof crossings is summarized in Table 1. From this table,we see that the proposed algorithm is quite successful androbust to various illuminations in detecting the existenceof crossings from real road images with different back-

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Fig. 1. Some day-time sample images: (a)–(e) are ofsunny condition; (f) and (g) are of cloudy condition; (h)is of rainy condition. All images are sufficiently illumi-nated. Crossing is detected in (a), (f) and (h); false nega-tive is obtained in (b), (c) and (g); no crossing is detectedin (d) and (e).

grounds. The method has not made any dangerous (falsepositive) error such that it decides the existence of a cross-ing for a scene without crossing. For a scene containingcrossing where the white paintings on the crossing aredamaged or the scene contains too few crossing bands -very short crossing (less than 4 white bands) then it de-cides nonexistence of crossing (i.e. false negative). Thishas happened only in 4 cases. Among them, 2 cases con-tain less than four white bands - an image of these type isshown in Fig. 1(b), a case shown in Fig. 1(g) where thewhite paintings on the crossing are damaged, and in therest case (shown in Fig. 1(c)) the segmentation step couldnot extract any highly bipolar wide area in this image, asthe white paintings of crossing are done on unusual pat-tern of road surface. Extremely high accuracy is requiredfor the detection of the existence of a crossing, as a falsepositive error is life threatening for a pedestrian. If there is

573

(a) (b)

(c) (d)

Fig. 2. Sample images of evening and night, which arenot sufficiently illuminated: (a) is of evening and (b) isits histogram equalized image; (c) is of night and (d) is itshistogram equalized image. Crossing is detected using therespective histogram equalized image.

0

1e+010

2e+010

3e+010

4e+010

5e+010

6e+010

7e+010

8e+010

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

direction [radian]

power spectrumGaussian threshold

(a) (b)

0

5e+008

1e+009

1.5e+009

2e+009

2.5e+009

3e+009

3.5e+009

4e+009

4.5e+009

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

direction [radian]

power spectrumGaussian threshold

(c) (d)

Fig. 3. (a) and (b) are the crossing candidate and powerspectrum versus direction, respectively, for the crossingimage of Fig. 1(a); (c) and (d) are the similar pictures forthe crossing image of Fig. 1(d).

a crossing in the vicinity, the algorithm is also able to givea voice message to the user on the basis of position (suchas left, right or more front) and direction of the crossing.

If the crossing is occluded by vehicles or other ob-structions then the method fails to detect the crossing. There-fore, detection of vehicles needs to be included with thepresent system as a preprocess. If there are vehicles onthe crossing, then a voice message will be delivered to theuser to take another image.

5. CONCLUSIONS

In this paper, a computer vision-based method, which isrobust to various illuminations for the detection of exis-

(a) (b)

(c) (d)

Fig. 4. (a) and (b) binarized crossing area and extractedfeature points, respectively, for the image of Fig. 1(h); (c)and (d) are the similar pictures for the image of Fig. 2(d).

tence of zebra crossings has been described. Experimen-tal results with real road scenes under various illuminationconditions confirmed its effectiveness. The method hasnot made any dangerous (false positive) error such thatit decides the existence of a crossing for a scene withoutcrossing. We hope this technique will help in improvingthe mobility of visually impaired people.

6. REFERENCES

[1] D. L. Morrissette et al., “A follow-up study ofthe Mowat sensor’s applications, frequency of use,and maintenance reliability,” J. Vis. Impairment andBlindness 75, pp. 244-247, (1981).

[2] M. S. Uddin and T. Shioyama, “Measurement of thelength of pedestrian crossings - a navigational aid forblind people,” Proc. IEEE Conf. Intelligent Trans-portation Systems (ITSC2004), Washington, USA,Oct. 2004, pp. 690-695.

[3] M. S. Uddin and T. Shioyama, “Detection of pedes-trian crossing using bipolarity and projective invari-ant,” Proc. IAPR Conf. Machine Vision Applications(MVA2005), Tsukuba, Japan, May 2005, pp. 132-135.

[4] Stephen Se, “Zebra-crossing detection for the par-tially sighted,” Proc. IEEE Conf. Comp. Vision andPattern Recognition (CVPR2000), South Carolina,USA, Jun. 2000, vol. 2, pp. 211-217.

[5] K. Borsuk and W. Szmielew, “Foundations of Ge-ometry,” North-Holland, 1960.

[6] I. Weiss, “Geometric invariants and object recogni-tion,” Int. J. Comp. Vision, 10, no. 3, pp. 207-231,1993.

574