shadows attenuation for robust object recognition

Shadows Attenuation for Robust ObjectRecognition

J. Gabriel Avina-Cervantes1, Leonardo Martınez-Jimenez1, Michel Devy2,Andres Hernandez-Gutierrez1, Dora L. Almanza1, and Mario A. Ibarra1

1 University of Guanajuato.Facultad de Ingenierıa Mecanica, Electrica y Electronica,

Avenida Tampico 912. Salamanca, Guanajuato. 36730, Mexico{avina,luzdora,ibarram}@salamanca.ugto.mx

http://www.ugto.mx2 Laboratoire d’Analyse et d’Architecture des Systemes,

LAAS-CNRS, 7, Avenue du Colonel Roche,31077 Toulouse Cedex 4, France

{gavina, michel}@laas.frhttp://www.laas.fr

Abstract. Shadows are useful for synthetic images in order to increaseextrinsically reality in image generation. However, in natural images,object recognition and segmentation are often negatively affected by castshadows. Since shadows are a physical phenomena observed in most na-tural scenes, we propose a fast and reliable procedure to detect andattenuate shadows effects based on color/brightness density. Detectedshadows are attenuated by modifying locally brightness and color thathave the same color/brightness density. Some color artifacts (false colorson shadows) produced by the acquisition devices have been detected anddiscussed, and it has been noticed that they may affect some of theclassical shadow removal methods. Finally, some experimental results ofthe proposed shadow attenuation method in real images are presentedand evaluated.

1 Introduction

Shadows are one of the principal factors affecting machine vision performance inoutdoor scenes. Many applications such as content based image retrieval, basedon the knowledge of color or texture features, are negatively affected by strongcast shadows. For instance, in applications using color segmentation, object andshadows are often associated to the same object, generating serious problems tothe capabilities of the recognition phase [1]. Furthermore, most image recognitionprocedures are based on texture as their principal feature. Texture has beendefined as a local arrangement of irradiance patterns projected over a surfacemosaic of perceptually homogeneous radiances (high concentration of localizedspatial frequencies). The analysis of texture in natural environments turns outto be very difficult, being shadows one of the problems to overcome.

Shadows are produced when there are objects between a light source and asurface, this produces an effect of division on the surface, as if it had two surfaces.

A. Gelbukh and A.F. Kuri Morales (Eds.): MICAI 2007, LNAI 4827, pp. 650–659, 2007.c© Springer-Verlag Berlin Heidelberg 2007

http://www.ugto.mx

http://www.laas.fr

Shadows Attenuation for Robust Object Recognition 651

Due to the reduction of the illumination level on the affected region, regionshaving high contrast are sometimes associated to the object. Consequently, theseillumination differences over the same surface produce serious problems in imageprocessing, especially in natural images; object recognition and segmentation areoften negatively affected by cast shadows [2,3].

Due to the effects of shadows in image processing tasks, there is some re-searches focused on shadow removal methods. The majority of shadow removalmethods have a thresholding step in order to find a shadow map of the edges.Other methods work controlling the properties of gamma, contrast and bright-ness of the image, to reduce the effect of the low illumination [4,5]. Figure 1shows some classical applications: visual robot navigation and image interpre-ting where shadows produce crucial failure in the routines [6]. In order to reduce,the negative effects of low illumination levels, other methods work controllingsome other properties on the image, such as nonlinear gamma, contrast andbrightness adjustments [2]

(a) Scene Interpretation (b) Robot Visual Navigation

Fig. 1. Cast shadows interference in image understanding and object recognition

When an image has regions affected by shadows, we may assume that theshadow affects only the brightness on the projected surface significantly and thecolor properties are only alter lightly [7]. Moreover, some color spaces are moresuitable for shadows removals than others, e.g., color segmentation. There arecolor spaces with uncorrelated components (I1,I2,I3), some other use non-lineartransformations (HSI, CIE-LUV, CIE-LAB) and in our application probably themost useful spaces are those which allow separate luminance from chrominancecomponents [8,9].

The CIELab color space is able to separate the color information from thebrightness. The L channel is the luminance channel, the a channel is a scalegoing from the green color with negative values to the red color with positivevalues, and the b channel is a scale that goes from the blue color with negativevalues to yellow color with positive values [10,11]. CIELab color space has beenadopted for an important part of our experiments.

652 J.G. Avina-Cervantes

This paper is organized as follows, in section 2 the basis of our shadowsattenuator is presented, in section 3 experimental results are shown and discussedand finally, in section 4 general conclusions of this work are presented.

Fig. 2. Inverse luminances for Shadowed Images

2 Shadow Attenuator

Considering the assumption that small changes on color channels a and b areproduced by shadows effects, we can compare two regions, in order to look forregions with similar color 1 density [12,5]. If we have regions s1 and s2, wecan say that s1 is a shadow region similar in color to s2 if both of them havesimilar color densities and s1 has smaller brightness level than s2. Consequently,we must compare the region s1 with a limited number of different s2 randomregions around s1, the comparison process stops for region s1 when a similarcolor density is found in region s2, and when the luminance s1 is less than theluminance of αs2. Where α is a threshold number in the range [0, 1] that let usdiscriminate shadow differences based on luminance and chrominance statisticson the image. In other words, a low intensity region suspected to be a shadowis compared chromatically against a set of regions located in its neighborhoodgiving a certain distance and a seek criterion.

The saturation of the image in the CIELab color space is given by the followingequation,

c =√

a2 + b2 (1)

where a and b are the chromatic components in the CIE-LAB color space. Forneighboring regions, s1 and s2 chromatic features c1 and c2 have been computedover all both regions, by using equation 1.


It has been considered that chromatics c1 and c2 represent similar color den-sities, if both c1 and c2 over delimited regions have closed values. In this way, c1and c2 are supposed to have similar numerical values if:

vcmin ≤ c1

c2≤ vcmax (2)

where vcmin and vcmax represent similarity boundaries estimated statisticallyover the image or fixed by the user.

Fig. 3. Corrected Luminances for Shadowed Images

In particular for the brightness channel in the region s1, Ls1 could be con-sidered as belonging to a shadowed region if Ls1 is smaller than the luminancein the region s2 affected by a parameter, αLs2. However, experimental observa-tions are shown that a shadow is often a region with a lower illumination thanthe overall averaged illumination on the scene or around its neighborhood (re-gions delimiting the suspected area). This can be expressed by the next relationbetween a luminances ratio,

Ls1

Ls2

< α (3)


where α is estimated dynamically using the mean of the luminance on the scene,i.e., E{L}. Given two regions s1 and s2 in a image, we can say s1 is a regionaffected by a shadow with similar color density to S2 if the next three conditionsare true:

Ls1 < E{L}

vmin ≤ c1

c2≤ vmax (4)

Ls1

Ls2

< α

where 0 < α < 1.The proposed algorithm takes a region s1 which is consideredas a shadow region, around it, a second region s2 is searched, both regions s1 ands2 must satisfy the three conditions given previously in order to start the shadowattenuation procedure. If the first condition is not satisfied, s1 is not conceivedas a shadow, because this region has a high illumination level (a shadow regionis assumed as a low illumination region) and consequently s1 is discarded as ashadow region.

If the first condition is true, but any other of the two remainder conditionsare not satisfied, the algorithm seeks for another suitable region s2 over randomcoordinates (m, n) around the suspected region. At the moment that a shadowregion in the scene is correctly identified, it is necessary to modify automaticallythe brightness level in areas affected ( e.g., increasing luminance). In order tofind the new brightness value for every shadow, a ratio for each couple Ls1, Ls2may be found.

ratio =Ls1

Ls2(5)

Adjusting Luminance is achieved by changing Ls2 with a new value L1new, aformula for L1new can be expressed as follows,

L1new =Ls1

ratio(6)

In our experiments, we compute the average of all ratios obtained in the image.These ratios are substituted by the average E{ratio} and multiplied by a factorof adjustment. Equation 5 is modified, and in this way, equation 7 is generatedfor a set of regions composing the neighborhood of the detected region.

L1new =1

β E{ratio} Ls1 (7)

If suspected region s1 is not surely identified as a cast shadow, its luminancechannel Ls1 should not be changed or affected. Thus, shadows coverage (darksurface) is considered uniformly distributed or illuminated is a strong assumptionwhich is valid in many circumstances but a non uniform shadows removal maybe necessary in some critical applications. However, for robotics applications


and some other machine recognition tasks where real-time routines are needed,the proposed approach is reliable and useful [13]. In the next section, someexperimental results will be presented and commented.

3 Experimental Results

In order to evaluate the capabilities of our algorithm, experimental results havebeen classified in three kind of tests. In the first category, the capacity to detectshadows in the images is correctly identified, in the second test the luminancein the output image over detected shadows is balanced and finally the capacityto attenuate shadows in RGB color image is shown.

Fig. 4. Corrected Images by proposed method

3.1 Shadow Detection Phase

With a light modification in the algorithm, the pixels classified as shadow canbe identified, painting detected pixels as shadows with maximum value (white)and zero for the rest elements on the image (black), see Fig. 2.

The results show that the proposed method is efficient to detect shadows inthe majority of the images; it detected 28 of a total of 31 test images. Thisresearch shows that it is necessary only a maximum of 15 comparisons to findthe appropriate region s2 needed to balance shadow brightness or considerate s1as a non shadow region.

In the images where the method failed, this situation was principally due to areduced value in the contrast between shadows and the scene, i.e., shadow is notso strong and the procedure could not discriminate it from the rest of the surfaceor when the shadow area covers the majority of the image surface (more than the


half of the total area). This non detection is easily explained due to the natureof the proposed algorithm, since non shaded area is above the averaged on theillumination. Another particular case occurs when the algorithm does not find asimilar color density area with respect to the suspected region and consequently,the original illumination can not be adjusted and balanced.

3.2 Adjusting Luminance Image

Experimental results concerning the modification of the luminance image showthat, with the correct value of the parameter β, the effect of the shadow in theluminance image is efficiently reduced or attenuated. In figure 4 (a), (b) and(c) images are the original images, next images (d), (e) and (f) are luminanceL original images and (g), (h) and (i) are the luminance adjusted or outputimages. Some values for the parameter β are estimated for this sample, image(g) was obtained using β = 1, image (h) with β = 0.93 and image (i) withβ = 0.80.

In images where the shadows are very strong, it is only possible to reducethe shadow effects with very small values of β (0.5 approximatively). In someimages, shadow has not the same value along certain perspectives of the image,in this case the method does not work appropriately, because the algorithm findsthe average value product in the reduction of illumination due to the effect ofshadowing to correct affected areas, i.e., a correction is uniformly applied undera region non uniformly shadowed.

A negative effect produced by modifying the luminance image may consist inthe presence of white-borders, which are caused by the appearance of differentlevels of illumination due to changes in the angle of light source (penumbra),in images where the shadow is distant or small, penumbra does not affect im-portantly shadows illumination. However, in images where the shadow is closeor big, the penumbra area is important, and the differences within the shadoware evident. This penumbra phenomena may be reduced by a smoothing filterfocused on the homogenization of dark zones.

Once the effect of the shadows in the luminance image are reduced, the colorimage can be reconstructed, with the luminance image adjusted and balanced,nevertheless, before the re-compose the three color channels, it is needed to fixthe luminance image in order to have values in the range [0, 0.95] to reduceover-brightness values in the image, this reduction helps to avoid overflow onsome values in the output image and increase the quality of the resulting image.Finally, it is necessary to use an average filter in order to avoid the possible falsecolor that could be generated.

Experimental results have been classified for all process inside of four cate-gories:

1. Shadow was detected and strongly attenuated.2. Shadow was detected and attenuated.3. Shadow was detected but not attenuated significantly.4. Shadow was not detected.


(a) Original Image (b) Shadow attenuated images

Fig. 5. Shadows influenced by neighbor objects producing false color

The results of the 31 completely different images that were processed givethe performance table 1, where it is possible to notice that the majority of theimages were at least detected satisfactory and attenuated. And, almost a thirdpart of them were strongly attenuated. From the total of 31 images, only 4show problems produced by acquisition devices, where false colors are producedmainly by the white point color balancing algorithms built in the cameras andwhich very often produce false color on the surfaces affected by shadows (seeFig. 5). In some cases the shadow is so strong, so it could destroy the textureand color, thus it is impossible to restore the color information under the shade.Another problem consists in the influence of external elements located aroundthe object which produces non-uniform shadows (including also the same object),because of these objects contribute with the spectral distribution of the lightingnear of the object. Therefore, complex shadow has color components conformedby several light reflections coming from many sources. Moreover, performanceof our algorithm was qualitatively compared with respect to the most popularshadow removal method, FHD [10]. Principal difference consists in preservingtexture quality with the proposed approach while FHD method corrects globallyshadows with no complex artifacts but affecting negatively textured regions. Thistexture degeneration is produced by the morphologic algorithm to enhance theshadow edges, see Fig. 6.

Table 1. Performance of the algorithm

Category Number of imagesShadows detected and strongly attenuated 10Shadows detected and attenuated 11Shadows detected 6Shadows no detected 4


Fig. 6. Qualitative comparison with the FHD algorithm and proposed algorithm. Ver-tically, (a) original images, (b) FHD method, and (c) proposed approach.

4 Conclusions

In this paper, a shadow detector and attenuator algorithm has been proposed.Experimental results show that it is possible to identify and efficiently cor-rect in an efficient way shadows in the 80% of our test images. Our procedurefails primordially when color covered by the shadow regions is not similar tothe surfaces around the shadows. It is important to express the importanceof the sensor (cameras) properties that can modify the chromatic propertiesover the shadows as well as the reflectivities produced by the interaction of thelight and the objects around of the shadows and penumbra regions. Therefore,the proposed approach is a reliable and fast procedure which can be used asa pre-processing stage on pattern recognition procedures affected by shadows,which occur mainly in outdoor applications as robot visual navigation in naturalenvironments.

Acknowledgement. We would like to thank to CONCYTEG and PROMEPfor the funding obtained through the projects: ”Application of the chromaticadaptation models into the texture analysis of natural scenes” (code 07-16-K662-061/03, CI3O38307) and ”Visual navigation of robots in agricultural environ-ments based on autonomous road detection and tracking” (PTC 103.5/05/1924).


References

1. Rosin, P.L., Ellis, T.: Image difference threshold strategies and shadow detection.In: Sixth British Machine Vision Conference, BMVC, Birmingham, Uk, pp. 347–356 (1995)

2. Hsieh, J.-W., Hu, W.-F., Chang, C.-J., Chen, Y.-S.: Shadow elimination for ef-fective moving object detection by gaussian shadow modeling. Image and VisionComputing 21(6), 505–516 (2003)

3. Liu, Z., Huang, K., Tan, T., Wang, L.: Cast Shadow Removal with GMM for Sur-face Reflectance Component. In: Proceedings of the 18th International Conferenceon Pattern Recognition, pp. 727–730 (2006)

4. Stauder, J., Mech, R., Ostermann, J.: Detection of moving cast shadows for objectsegmentation. IEEE Transactions on Multimedia 1(1), 65–76 (1999)

5. Baba, M., Mukunoki, M., Asada, N.: Shadow Removal from a Real Image Basedon Shadow Density. In: 31st International Conference on Computer Graphics andInteractive Techniques (2004)

6. Gu, X., Yu, D., Zhang, L.: Image Shadow Removal Using Pulse Coupled NeuralNetwork. IEEE Transaction on Neural Networks 16(3), 692–698 (2005)

7. Tadjine, H.H., Joubert, G., Mouattamid, S.M.: Colour object detection withshadow using virtual electric field. In: International conference on image and signalprocessing, ICISP, Agadir, Morocco, vol. 1, pp. 53–60 (2003)

8. Finlayson, G., Hordley, S., Drew, M.: Removing Shadows from Images UsingRetinex. In: 10th Color Imaging Conference (2002)

9. Buluswar, S.D., Draper, B.A.: Color Recognition in Outdoor Images. In: Proceed-ings of the Sixth International Conference on Computer Vision, pp. 171–177 (1998)

10. Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing Shadows from Images. In:Tistarelli, M., Bigun, J., Jain, A.K. (eds.) ECCV 2002. LNCS, vol. 2359, pp. 823–836. Springer, Heidelberg (2002)

11. Jiang, C., Ward, M.O.: Shadow Segmentation and Classification in a ConstrainedEnvironment. CVGIP: Image Understanding (1994)

12. Salvador, E., Cavallaro, A., Ebrahimi, T.: Shadow Identification and Classificationusing Invariant Color Models. International Conference on Acoustics, Speech, andSignal Processing 3, 1545–1548 (2001)

13. Mateus, D., Avina-Cervantes, J.G., Devy, M.: Robot visual navigation in semi-structured outdoor environments. In: IEEE International Conference on Roboticsand Automation (ICRA), Barcelona, Spain (2005)

shadows attenuation for robust object recognition

Documents