Robust Computation of the Polarisation Image

Gule Saman and Edwin R. Hancock ∗

Department of Computer Science, University of York, York, YO10 5DD, UKsaman,erh@cs.york.ac.uk

Abstract

In this paper we show how to render the computationof polarisation information from multiple polariser an-gle images robust. We make two contributions. First, weshow how to use M-estimators to make robust momentsestimates of the mean intensity, polarisation and phase.Second, we show how directional statistics can be usedto smooth the phase-angle, and to improve its estima-tion when the polarisation is small. We apply the result-ing techniques to polariser images and perform surfacequality inspection. Compared to polarisation informa-tion delivered by the three-point method, our estimatesreveal finer surface detail.

1. Introduction

In computer vision polarisation information has beenused to develop a diverse range of algorithms for prob-lems ranging from surface inspection to surface recon-struction. The approach adopted is to use the Fres-nel theory to determine the parallel and perpendicu-lar components of the electric field for incident light,and to model the transmission and reflection of thesecomponents[2]. Generally, speaking the modelling ofdielectrics is more straightforward than the modellingof metals, since in the latter case the incident electro-magnetic field induces surface currents and the Fresneltheory alone is insufficient. For dielectrics, it is con-venient to distinguish between the processes of specu-lar and diffuse polarisation. In the case of specular po-larisation, initially polarised light is reflected from thesurface and the plane of polarisation is determined bythe orientation of the reflecting surface. In the case ofdiffuse polarisation, initially unpolarised light sponta-neously acquires polarisation as it is subjected to sub-surface scattering and is remitted [7].

∗Edwin Hancock is supported by a Royal Society Wolfson Re-search Merit Award.

Using a polaroid filter as analyser, the degree of bothdiffuse and specular polarisation may be measured to-gether with the phase angle. Physically, the degree ofpolarisation is determined by the zenith angle betweenthe remitted light and surface, together with the refrac-tive index of the surface. The phase angle is determinedby the azimuth angle of the remitted light to the surface.Depending on the geometry of the scattering processand whether the origin of the polarisation is specular ordiffuse, the Fresnel theory may be used to infer surfaceorientation given known refractive index, or refractiveindex given known surface orientation. Hence, polari-sation information can be used to infer information con-cerning surface quality via refractive index determina-tion or surface shape for surfaces of constant refractiveindex.

Unfortunately, the recovery of the polarisation imagepose an obstacle to the reliable realisation of these twotasks. A polarisation camera, can determine the threecomponents of the polarisation image (i.e. mean in-tensity, polarisation and phase) from just three imagesobtained with the polariser set to three different angles[4]. However, when the polarisation is small then accu-rate determination of the phase angle becomes unstable.Moreover, impulse noise at a pixel or slight misalign-ments of the analyser images can make the estimatesunreliable.

The aim in this paper is to explore different strate-gies for improving the robustness of the estimated po-larisation image, and to investigate the knock-on ben-efits for surface inspection tasks. We make two con-tributions. First we explore how to make moments es-timates from multiple polarisation images, and use M-estimators to weight against outlying pixel values. Inthis way we improve the robustness of both the polari-sation and phase measurements. Second, we make useof techniques from directional statistics to improve thereliability of the phase estimates. When the polarisationis small the phase-angle can prove difficult to locate dueto small variations in the pixel values with polariser an-gle. To overcome this problem, we make use of recently

developed methods for smoothing vector fields from di-rectional statistics [8].

We experiment with the methods that have been usedto develop techniques for computer vision for both sur-face shape geometry recovery [4], [3] and surface qual-ity inspection [6], [5]. In the former case, we use imagesof objects with known geometry and refractive index,where we test the consistency of the estimated zenithand azimuth angles with ground truth. In the latter case,we use samples of damaged fruits to explore how wellthe method reveals local variations in refractive index.

2. Robust Estimation of the PolarisationImage

We follow the method that was proposed in [1] fordiffuse reflection, where subsurface reflections occurbefore the light is re-emitted. As already mentionedthe Fresnel theory provides the relationship between thestate of polarisation of the reflected light and the angleof reflection.

The approach that has been adopted is to take asuccession of images of the object at different orien-tations of the analyser polaroid angle for measuringthe polarisation state. The images were obtained us-ing a Nikon D200 digital SLR camera, with fixed ex-posure and aperture settings. Both polarised and un-polarised light sources have been used. In both casessequences of images are captured as the analyser angleis increased by increments of 10 ◦ to give 19 images perobject. The reflections from the environment are min-imal as the experiments are conducted in a dark roomwith walls painted matte-black and the equipment iso-lated by black screens.

2.1 Robust Moments Estimators

The conventional way to estimate the components ofthe polarisation image (mean-intensity I0, degree of po-larisation ρ and phase φ ) is to use the least-squares fit-ting to 3 images [1]. Here we explore how use robustmoments estimators to recover the components of thepolarisation image from large sets of analyser images.

Suppose that the angle of the analyser is αi, where iis the index of the analyser setting. The predicted pixelbrightness as a function of analyser angle is given by

I(αi) = I0

{1 + ρ cos(2αi − 2φ)

}(1)

To develop our M-estimators for the polarisation imagewe require a weighting function. Let Ii be the mea-sured pixel brightness with analyser angle αi, then we

can compute a weight-function based on the differencebetween predicted and measured pixel brightness. Herewe use the Gaussian, and the weight associated with themeasured brightness at a pixel with analyser setting αi

iswi = exp[−k1{Ii − I(αi)}2]

where k1 is a constant. The weighted moments estima-tors for the polarisation image are computed as follows.The estimated mean intensity is simply the weightedmean of the intensity values at a pixel over the differ-ent analyser angles, i.e.

I0 =∑N

i=1 wiIi∑Ni=1 wi

To compute the polarisation, we commence by comput-ing the weighted variance of the pixel brightness valuesover the analyser angles, i.e.

σ2 =∑N

i=1((Ii − I0)/(I0))2∑Ni=1 wi

and the estimated degree of polarisation is

ρ =σ√

2√π

(2)

Finally, to compute the phase angle we commence bycomputing

Y =

∑Ni=1 wi

Ii−I0I0

cos 2αi∑Ni=1 wi

then the moment estimator of the phase angle is

φ = arccos2〈Y 〉πρ

(3)

Our estimation of the polarisation image is an iter-ative process. We commence with the weights set tounity. With each iteration we estimate, I0, ρ and φ.Using the parameters we compute the predicted pixelbrightness I(αi) and the fit residual (Ii−I(αi))2 whichare then used to refine the weight wi. The revisedweights are then used to make new moments estima-tors of the polarisation parameters, and this process isiterated. Here we use 10 iterations.

Finally, we note that from the Fresnel theory for un-polarised incident light the relationship between the de-gree of diffuse polarisation, the angle between the sur-face normal and the remitted light θ and the refractiveindex n is

ρ=(n− 1

n )2 sin2 θ

2+2n2−(n+ 1n )2 sin2 θ+4 cos θ

√n2−sin2 θ

(4)

2

(a)

(b)

(c)

Figure 1. Shows the (a) mean intensity, (b)Polarisation and (c) Phase Image for iter-ation 1, 5 and 10

(a)

(b)

Figure 2. Shows the Mean Intensity , Po-larisation, Phase and Smoothed PhaseImage for the (a) proposed method and (b)three images

2.2 Improving estimates of φ

One of the problems the hinders the estimation ofthe polarisation image, is that when the degree of polar-isation is small then the variation in the measured pixelbrightness are small. As a result it becomes difficult toreliably estimate the phase angle, φ. As a second step inthe robust estimation of the polarisation image we useideas from directional statistics to smooth the field ofphase angles.

To this end we introduce the weight function for thepixel indexed j

w(ρj) = 1− e−ρ2

k2 (5)

where k2 is a scale-factor. We treat the polarisationweight and phase at the pixel indexed j as a 2-vectorwith magnitude w(ρj) and direction φj . We can re-solve this vector into components xj = w(ρj) cosφj

and yj = w(ρj) sinφj . Our idea is to smooth the result-ing vector field over the pixel sites of a neighbourhoodN of j, weighting against pixels with small values ofρ. To this end, we compute weighted averages of thecomponents of the weight vector field,

xj =

∑j∈Njw(ρj)cosφj∑

j∈Njw(ρj)andyj =

∑j∈Njw(ρj)sinφj∑

j∈Njw(ρj)(6)

and the smoothed value of the phase angle is

φj = arctanyj

xj(7)

When k2 is small then the smoothing becomes equiva-lent to smoothing a vector field in which the length ofthe vectors is ρ.

3 Results

We have tested our new method for computing thepolarisation image on various objects. These includefruits and vegetables, Figure 4 where our aim is to usethe polarisation image for quality inspection. These in-clude a slightly rotten pear, tomato and apple. We havealso studied contrived objects to allow us to explore theeffectiveness of the method under controlled conditions.Here we have used a ping pong ball, with layers of ma-terial applied. We have covered the surface with stripesof vaseline and olive oil, and then rolled part of the sur-face in fine cement powder.

Our first experiment involves the ping-pong ball, andexplores which of the available methods for comput-ing the polarisation image best locates the changes insurface properties. Figure 1 shows the results ob-tained. We have compared our method with a)the 3image method, with and without smoothing the phaseangle field, and b) the method of moments. It can beseen from the results that the proposed method high-lights the changes in the surface thickness, and revelsareas where the surface has been rolled up in cementwhich is not an even coating. It was observed that usingthe 19 images gives finer surface detail than using 3 im-ages as shown in Figure 2. For the polarisation image tobe visually more comprehensive histogram equalisationhas been applied to enhance the contrast. As the valuesof the degree of polarisation increase at the boundary of

3

(a)

(b)

Figure 3. Shows the vector field for thesmudged, central and lower part of theball where (a) is the Smoothed phase and(b) is the Unsmoothed phase

(a)

(b)

(c)

Figure 4. Shows the Mean Intensity , Po-larisation and Smoothed Phase Image for(a) tomato, (b) pear and (c) apple

the object resulting in a highlight at the boundary, whilethe rest of the object appears dark.

In particular the phase images are more detailed andless noisy when the moments method is used for 19 im-ages, as opposed to the 3 image method. Similarly,the degree of polarisation are also more detailed aftersmoothing.Figure 1 illustrates the mean intensity, po-larisation and phase images that have been computed

using the moments method, for 10 iterations. It was ob-served that the moments method stabilized in the thirditeration, determining the accuracy of the values com-puted by the M-estimators. In the vector fields, Fig-ure 3 it can be observed that when the material changesthere is a change in the vector fields also the change isvisible in the phase images and more pronounced in thesmoothed phase images.

4 Conclusion and Future Work

To conclude, it can be seen from the results that theuse of 19 images improves the results as there are lesserestimations in that case as compared to the use of 3 im-ages to determine the degree of polarisation and phase.The seed values computed by the moments method areas accurate as the least square fits and hence better re-sults. The smoothing of the polarisation data gives abetter estimation. In order to further this work it can becarried on to approximate the refractive indices of ob-jects that do not have known geometry as now we haveonly used it to compute the refractive indices of objectsof known geometry.

References

[1] G. A. Atkinson and E. R. Hancock. Robust estimation ofreflectance functions from polarization. Springer Berlin /Heidelberg, 2007.

[2] E. Hecht. Optics. Addison Wesley Longman, 1998.[3] D. Miyazaki, M. Kagesawa, and K. Ikeuchi. Determining

shapes of transparent objects from two polarization im-ages. IAPR Workshop on Machine Vision Applications,pages 26–31, 2002.

[4] D. Miyazaki, R. Tan, K. Hara, and K. Ikeuchi.Polarization-based inverse rendering from a single view.volume 2, October 2003.

[5] O. Morel, F. Meriaudeau, C. Stolz, and P. Gorria. Polar-ization imaging applied to 3d reconstruction of specularmetallic surfaces. SPIE, 5679, January 2005.

[6] G. Sun, E. Onoichenco, Y. Fu, Y. Liu, R. Amell, C. Mc-Candless, R. Reddy, G. Kumar, and M. Guest. High-throughput polarization imaging for defocus and dose in-spection for production wafers. volume 6518. SPIE Pro-ceedings, 2007.

[7] L. Wolff and T. Boult. Constraining object features usinga polarisation reflectance model. IEEE Transactions onPattern Analysis and Machine Intelligence, 13(7):635–657, July 1991.

[8] F. Zhang and E. Hancock. New riemannian techniquesfor directional and tensorial. Pattern Recognition, 2010.

4