ear identification based on surveillance camera images

PROCEEDINGS

Ear identification based on surveillance camera images

AJ HOOGSTRATE, H VAN DEN HEUVEL and E HUYBEN

Netherlands Forensic Institute, Ministry of Justice, Volmerlaan 17, 2288 G D Rijswijk, Netherlands

Science & Justice 2001; 41: 167-172

PROCEEDINGS OF THE FORENSIC SCIENCE SOCIETY MEETING 'EUROPEAN CONNECTIONS'

APRIL 2001

Introduction test motivated by the above-mentioned case. As there are no In The Netherlands, as well as in many other countries, the recognized experts in identifying individuals based on use of surveillance cameras has grown rapidly. As a conse- images of ears available at our laboratory, we asked foren- quence an increasing number of crimes are recorded on sic experts from several forensic disciplines with a thorough video by surveillance cameras. Therefore, the question of knowledge of individualization principles and laymen to identification of the person(s) appearing in the recorded individualize persons by ear from surveillance camera images is put to the forensic expert more and more fre- video film under closed set assumptions. quently. However, this can be a formidable task, especially

The reason for asking laymen and trained investigators is when the recordings are of poor quality and large areas of

that the answer to the question whether you have sufficient the body are covered with clothes andlor masks.

information to make an individualization or exclusion is Sometimes an expert is asked to, if possible, identify a person based on a visible body part, for example, part of the skin, hands, arms, etc. A similar question was posed to the experts at our laboratory. It concerned several robberies of gas stations in Utrecht, The Netherlands. Video camera surveillance film was obtained from the scenes of the crime. The offender was wearing a baseball cap, shawl and a cloth hanging from the cap thereby covering his face. However, the ears of the offender were visible and the question was raised whether a certain suspect could be identified as being the offender.

For a possible identification two questions have to be answered. First, are ears unique or rare enough to enable individualization? Second, are the recordings of a quality good enough to distinguish sufficient information for individualization, and is the information picked up and correctly analyzed by the person performing the individualization?

In this paper we discuss both questions. We conclude that the scientific answer to the first question is at this moment inconclusive, although there is at the moment some evidence in favour of the ear showing sufficient variability to

rather subjective, as long as it is not clear exactly what and how much information is necessary to individualize. Trained investigators are more likely to be (too) careful as it is not yet known which characteristics can be attributed as being individual and which as class. They might therefore conclude that there is insufficient information while actually there is. It is hoped that the laymen will pick up this information as well. The closed set assumption is imposed to circumvent the uniqueness question; i.e. the ears of the individuals taking part in the test are distinguishable.

The results show that in this test the experts were able to determine whether they had sufficient information to make an individualization or exclusion. Once they concluded they had sufficient information they made no false positives. However they were also, as expected, more careful than the laymen. Further, it is found in this data set that the percentage of false negatives is 35%, i.e., in the situations where the two movies show the same person, 65% were identified correctly. However, the most important conclusion is that the results stress the importance of good quality surveillance equipment.

enable individualization. However, this only holds if one Ears: some history can observe the ears in enough detail. The question of uniqueness and classification of ears

already has some history, although not always in a forensic To investigate the second question we performed a small

context. Some results follow, not exhaustive, from previous 8 The Forensic Science Society 2001

Key Words: Forensic science, ear identification, surveillance film, identification, film, individualization.

Science & Justice 2001; 41(3): 167-1 72 167


studies on ear biornetrics and individualization. The most prominent is by Iannarelli [I]. He considered over 10,000 ears and found no indistinguishable ears. Imhofer [2] reported that in a sample of 500 ears he only needed four characteristics to uniquely distinguish them. Other references to ear identification include ear prints and research about heredity factors [3-81. Most of this literature indicates that the variability between ears is so large that it might be possible to say that ears are unique, and possibly uniquely distinguishable on a limited number of features or characteristics. Comparing ear prints to fingerprints we note that before fingerprints were accepted as unique a very large number of comparisons contributed to the evidence in favour of the uniqueness hypothesis. In our opinion similar work must be completed before introducing ear identification on a large scale. We further note that with the growing use of other biometric measures such as for instance iris- scanning and facial features as mean of identification, a new discussion is surfacing among scientists. For fingerprints and ear prints the focus in this discussion is on how to asses the evidentiary value of the relation between the relative incomplete information usually available from the marks left at the scene of a crime and the much more detailed information available from reference prints, or even the physical finger or ear. This discussion will not be easy, nor will be the answers [9].

During the past decades in The Netherlands, the United Kingdom, West Germany, Austria and the United States, a substantial number of cases involving evidence based on ear prints and some isolated cases of video images of ears have appeared in court and in some cases accepted as evidence [8,10-131. We consider the case discussed here, which our laboratory investigated, as one of these isolated cases. However, a formal validation of the use of ear identification, being either ear prints or images of ears, has not taken place yet, to our knowledge. Finally, some experi- ments have been performed to investigate the possibilities for the use of video images of ears as means of verification

U41.

Individualization As identification (individualization) relies on uniqueness, the first hypothesis which has to be tested is whether ears are unique. Such a hypothesis is very hard, if not impossi- ble, to test formally and therefore the way forward is the same as in other biornetrics identification issues such as fin- gerprinting, facial recognition and DNA. The first step is the recognition that nature creates things and shapes with great variation between individuals and that there might be enough variation for an ear to be individualized. The second step is to collect as many observations as possible and check whether ears can be distinguished at a certain level of measured or observed accuracy. If it is the case that none of the observed ears can be classified as indistinguishable this

is regarded as evidence in favour of the hypothesis that ears are unique. Final proof.. . will never be obtained but can only be inferred [16]. Therefore, as in many other biometrics identification problems, uniqueness of ears is con- demned to be an unproved working hypothesis. A more promising way to go is to follow the path paved by DNA evidence and try to develop a model for the ear and estimate the probability of occurrence in the population and present that, in the form of a likelihood ratio, as the strength of evidence to the courts [lo].

The discussion on uniqueness or occurrence rate is difficult and will probably go on for some time. We consider that it is valuable to develop ideas and techniques that might be used once these points have been established satisfactorily by scientific or legal standards. If it turns out that it is not possible to develop good and well-validated procedures for individualization in closed set situations, the discussion on uniqueness or occurrence rate will become futile.

Besides the uniqueness or rarity of the ear, the quality of the video images and the associated possibility to distinguish sufficient characteristics of the ear are of the utmost importance, the images must enable one to individualize the ear, which means:

"The individualization of an impression [or other piece of physical evidence] is established by finding agreement of corresponding individual characteristics of such number and significance to preclude the possibility (or probability) of their having occurred by mere coincidence, and establishing that there are no differences that cannot be accounted for." [15]

As we were only interested in whether the respondents, laymen and experts, were able to correctly identify or exclude persons as being the same in two different video clips based on ear characteristics, we did not ask the experts how they came to their conclusion. This means that the reasoning on how they came to individualization was not verified, which, of course, in practice is of utmost importance. However, we leave this aspect of the individualization process for further research.

According to the definition of individualization above, individualization has to be performed based on precise defined characteristics or points (of which also the probability of random occurrence is [approximately] known). In photo- graphic and video film comparisons, like the case above, it is not always possible to define exact characteristics [16, chapter 51, or only with a great loss of information. In the test above we were actually interested in the ability of the test respondents to make the decision of individualization making use of all available information, definable or not.

One way around the dependence on precise defined points is the expert's opinion. However, in such cases the expert will often not be able to give satisfactory information on the quality of the opinion although it might be entirely correct.

Science & Justice 2001; 41(3): 167-172

AJ HOOGSTRATE, H VAN DEN HEUVEL, E HUYBEN

To get around this problem one might put the problem, whether or not put in the form of a (double) blind test, to a group of independent proven experts, collect their opinions and if they agree, present their opinion as evidence. This way one is able to get some idea about the relative value of the evidence, but it is very impractical.

The role of surveillance camera systems In general, the ability to individualize increases if the number of observable details in the images of the surveillance camera increases. Firstly, this should be an incentive to install good quality cameras and recording equipment. Secondly, it is worth thinking carefully about the positioning of the cameras and the availability of light. In practice however, numerous systems are installed with no consideration to the optimal positioning of the camera and available light. Further, it frequently occurs that the recording equipment is badly maintained, the light changed after installation of the system and that tapes are used over and over again thereby recording images with very poor quality. From a forensic point of view it is wise for the users of surveillance cameras to prevent the above problems as much as possible as it makes a successful individualization more likely.

For the forensic expert, however, these factors are given. The quality of the images is therefore influenced by a large number of uncontrollable factors. Some of the factors are associated with the surveillance system, such as the position of the camera, distance between camera and offender, quality of the tape, type of camera, etc. The available light is extremely important, especially as the ear is relatively small and rather curved, and it is the light falling on the ear which determines whether there is enough contrast to distinguish features from false shadows. Other uncontrollable factors are associated with the behaviour of the offender such as hislher clothing, hair, disguise, movements with respect to the lens, etc. All of these factors add possible complications to the process of individualization.

In the test described below we considered video images from two different gas stations with the originally installed surveillance equipment. By doing so we hope to be able to evaluate the effect of some of the uncontrollable differences between the stations and the resulting quality of the images.

second badge as masked suspects and the third badge as unmasked suspects.

The setup was motivated by several aspects, first by vary- ing the stations we were able to capture different camera positions, lighting, position and movements of the perpetra- tor and other uncontrollable factors. Second, the decision to have unmasked suspects was that in the actual case an expert had made an identification by comparing a video of a masked robber with photographs and video of an unmasked suspect. Third, the men were only available for one evening and this meant a restriction on the total number of video clips that could be made.

Before presenting the video clips to the participants the clips where digitized and put on a computer. To avoid influ- ence by other characteristics of the person (height, size, way of moving, etc.) a mask was put over the video with an opening showing the ear and as few as possible further aspects. Finally, the opening was centred in the middle of the screen.

The main goal of the test was to investigate whether the quality of the images was sufficient for individualization and whether the respondents were able to determine if they had sufficient information. We therefore asked six forensically trained persons (the "experts") and six laymen to do the test. The experts were well trained in the individualization process in their own field of expertise. As explained before, the laymen were included in the test to check on the possible over-cautiousness of the experts.

All the respondents participating in the test were presented with 40 sets of two video film clips and asked two questions. First, do you think there is enough information in these video films to make an individualization or exclusion? Second, are the individuals in the videos the same person or not? If they answered the first question positive the answer to the second question was interpreted as "hard" while if they answered the first question negative their answer to the second question was interpreted as an informed guess. The motivation for this line of questioning was twofold. We first of all wanted to know how much information there actually was in the films and whether this information was picked up consciously or unconsciously. Second, we were interested whether the respondents were able to determine whether

Description of the test they had enough information for individualization. Surveillance camera video clips were recorded at two of the gas stations that were robbed. The standard surveillance TABLE 1 Distribution of the characteristics of selected

couples. equipment present was used to make the recordings using a - -- - -

VHS-video recorder. From the Utrecht police department 22 male volunteers were asked to make a set of prescribed moves. At each station a group of the men performed the

Station 1 movements in front of the cameras wearing the same type

Person in the clips The same Not the same Total

of overalls. This was done in three badges; the first badge Station 2 9 12 2 1 were looked upon as perpetrators wearing a mask, the Total 17 23 40

Science & Justice 2001; 41(3): 167-1 72 169


TABLE 2 Number of times the participant takes a certain decision if helshe indicates having sufficient information.

Roman type: correct decisions. Bold type: erroneous decisions.

Different persons in clips Same person in clips Decision: Exclusion Ident8cation Exclusion Identification

Experts, Station 1 24 (100%) 0 4 (31%) 9 (69%)

Experts, Station 2 2 (100%) 0 0 (--I 0 (-->

Laymen, Station 1 45 (83%) 9 (17%) lS(49%) 19(51%)

Laymen, Station 2 21 (100%) 0 5 (38%) 8 (62%)

Before the respondents started the test they were given four samples to get acquainted with the video film ear compari- son. They were also told that ears of different people were different from each other, i.e., we introduced the closed set assumption.

Test results From the available films, 17 from Station 1 and 22 from Station 2, 40 sets of two video film clips were selected. In each couple both clips came from the same station. The ran- domization was performed by first putting the available combinations in a list, stratified by station and whether or not the pair of clips had the same person in each of the clips. We thereafter selected the pairs of clips randomly within the strata without replacement. We note that some of the persons featured in several clips. The distribution of the selected pairs is given in Table 1.

The same test was given to each of the six experts and six laymen. When interpreting the results one should keep in mind that the video films from Station 2 were clearly of less quality than those of Station 1. The results are presented below. There appears to be a substantial difference between the experts and the laymen.

In Figure 1 we see that experts indicate less frequently than the laymen that they have sufficient information, the experts appear to be more careful. A further striking fact is the large difference between the two stations with respect to the number of times the respondents indicate they have suf-

Expetts Laymen Expetts Laymen Station 1 Station 1 Stailon 2 Station 2

FIGURE 1 The number of times participants indicate they have sufficient or insufficient information.

A Insufficient information. W Sufficient information.

ficient information. This difference between the two stations strengthens the view that the images from Station 1 are clearer than from Station 2. The quality of the images is apparently a major factor in the decision to indicate whether they had sufficient information.

A second question is how often the participants make a correct identification if they have indicated they have sufficient information for individualization. The results are given in Table 2. It shows again that the experts are more careful when making decisions than the laymen. They did not make any incorrect identifications, although they incorrectly excluded individuals four times.

The laymen identify at Station 1 too often - nine times, incorrectly. This percentage of false identification would in practice be unacceptable. It appears as if they were led by the apparently clearer images to make more positive decisions. Another noticeable fact is that at Station 2 the laymen make no erroneous identifications and often make correct decisions. This indicates that these images contain information relevant for individualization that the experts decide to ignore.

Another interesting question is how these results compare with the results when the participants indicate having insufficient information for identification (Table 3). The results clearly show (bold type) that many more errors are made when the participants indicate having insufficient information. But the results also show that laymen as well as experts make a wrong decision less often than might be expected if they were gambling, except at Station 2, if the laymen indicate they have insufficient information. These results indicate that the images contain relevant information although the question still remains whether it would be enough to use for individualization if one had to point out what information one actually used.

To give an idea about the difference in strength of evidence between the two situations of respondents indicating having sufficient or insufficient information, we estimated the likelihood ratios of the situation that the respondents give a positive identification. The likelihood ratios range from 4.5 to infinity in case of sufficient information and from 2.6 to 3.5

170 Science & Justice 2001; 41(3): 167-1 72

AJ HOOGSTRATE, H VAN DEN HEUVEL, E HUYBEN

TABLE 3 Number of times the participant takes a certain decision if helshe indicates to have insufficient information.

Roman type: correct decisions. Bold type: erroneous decisions.

Different persons in clips Same person in clips Decision: Exclusion Identification Exclusion Identification

- --

Experts, Station 1 29 (69%) 13 (31%) 8 (23%) 27 (77%)

Experts, Station 2 58 (83%) 12 (17%) 19 (35%) 35 (65%)

Laymen, Station 1 6 (50%) 6 (50%) 5(45%) 6(55%)

Laymen, Station 2 42 (82%) 9 (18%) 15 (37%) 26 (63%)

in cases of insufficient information. We note that these 11.16) however, does not differ significantly. The graphs numbers can only be rough indications because much larg- again illustrate the effect that forensically trained people are er numbers of observations are needed to evaluate with better in deciding whether they have sufficient information. accuracy the strength of the evidence. Compare this to practice where, except in cases of DNA evidence where very large likelihood ratios are presented, most courts are content to regard as strong evidence likelihood ratios between 10 and 50 [17].

When we combine the results of the Tables 2 and 3 then we see that the experts are especially careful when coming to conclusions. As soon as they indicate that they have insufficient information the percentage of erroneous conclusions increases dramatically. Needless to say that false positives are very unwelcome in forensic practice.

The distribution of the erroneous decisions per respondent unconditional and conditional on the indication of having sufficient information is given in Figure 2.

There is a clear difference in the distribution of the errors between the groups of respondents and also between individual respondents. For instance Respondent 1 made half the errors that Respondent 6 made. The difference between the averages of the two groups (experts 9.3 and laymen

Total Sufficient inforrnat~on lnsufficlent ~nforrnation

-10 0 5 ID -10 0 5 10 -10 0 5 10

123 erroneous conclusions 36 erroneous wnclusions 87 erroneous conclusions of480 out of 164 out of316

FIGURE 2 Distribution of erroneous decisions. Respondents 1-6 are experts, 7-12 are laymen, negative numbers indicate the number of false negatives, the positive numbers the false

positives.

Another interesting question is whether some comparisons were more difficult than others, which would indicate that some ears are harder to individualize than others. First we note that there is no significant difference in the total number of erroneous decisions between the two stations. This is a little surprising, most respondents clearly indicate not to have sufficient information at station 2 as the images are certainly less clear than those recorded at station 2. However, Figure 3 shows that there are considerable differ- neces between the pairs. From Figure 3(a) and 3(b) we see that the distribution of decisions differs substantially between experts and laymen. We see that some comparisons were apparently more easy than others. The reasons for this are not yet clear and left for further research. The most important lesson probably from the above is that the forensically trained persons were able to determine whether they had sufficient information for individualization. Further it is important to note that the quality of the images influences this decision to a large extent. Finally, we observe that the images contain information that can be used to distinguish between individuals in this data set.

Conclusions In this paper we discussed the possibility of identification by ear from surveillance camera film. First, the individualization of ears was discussed. It is argued that the literature indicates that it might be possible that ears, and maybe images of ears and ear prints, show enough variability to enable identification or individualization based on an individual's ear morphology. However, it is pointed out that more evidence needs to be gathered to make it an accept- able working hypothesis. Further, we investigated with a small experiment whether in a closed set situation respondents, forensically trained persons and laymen, can individualize individuals by ear from surveillance camera film and whether they can decide whether they have sufficient information to come to an individualization or exclusion. Both questions can be answered positively for this test. On the

Science & Justice 2001; 41(3): 167-172 171


False negative

d ip no. (a)

dip no. (b)

FIGURE 3 Classification of decisions per pair of clips when the respondents indicate having sufficient information. (a) Ex~erts. (b) Lavmen.

whole 65% of possible matches were identified and especially the forensically trained respondents were able to determine whether they had sufficient information for identification, they made no false positives. Also, the use of better camera surveillance equipment together with proper installation and handling will increase the probability of actually being able to identify offenders.

Acknowledgements In this research numerous people have participated, therefore we wish to thank: the Utrecht Police Department for organizing the shooting of the film clips for the test; Adee Schoon for advice on the design of the test; Jurrien Bijhold for his expertise in digital imaging; and Marjan Sjerps for many helpful discussions. We especially thank the NFI staff that cooperated in the test.

References

1. Iannarelli A. Ear Identification. Forensic Identification Series. Paramount Publishing Company, Fremont, California 1989.

2. Irnhofer R. Die Bedeutung der Ohrmuschel fiir die Feststellung der Identitat. Archiv fur die Kriminologie, Bd. 26, 1906; 150-163.

3. Hirschi. Identifiziemng von Ohrabdriicken. Kriminalistik 1970; 24: 75-79.

4. Hammer H-J, Neubert F. Experimentelle Untersuchungen zur Auswertung van Ohrabdruckspuren. Krirninalistik und forensische Wissenschaften 1989; 73,74: 136139.

5. Hunger H, Hammer H-J. Zu Fragen der Identifikation durch Ohrmerkmale. Kriminalistik und forensische Wissenschaften 1987; 65,55: 73-79.

6. Oepen. Der Identifizierungswert des Menschliches Ohres. In: Protokoll zur Arbeitstagung an der PFA Munster, 1976; 39-54.

7. Rother. Diskussionsbeitrag zum Referat 'Der Identifizierungswert des Menschliches Ohres'. In Protokoll zur Arbeitstagung an der PFA Miinster, 13-15.10.1976 pp.55-61. Mit vielen Literatuhrhinweisen.

8. Van der Lugt C. Ear Identification-State of the Art. Information Bulletin for Shoeprint/Toolmark Examiners 1998; 4(1): 69-81.

9. Saks MJ. Merlin and Solomon: Lessons form the Law's Formative Encounters with Forensic Identification Science. Hastings Law Journal 1998; 49: 1069-1141.

10. Ingleby M, Mason J, Turner D and S Langdell. Ear-prints a neglected forensic resource? Working paper, Dept. of Mathematics, Huddersfield University, 2000.

11. Hammer H-J. The identification of ear-prints secured at the scene of the crime. Fingerprint Whorld 1986; 49-5 1.

12. State v. David Wayne Kunze. Court of Appeals of Washington, Division 2. 97 Washington, Division 2. 97 Wash. App. 832, 988 P.2d 977 (1999).

13. Morgan J. Court Holds Earprint Identification Not Generally Accepted In Scientific Community. http:llwww.forensic- evidence.com/site/ID~Kunze.html. Editor A. Moenssen, 2000.

14. Burge M and Burger W. Ear biometrics. In: Biometries: Personal Identification in Networked Society. Ed: Jain A. et al., Kluwer Academic Publishers 1989.

15. Tuthill H. Individualization: Principles and procedures in criminalis- tics. Salem, OR: Lightning Powder Company, Inc. 1994.

16. Kwan QY. Inference of Identity of Source. Ph.D. dissertation. Department of Forensic Science. Berkeley, CA: University of California, 1976.

17. Robertson B and Vignaux GA. Interpreting evidence. Evaluating Forensic Science in the Courtroom. Chichester: John Wiley and Sons. 1995.

Science & Justice 2001; 41 (3): 167-1 72

ear identification based on surveillance camera images

Documents