iso/iec standards for on-card biometric comparison · iso/iec standards for on-card biometric...

30 Int. J. Biometrics, Vol. 5, No. 1, 2013

Copyright © 2013 Inderscience Enterprises Ltd.

ISO/IEC standards for on-card biometric comparison

Tai-Pang Chen* and Wei-Yun Yau Institute for Infocomm Research, 1 Fusionopolis Way, #21-01 Connexis (South Tower), 138632, Singapore E-mail: [email protected] E-mail: [email protected] *Corresponding author

Xudong Jiang School of Electrical and Electronic Engineering, Nanyang Technological University, S1-B1c-105, EEE, NTU, 50 Nanyang Avenue, 639798, Singapore E-mail: [email protected]

Abstract: On-card biometric comparison is getting more attention from government and the IT industry because of the higher level of security and its ability to prevent the enrolment template from leakage that can cause privacy concern. To address the interoperability needs, the ISO/IEC SC17 WG11 published a standard entitled ISO/IEC 24787 – ‘On-card biometric comparison’. This paper will give an introduction to this new standard and analyse the limitation of implementing minutiae matching algorithm using this standard on low-cost smartcard. We will present a novel algorithm to perform off-card template alignment using the work-sharing mechanism. The final matching process is executed on-card to ensure that the security is not compromised. The average verification time of our proposed algorithm is about 2.5 seconds with an 8-bit Java card with an average EER <= 4.3% using FVC2000 and FVC2002 databases. Hence, the performance is comparable to those minutiae matching algorithms running on PC.

Keywords: fingerprint; biometric comparison; minutiae matching; local and global structure; work-sharing mechanism; smartcard.

Reference to this paper should be made as follows: Chen, T-P., Yau, W-Y. and Jiang, X. (2013) ‘ISO/IEC standards for on-card biometric comparison’, Int. J. Biometrics, Vol. 5, No. 1, pp.30–52.

Biographical notes: Tai-Pang Chen received his BEng in 1996 and MEng by research in 1998 in Computer System Engineering, both from the Royal Melbourne Institute of Technology University, Australia. Currently, he is working at the Institute for Infocomm Research as an Assistant Programme Manager for the Interactive Social Tele-Experience Programme. From 2008 to 2010, he was the Project Editor of ISO/IEC 24787 – information technology – identification cards – on-card biometric comparison in ISO/IEC JTC1 SC17/WG11. He is the recipient of the Standard Council Merit Award 2009. He is also pursuing a part-time PhD at the School of Electrical and Electronic Engineering, Nanyang Technological University. His research interests are in the area of fingerprint authentication, minutiae matching on resources-constrained platform, pattern recognition and parallel processing algorithm for computer vision and secured biometrics.

ISO/IEC standards for on-card biometric comparison 31

Wei-Yun Yau received his BEng from the National University of Singapore (1992), and MEng (1995) and PhD (1999) from Nanyang Technological University. Currently, he is a Programme Manager at the Institute for Infocomm Research, A*STAR. He also serves as a member of the IAPR’s Technical Committee on Biometrics and Chairman of IPTV Working Group and Biometrics Technical Committee, Singapore. He is the recipient of TEC Innovator Award 2002, Tan Kah Kee Young Inventors’ Award 2003 (Merit), IES Prestigious Engineering Achievement Awards 2006 and Standards Council Distinguished Award 2007. His research interest includes biometrics, active vision system, and interactive TV and has published widely, with eight patents granted and over 100 publications. One of his papers received the Pattern Recognition Journal Honorable Mention 2010.

Xudong Jiang received his BEng and MEng from the University of Electronic Science and Technology of China in 1983 and 1986, respectively, and PhD from Helmut Schmidt University Hamburg, Germany in 1997, all in Electrical Engineering. Currently, he is a tenured Associate Professor at Nanyang Technological University, Singapore. He has published over 100 papers and holds seven patents. His research interest includes signal/image processing, pattern recognition, computer vision, machine learning and biometrics.

1 Introduction

Biometrics has been deployed for cross border control extensively in the last decades. For example, Hong Kong, Malaysia and Singapore are already using biometric enabled automatic gantry at checkpoints to allow eligible residents to use electronic ID (e-ID) card or electronic passport to cross the border which reduces the queuing time for immigration clearance significantly. Many other countries in the world are issuing e-ID card for their citizen to access government services and e-passport for immigration. USA requires all VISA waiver countries to issue e-passports with biometrics for their citizens to enter the US border. In order to proof the identity of the owner of the card or passport, it contains the facial photo and usually also the fingerprint(s) of the owner. Facial photo is needed as it can be used for manual verification while fingerprint is chosen because of its good performance and acceptance by the court of law.

Automated fingerprint authentication research and development has been popular for decades. The advancements in computer system, fingerprint sensor technology, smartcard and energy efficient processor have enabled automated fingerprint authentication applications involving smartcards to be affordably deployed on a large scale. However, most of the existing solutions use the smartcard only to securely store the fingerprint reference data, also known as the template. The fingerprint reference data is sent to the biometric terminal via external communication to verify the person’s identity. Such system is categorised as off-card biometric comparison since the biometric comparison is executed external to the smartcard. The major advantages of such technique are:

1 ease of implementation

2 ability to use lower cost smartcard since the only minimal computation is required to be done at the smartcard.

32 T-P. Chen et al.

However, as external communication is involved during the verification process, such communication becomes a security loophole. If such external communication is not protected properly, it could compromise the overall security of the system. In order to protect the biometric reference data stored in the smartcard, cryptographic protection scheme such as secure messaging in smartcard is necessary. Another weakness is if the key for the crypto-operation is compromised or the crypto-mechanism is broken, the user’s information and biometric reference data will be lost and revealed. Unlike the personal identification number (PIN), the biometric reference data cannot be revoked. Thus if a particular biometric reference data (e.g., the minutiae template of the right thumb of a user) is lost, the user will not be able to use the particular body part for authentication anymore.

To overcome the potential security loophole of off-card biometric comparison, on-card biometric comparison was proposed. The on-card biometric comparison system performs secured authentication inside the smartcard itself. The user sends his or her biometric query template to the smartcard to request for on-card authentication. As the external terminal did not have a copy of the biometric reference data stored in the smartcard, it is relatively more secure than off-card biometric comparison. Hence, on-card biometric comparison provides stronger security protection for biometric authentication. In 2006, the sub-committee 17 (SC17) under the Joint Technical Committee of International Organization for Standardisation (ISO), and International Electrotechnical Commission (IEC) formed a new Work Group 11 (WG11) to look into standardisation of technologies related to smartcard with biometrics. An outcome of WG11 is the standard ISO/IEC 24787 (2010) – ‘Information technology – identification cards – on-card biometric comparison’ with the scope to define the functional blocks and components for the use of smartcards in applications where the comparison of biometric identifiers is to be performed on-card. ISO/IEC 24787 specifies the functional blocks and components to implement on-card biometric comparison, which has been published as an international standard on 15th December 2010. Certain standardised components have constraints that limit the existing algorithms for biometric comparison to be applied directly. Hence, we used the framework from the ISO/IEC 24787 specification to design an on-card fingerprint authentication algorithm with work-sharing mechanism that allows low-cost smartcard or Java card to be used for fingerprint authentication. The work-sharing mechanism is used to compute the off-card reference minutiae computation and the off-card query template alignment which can speed up the overall matching time as low-cost card is not able to handle those processes with intensive computation. Our proposed algorithm is able to handle the constraints of respective standardised components to optimise the overall performance in terms of matching speed and accuracy for low-cost card. The smartcard using our algorithm does not need to send the minutiae points stored in the enrolment template to any external terminal for authentication that overcomes the security loophole of traditional off-card biometric comparison.

In this paper, a novel algorithm is presented that performs off-card reference minutiae computation and off-card query template alignment. First, we give an introduction on the implementation of on-card fingerprint comparison using ISO/IEC in Section 2. Next, an overview of the existing fingerprint matching technique is given in Section 3. In Section 4, an introduction and description of our proposed method to perform secure fingerprint comparison using minutiae information with a novel alignment scheme by searching a cluster of connected minutiae structures is stated. A new methodology of


using the work-sharing mechanism specified in ISO/IEC 24787 for minutiae matching is addressed in the same section. Section 5 provides a detailed analysis of the performance of the proposed method while Section 6 concludes the paper. The security policy mentioned in the appendix of ISO/IEC 24787 which is application dependent will not be addressed in this paper.

2 Relevant standards for on-card biometric comparison

The ISO/IEC 24787 specifies the general requirements for performing comparison of biometric samples, returning decisions on a smartcard, and implementing security policies of on-card biometric comparison. In addition, other standards document needed as normative references are:

1 ISO/IEC 7816-4 (2005) ‘Identification cards – integrated circuit cards – Part 4: organisation, security and commands for interchange’

2 ISO/IEC 7816-11 (2006) ‘Identification cards – integrated circuit cards – Part 11: personal verification through biometric methods’

3 ISO/IEC 19785-1 (2006) ‘Information technology – common biometric exchange formats framework – Part 1: data element specification’

4 ISO/IEC 19785-3 (2007) ‘Information technology – common biometric exchange formats framework – Part 3: patron format specifications’

5 ISO/IEC 19794-2 (2005) ‘Information technology – biometric data interchange format – finger minutiae data’

6 ISO/IEC 29794-1 (2009) ‘Information technology – biometric sample quality – Part 1: framework’.

The above standards cover three main areas: smartcard operation, biometric sample quality and data structure for template storage. Smartcard operation is supported by the smartcard chip manufacturer and the operating system developer. Designer can refer to the data sheet of the smartcard chip to control the operation of smartcard using ISO/IEC 7816-4 conformed APDU commands. For example, accessing the memory of the smartcard requires the GET/PUT command specified in ISO/IEC 7816-4 clause 5.1 command-response pairs of application protocol data unit (APDU).

The biometric sample quality provides guidance for designer to implement algorithm to detect low quality biometric samples which are not suitable for use as poorly captured biometric sample increases the error rate. The third area is the data structure of the biometric reference data or biometric template. Biometric template is a representation which contains the properties of extracted features of a particular biometric sample. Standard for the biometric template is needed to support interoperability and interchange of biometric data. As per ISO/IEC 7816-11 and ISO/IEC 24787, a compact size format specified in ISO/IEC 19794-2 shall be used to encode the finger minutiae template for on-card fingerprint comparison.

Table 1 shows the structure of biometric information template (BIT) as specified in the compact size format of fingerprint minutiae data.

34 T-P. Chen et al.

Table 1 BIT of the fingerprint template

Tag Length Value Presence 7E2F Variable Biometric data template Mandatory Tag Length Value 90 Variable Finger minutiae data with

compact card format. Mandatory

82/A2 Variable Biometric data with proprietary format.

Optional

A tag-length-value (TLV) encoding scheme as specified by ASN.1 described in X.680 (2002) is used to encode the BIT to store the finger minutiae data. In the above table, the minutiae data is stored under the Tag 90 using the compact size format. Core, delta, ridge count, cell quality and data with proprietary format may be optionally encoded under the tags listed in ISO/IEC 19794-2 inside the BIT based on the requirement of the fingerprint matching algorithm. The compact size format of finger minutiae is shown in Table 2. Table 2 Format of single finger minutia in compact size format

x-coordinate y-coordinate Type 2-bit angle 6-bit

1 byte 1 byte 1 byte

The Cartesian coordinate, the type and the ridge direction of given minutiae are recorded in the compact size format with 3 bytes per minutia. The Cartesian coordinate uses two bytes to record while the type and the ridge direction (angle) are combined into a single byte to encode. The unit of the Cartesian coordinate is 0.1 mm per step. For the type-angle encoding, the most significant two bits are used to encode the type of minutiae (00b – other, 01b – ridge end, 10b – ridge bifurcation, and 11b – reserved) while the least significant six bits are used to encode the angle of the ridge direction in the unit of 2π/64. A normal size finger minutiae format is also available, but it consumes more memory (five bytes per minutiae) and requires 16-bit computation which may not be suitable for low-cost 8-bit smartcard. Due to the limited minutiae information in the compact size format and the constraint in the computing resources in the smartcard, it is not easy to design a matching algorithm to perform minutiae matching with similar accuracy and speed as the PC. It is useful to keep in mind that the computational power and storage space of a smartcard is generations behind that of a PC.

In this paper, we will only focus on the use of compact size format of finger minutia data in order to cater to the smartcard’s requirement. The key contributions of this paper are description of the use of ISO/IEC 24787 (2010) clause 8 work-sharing scheme to perform external alignment at the biometric terminal that speeds up the overall fingerprint matching process on an 8-bit low cost smartcard and a fingerprint matching scheme using such minutia data.

3 Overview of fingerprint matching algorithm

In general, fingerprint matching can be classified into three categories: correlation-based, minutiae-based and ridge feature-based. Among these three categories, minutiae-based method is the most suitable method to be implemented in smartcard as this method


requires the lowest memory and computational cost for the matching operation. Moreover, minutiae-based matching has good authentication accuracy. The algorithm of minutiae matching can be generalised into the following equation (Maltoni et al., 2003):

( )( ), , ( ), , ,1

Maximise ,m

x y θ Search i ix y θ Pi

Score Compare TranslateΔ Δ ΔΔ Δ Δ

=

′= ∑ m m (1)

where Compare(.) is the function to perform minutiae comparison, Translate(.) is the function to perform geometric translation of the set of minutiae given by Δx, Δy and Δθ, m and m′ are vectors of minutiae points from the enrolment template and the query template respectively. The score computation is an optimisation process to search for the inter-template similarity using minutiae points. The goal of the minutiae comparison process is to find out the maximum number of pairs of matched minutiae between the enrolment template and the query template while keeping the matching error to be minimum within the matched sub-structures inside the minutiae map. The Search(.) function is to find a minutiae index as the common reference base point for comparison. The easiest way is to use brute-force search which examines every minutia in the template as reference base point to test for similarity with angular correlation within a given range. Such correlation is very time consuming and not reliable as the uniqueness of such small structure is not high. Due to the distortion caused by elastic deformation of finger skin and the inexactness in the finger placement, the minutiae map generated during verification will not be identical to the minutiae map generated during enrolment. Hence, for fingerprint comparison, we can only measure the similarity within certain tolerance whether the query fingerprint is similar to the enrolled fingerprint.

Minutiae-matching is a kind of point matching method to find out the similarity of two sets of data using the features extracted from the minutia. Each minutia contains three major features: coordinate, ridge direction and type, but each fingerprint may contain different number of minutiae. Additional features, such as ridge count between minutiae, can be used to enhance the accuracy. One of the classical methods is using geometric hashing to compare the similarity of two sets of points. Germain et al. (1997) and Chung et al. (2005) suggested using geometric hashing to perform fingerprint matching in the database. The major advantage of this method is its ease of implementation with reasonably good accuracy. However, geometric hashing requires rotating the query template in multiple steps to generate different hashes to compare with the enrolment template. The search dimension to compare hashes is proportional to the angle of the allowed rotation of fingerprint. As such, this method is not efficient to be used in smartcard. Huvanandana et al. (2000) proposed coarsely quantising the locations of minutiae in both enrolment and query. A brute-force search can be applied within the coarsely quantised region for alignment but the speed-up is still not satisfactory. To speed up the matching speed using hashes, Pan et al. (2003) proposed a memory efficient algorithm using multi-scale geometric hashing technique to perform on-card matching which lowers the requirement, but this method still requires more than 1 K RAM with a 32-bit processor which is only suitable for expensive high-end smartcard.

Another matching approach is to decompose the computation of the fingerprint into two separate sections: a local structure and a global structure (Jiang and Yau, 2000). The local structure contains the relative distance, relative direction and relative ridge count between minutia neighbours, which is used to search for the common reference point.

36 T-P. Chen et al.

The found reference point is then used to align the query and enrolled template for the final minutiae comparison process. The global structure, which contains the absolute information such as locations of minutiae, is used in the final minutiae comparison process to compute the similarity level as the final matching score. This method searches for the pair of local structures with least error, and then uses this matched pair to perform alignment, and then calculate the matching score. The advantage of this method is able to simplify the exhaustive global search by limiting the search space to a fixed number of nearest neighbours of each minutia which reduces the search dimension significantly compared to brute-force search. However, the intensive computation of the cross-match process for the local structures is not suitable to be used for smartcard directly. Improved algorithms such as Cao et al. (2009) and He et al. (2006) suggested using similar method but introducing the penalised logistic regression and global compressive similarity with ridge-based features respectively to enhance accuracy. Both methods introduced additional rotational invariant features to assist the local structures search for alignment and matching such as ridge-based features. All these methods do not have any suggestion to speed up the matching process for resource-constrained device. Jain et al. (1997) introduced a matching method which is associated with minutiae and ridge line structure. Feng et al. (2006) proposed a matcher that was based on ridge and minutia structure to perform matching. The combined features can improve the matching accuracy over minutiae alone but it incurs additional time to perform the matching. Moreover, this method is susceptible to noisy image and strong deformation.

Some researchers suggested using hybrid approaches to enhance the accuracy. Feng (2008) proposed combining minutiae descriptor which comprises the minutiae information and texture-based information to improve the accuracy. Krivec et al. (2003) proposed a hybrid fingerprint matcher, which combines minutiae matcher and homogeneity structure matcher, to perform authentication on the smartcard. However, this hybrid approach cannot increase the accuracy significantly compared to minutiae matcher alone but incurs extra processing time to perform host side matching. Besides solely using minutiae information, Rikin et al. (2005) proposed using minutia ridge shape for fingerprint matching. The ridge shape information is used during the minutiae matching to improve the matching accuracy. In their experiment, they showed that the accuracy was comparable with the conventional matching but with a faster matching speed.

Some researchers attempted to design minutiae matching algorithm for smartcard. Allah (2005) proposed a memory efficient scheme implemented on 32-bit DSP using line extraction of fingerprint that could speed up the matching process. Mimura et al. (2002) proposed a pattern-based scheme to perform fingerprint match-on-card. The system sends the core to the smart card to obtain the correction of translational parameters. Sanchez-Reillo et al. (2002, 2003) are other reference implementations of fingerprint match-on-card for specific application. These two reference implementations addressed the basic components and requirement for the match-on-card. Bistarelli et al. (2005) proposed a matching method using local relative information between nearest minutiae. In general, all the above methods are attempts to implement minutiae matcher on smartcard but require a 16-bit or higher MCU to implement.

Smartcard with 16-bit or higher MCU is available nowadays but the cost may not be affordable for many applications. The low-cost 8-bit card is still the mainstream smartcards in the market. In order to implement on-card fingerprint comparison using


low-cost card, based on the above review of the current state-of-art, we have the following problems that need to be solved:

1 minutiae template conformance to the ISO standard for interoperability contains limited information for matching

2 the matching engine should be light weight enough to be executed on low-cost smartcard or Java card

3 efficient and accurate reference base search scheme is necessary but smartcard may not be able to handle such intense computation

4 the enrolment template with absolute minutiae information should never be revealed to the public.

4 Fingerprint matching on smartcard

According to ISO/IEC 24787 and ISO/IEC 19794-2, the minutiae data format shall be used to encode the minutiae template for on-card fingerprint comparison. Table 1 shows the template in ASN.1 encoding format. The template can be separated into two sections: mandatory section and optional section. The mandatory section contains the Cartesian coordinate, ridge direction and minutia type. This part of information is the descriptor of the minutiae template which shall not be revealed to the public as it allows for possible reconstruction of a template. Hence, we call this portion as the secured portion of the minutiae template and shall be protected. During enrolment, once the secured portion is generated, this portion shall be stored inside the secured memory section of the smartcard. No application can send this portion to any external device.

However, the secured portion contains only the graph structure of minutiae map which is difficult to perform minutiae matching inside the smartcard as mentioned in the last section. An extra portion is necessary to be generated to assist matching process especially for the reference base search process which involves intensive computation that a low-cost smartcard is not able to handle. It is fortunate that the standard specifies an optional section which allows extra information to be stored inside the smartcard which can be used to assist the overall matching process. According to the specification of ISO/IEC 24787, this optional section belongs to the public section which the smartcard can send to the terminal. Using this portion to perform computation, the on-card matching process can be speed up. This new portion of the minutiae template, called the open portion, can be sent to any external device. In the next section, we will introduce the method of using the open portion to assist the off-card reference minutiae search process on the external terminal.

4.1 Minutiae template and off-card operation

A minutiae template can be divided into a local structure and a global structure (Jiang and Yau, 2000). The local structure is a sub-structure which contains the relative distance, relative direction, and relative ridge count among minutia neighbours. Since it describes the minutia, the local structure is used to search for the common reference point and then to align the global structure for subsequent comparison process. The global structure,

38 T-P. Chen et al.

which is in Cartesian coordinate representation, contains the physical properties of the detected minutiae points in the template. Each minutiae point can be described as a feature vector that is given by:

( )Tk k k k kx y tθ=M (2)

where xk and yk are the coordinates of the given minutia, θk is the local ridge direction, tk is the minutiae type (bifurcation or ridge end) and k is the index of the minutia. These parameters can be considered as a vector of geometric information. We can construct a minutiae template with all detected minutiae Mk, k = 1, 2, .., N, where N is number of detected minutiae in the template. All these features, which are specified as finger minutiae data in ISO/IEC 19794-2 as mandatory features, shall be used only for internal smartcard fingerprint matching process. Hence, this portion of template is called the secured portion of minutiae template. A template T can be constructed by combining all feature vectors M together. In this paper, Te and Tq denote as the secured portion of enrolment template and query template respectively. Enrolment template Te, which is constructed and stored in the card during enrolment, shall be protected and never to be released to the public. Hence, Te is the secured portion of minutiae template. However, by using this template solely for the matching is not easy for low-cost smartcard or Java card. Hence, we propose another portion that contains relative information to assist the smartcard to perform reference base finding and alignment. This portion is the relative minutiae information template which does not contain any information in the secured portion and difficult to be used to reverse-engineer the original template Te. Tq will not be sent to the smartcard directly for matching. The details of using Tq for minutiae matching will be addressed in Sections 4.2 and 4.3.

In Figure 1, a fingerprint pattern with detected minutiae is presented. Let Mi be one of the detected minutia, MM1

i, MM2i and MM3

i are the neighbours of Mi and i be the index of specific minutia. Three nearest neighbours were used in our implementation. The selection criteria of neighbours can be nearest distance, sequential index or random index. In this paper, the nearest distance method is used for selection of neighbours. Between the particular minutia Mi and its nearest neighbours, it is possible to compute relative information based on the relative difference of particular biometric property between minutiae specified as the following equation.

( ) ( )

( )

2 2

( , )

ji j i ji

ji

ji i j

x x y yd

r RidgeCount i j

normϕ ϕ ϕ

⎛ ⎞⎛ ⎞ − − −Δ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎜ ⎟Δ = Δ⎜ ⎟⎜ ⎟ ⎜ ⎟⎜ ⎟Δ Δ − Δ⎜ ⎟⎝ ⎠

⎝ ⎠

(3)

,( ) 2 ,

2 ,

x xnorm x x x

x x

− ≤ ≤⎧⎪= − >⎨⎪ + < −⎩

π ππ ππ π

(4)

where Δd, Δr and Δϕ are distances between minutiae, number of ridge count between minutiae and difference in ridge directions respectively.


Figure 1 Diagram illustrates a minutia Mi and its neighbours

The relative minutiae information L of given minutia with index i is defined as:

( )1 1 1 2 2 2 3 3 3i i i i i i i i i id r d r d rϕ ϕ ϕ= Δ Δ Δ Δ Δ Δ Δ Δ ΔL (5)

where Li contains the relative minutiae information of three neighbours computed by equations (3) and (4), i is the index of given minutia, i=1..m and m is the number of minutiae and j is the index of Mi’s nearest neighbour (see Figure 1). Equation (4) is used to normalise the difference of the ridge directions within the range of ±2π.

The open portion of minutiae template (Te_open and Tq_open for the open portion of enrolment template and query template respectively) can be constructed by combining all local feature vectors Li for all minutiae in the template. Of course, it is possible to use more neighbours but the computation time will be longer as well as a larger template needs to be stored. This portion of template can be sent to the external terminal to perform reference minutiae search and template alignment with the query template. It will be stored inside the TLV structure under the tag ‘82’/‘A2’ – biometric data with proprietary format as shown in Table 1.

In summary, each minutia associates with two vectors: Mi and Li. Te/Tq which consists of geometric information stored in secured portion of minutiae template, and Te_open/Tq_open which consists of relative minutiae information stored in the open portion of the minutiae template.

Te_open contains only the relative minutiae information with limited number of nearest neighbours. Hence, such information cannot be easily used to reverse-engineer the information in Mi as specified in equation (2) because:

1 The indexes of each nearest minutiae are not revealed.

2 Difficult to guess the connectivity of the local structure as the order of the indexes can be scrambled.

40 T-P. Chen et al.

3 Each nearest neighbour may or may not connect to another local structure. Possible of existing standalone L without any connectivity to others relative information structure.

4 It is possible to add ‘fake’ relative information structures inside the template to further enhance the security but the size of the template will be increased.

The Te_open/Tq_open, which can be considered as an open portion, will be encoded in the proprietary section as specified in ISO/IEC 24787 during the enrolment process. The open portion can be sent out to the biometric verification terminal (BVT) to compute the alignment process with the query template. Template Te/Tq can be encoded as secured portion of reference data according to ISO/IEC 19794-2 Finger Minutiae Speciation and stored in the smartcard as secured portion which shall never be sent out to the BVT or any other external device. The overall encoding of open template and secured template in TLV structure is shown in Table 1 as specified in ISO/IEC 24787.

Since two portions of minutiae template are defined for on-card fingerprint comparison, it is necessary to have a mechanism to perform fingerprint matching using work sharing mechanism to allow the smartcard to work with the BVT together to perform fingerprint authentication. In the next section, a work-sharing mechanism and related protocol specified by ISO/IEC 24787 will be presented to address this problem.

4.2 On-card fingerprint comparison using work sharing mechanism

During enrolment, a fingerprint template is generated which is divided into two portions: a mandatory minutiae portion Te and an optional portion Te_open. Both portions are stored inside the smartcard but the mandatory portion will never be revealed to the public. During verification, Tq and Tq_open are generated and combined as query template in the BVT. All templates can be considered in matrix form. The following diagram shows the methodology to compare the enrolment template and the query template with the aid of work-sharing mechanism.

Figure 2 shows the architecture of work-sharing mechanism which allows for on-card fingerprint verification to be aided by an external computing device. This type of on-card comparison accomplishes certain processes, such as template alignment, outside the smartcard and is designed for low-cost smartcard that does not have sufficient computing power to execute the entire matching process of biometric data. In this case, any process that is computational intensive, such as template alignment, is sent to the BVT for processing. The result of the processing, such as aligned template, is then sent back to the smartcard so that the final determination of the matching score is calculated on the card side. It is recommended that the communication between the smartcard and the biometric verification system is protected using secure messaging mechanism as specified in ISO/IEC7816-4.


Figure 2 Architecture for on-card fingerprint comparison with work-sharing

The BVT captures the fingerprint image and encodes a query template Tq for query. The BVT sends the Verify APDU command as mentioned in ISO/IEC 24787 to start the on-card comparison process. Upon receiving the Verify command, the smartcard will send the open portion of the enrolment template Te_open to the BVT. The BVT uses Tq, Tq_open and Te_open (from smartcard) to estimate the reference positions in both enrolment template and query template for alignment. Based on the reference position, an aligned query template T’q and an alignment info A (a vector consists of coordinates of reference minutiae position in Tq and the indexes of selected relative minutiae information in Te_open) are generated for subsequent on-card comparison. The T’q and A will be sent back to the smartcard. The smartcard will first use A to perform on-card template alignment to compute T’e, and then the on-card comparison module matches the aligned templates T’e and T’q to compute the similarity score S. Finally, the similarity score S will be compared with the internal predefined security threshold to decide whether the query template is from the genuine user or imposter. The template transfer mechanism is described in the work-sharing request (WSR) protocol in ISO/IEC 24787 clause 8. The above sequence can be summarised in Figure 3.

42 T-P. Chen et al.

Figure 3 Sequence diagram for on-card work-sharing mechanism using WSR protocol

The details of using APDU commands including the structure of APDU commands and the meanings of respective response words to perform work-sharing mechanism can be found in ISO/IEC 24787 and ISO/IEC 7816-4.

4.3 Off-card reference minutiae search and alignment for query template

This section will describe how to use the open template with limited information to perform reference minutiae search for alignment of the enrolment template (on-card) and the query template (off-card). This is achieved using a novel propagative minutiae structural search (PMSS) mechanism.

Figure 4 illustrates the idea of PMSS. The open template Te_open is sent to the BVT. As mentioned in Section 4.1, the Te_open contains relative information Li (where i=1…m and m is the number of minutiae) specified in equation (5). Each minutia associated with the respective vector Li contains the relative information of itself with its three neighbours. For clarity and simplicity, only those matched pairs of minutiae (with respective two neighbours) are shown in the diagram. The proposed method uses Te_open to search for a cluster of connected relative minutiae information structures in Tq_open. It is possible to find more than one cluster. Hence, the cluster with the highest number of connected structures shall be used to find the reference minutiae for alignment. Within the biggest cluster, the matched minutiae triplets with largest area or the lowest matching error will be selected as the reference minutia.


Figure 4 Pictorial illustration of propagative minutiae structural search (PMSS), (a) enrolment template, Te_open (b) query template, Tq_open (see online version for colours)

(a) (b)

The following equation calculates the similarity of each triplet using the relative information structure specified in (5) between Te_open and Tq_open:

( )( , ) ' ' 'ij ik jl ik jl ik jlE k l d d r r normα β γ φ φ= ⋅ Δ − Δ + ⋅ Δ − Δ + ⋅ Δ − Δ (6)

1α β γ+ + = (7)

where Eij is the weighted sum of absolute error to compare the relative information between the enrolment template and query template, i and j are indexes of minutia in Te_open and Tq_open respectively, k and l are indexes of minutia neighbours of given minutia i and j in Te_open and Tq_open respectively, and α,β and γ are parameters to calculate the weighted error.

Each L contains three neighbours. In order to find whether the relative minutiae information L (inside Te_open) and L’ (inside Tq_open) are similar, we need to match all neighbours by using the following pseudo code.

Figure 5 is the simplified pseudo code to compute the similarity using relative minutiae information between Te_open and Tq_open. At the initial stage, we can start from the first index in Tq_open to find the first matched local feature vectors between Te_open and Tq_open. Once a matched pair is found, the PMSS process can be started.

For example as shown in Figure 2, if L2 in the enrolment template is found to be matched with L’1 in query template using the above pseudo code, the neighbours (L’3 and L’4) connected to L’1 in query template can be considered as next starting point for matching. Therefore, the PMSS tries to search for another similar relative information structures in Te_open which are similar to L’3 and L’4 in the query template. PMSS will consider the lower index first and start from L’3. If L’3 is found to be similar to L4, PMSS will consider the neighbours of L’3 as the next starting point and maintain this propagation search. The arrow in Figure 2 shows how the matching propagates from

44 T-P. Chen et al.

minutia to minutia. In summary for the propagation matching in Figure 2, L’1 L’3 L’7 from the query template Tq_open are found to be matched with L2 L4 L5 from enrolment template Te_open with the corresponding sequence. Once the PMSS fails to find any matched local feature vectors from the Te_open, the process will go back to L’4 to repeat the search of another sequence of matched local feature vectors until the termination criteria is met. Hence, a sub-graph of matched local feature vector in a spanning-tree like structure will be found within a cluster of minutiae.

Figure 5 Simplified pseudo code to compute the similarity using relative information

function L_MATCH ( Lie_open, Lj

q_open) {initialisation} NumMatch ← ∅ {to store the number of matched pairs} for k ← 1 to 3 do for l ← 1 to 3 do if Eij(k, l) < Threshold then NumMatch ← NumMatch + 1 Endif Endfor Endfor if NumMatch >= 2 then {at least 2 match pairs are found} return 1 endif return 0

The process is stopped when one of the following termination criteria is met:

1 no more matched minutia can be found

2 the search exceeded the maximum number of propagation allowed

3 a loop is found

4 all minutiae in the query template have been inspected.

Criterion 3 is necessary to avoid dead lock. Please note that the connection of the minutiae in Te_open shown in Figure 4 is for illustration only as it is not known at the BVT. However, based on the PMSS, the relation among the local feature vectors can be estimated in the terminal. Hence, the PMSS process has to be executed in a secured terminal.

The PMSS algorithm may be able to find one or more cluster of matched minutiae between the enrolment template and the query template, provided that both templates are from the same finger. If the number of matched clusters is more than one, only the cluster with highest number of matched minutiae will be selected. Within the selected cluster, the matched relative minutiae structure with minimum matching error will be chosen to be the reference minutiae for alignment. As the whole process of the PMSS is executed in the terminal without the actual template Te, the terminal can only perform alignment for


the query template Tq to T’q. For the alignment of the enrolment template, the terminal encodes the alignment information A, which comprises the indexes of the selected relative information structures of the Te_open and the selected minutiae coordinates of query template. Finally, the alignment information A is sent to the smartcard to perform on-card template alignment for the enrolment template as well as the final on-card comparison.

4.4 Alignment and final on-card comparison

Since the reference pairs of minutiae are determined using PMSS in the previous section, the alignment of the query template can be computed at the terminal side. Equations (8) and (9) compute the alignment template of the query template and the enrolment template respectively.

( ) ( )2 2_ _

_1q

_( ) tan

queryi i base q i base q

i base qqueryi

i base q

queryi i

r x x y y

y yi

x xα

ρ ϕ

−

⎡ ⎤= − + −⎢ ⎥

⎢ ⎥⎛ ⎞−⎢ ⎥

′ = = ⎜ ⎟⎢ ⎥⎜ ⎟−⎢ ⎥⎝ ⎠⎢ ⎥

=⎢ ⎥⎣ ⎦

T (8)

( ) ( )2 2_ _

_1e

_( ) tan

enroli j base e j base e

j base eenroli

j base e

enroli j

r x x y y

y yj

x xα

ρ ϕ θ

−

⎡ ⎤= − + −⎢ ⎥

⎢ ⎥⎛ ⎞−⎢ ⎥

′ = = ⎜ ⎟⎢ ⎥⎜ ⎟−⎢ ⎥⎝ ⎠⎢ ⎥

= −⎢ ⎥⎣ ⎦

T (9)

where (xbase_q, ybase_q) and (xbase_e, ybase_e) are the Cartesian coordinates of the reference base minutiae of query template and enrolment template respectively, i and j are indexes of minutiae in the query template and the enrolment template respectively, and θ is the orientation difference between two templates. T’q and T’e are aligned query template and aligned enrolment template respectively. The alignment process is to convert the Cartesian coordinate to the Polar coordinate based on the reference base minutiae as the centre to perform final matching. Three types of information are calculated: the distance between the reference base and given minutia, the gradient of the base minutia with given minutiae and the aligned ridge direction denoted as r, α and ρ respectively as mentioned in equations (8) and (9). The computation of T’q using equation (8) is performed at the terminal side. The computed T’q is subsequently sent to the smartcard to perform the final matching. The orientation difference θ is not known at the terminal as there is no Cartesian coordinates of the enrolment template in the terminal to calculate the orientation difference between the templates. Hence, the coordinates of those matched relative information structures from the query template Tq and the indexes of the matched relative information structures in Te_open computed by PMSS are encoded as alignment info A, which is then sent to the smartcard to compute the orientation difference – θ .

46 T-P. Chen et al.

Once the orientation difference – θ is computed, equation (9) can be used to compute the aligned enrolment template T’e inside the smartcard.

Once all the aligned templates are computed, the smartcard can compute the matching score S to decide the similarity between the enrolment and the query templates. The matching process is described using the simplified pseudo code in Figure 6.

Figure 6 Simplified pseudo code for final matching process in the smartcard

function MINUTIAE_MATCH (T’e, T’q) {initialisation} S ← ∅ {to store the number of matched pairs} for i ← 1 to n do min_error ← max_int {Max. value of integer} for j ← 1 to m do error ← m_match(T’e(i), T’q(j)) if error < min_error then min_error ← error endif endfor if min_error < Tolerance S ← S + 1 endif endfor S = S * 100 /( (n + m) / 2) return S

where n and m are the number of minutiae in the enrolment template and the query template respectively. This function scans all the minutiae inside the transformed templates to search for the number of matched minutiae of which the matching errors are lower than a predefined Tolerance. The m_match(.) function can be computed using the following equation:

1 2 3enrol query enrol query enrol query

i j i j i jerror w r r w wα α ρ ρ= − + − + − (10)

Once the number of matched minutiae is computed, the matching score S can be computed by dividing the average number of minutiae between the enrolment template and the query template. Finally, the matching score can be used to compare with the internal predefined security threshold as per ISO/IEC 24787 clause 7.1.3.2 to decide whether the query template is from genuine user or imposter for subsequent transaction.

5 Performance

We have implemented a prototype on Java card using Java card Development Kit (JDK) 2.1. We have also implemented another prototype to be executed on PC for benchmarking. The Java card prototype is used to test the actual matching speed. For benchmarking databases, we used FVC 2000 DB1, DB2 and DB3, and FVC2002 DB1 and DB2, which can be obtained from fingerprint verification competition (FVC) website


(http://bias.csr.unibo.it/fvc2000/ and http://bias.csr.unibo.it/fvc2002/). For minutiae extraction, we used the open source Fingerprint SDK from National Institute of Standards Technology (NIST) to generate minutiae template (http://fingerprint.nist.gov/NFIS). The maximum number of minutiae is set to 80 for template generation. If the total number of detected minutiae is more than 80, those poor quality minutiae will be rejected. The template will be converted to compact card format. Based on the compact card format, we wrote a generator to compute the relative minutiae information template based on the extracted minutiae template and the thinned fingerprint image from the NIST extractor. The relative minutiae information template will be saved in the proprietary section of the ISO template. NIST extractor is used because it can generate the template in ISO/IEC 19794-2 compact size format. Although the performance of this extractor is not good with quite a number of false minutiae, it can serve as a basis to perform performance benchmarking.

All FVC databases contain 100 fingers and each finger has eight impressions. Based on our benchmarking, we found that the average equal error rate (EER) using our new scheme for all databases is approximately 4.3%. Figures 7 to 11 show the receiver operating curves of the respective databases. The EERs of respective database are shown in Table 3. In terms of accuracy, it is comparable to the implementation by Jiang and Yau (2000), but their implementation was using a better minutiae extractor (by using more sophisticated ridge extraction method and segmentation method), and the matching engine was executed on a PC with floating point unit that can accelerate certain computations such as square root with high precision especially when computing the alignment process in Section 4.4. Our implementation is implemented using 8/16-bit integer arithmetic. All transcendental operations in our prototype are implemented using look up table method with precision of 4 decimal places (fixed-point).

The Java card executes the on-card matching applet using the built-in Java virtual machine (JVM) which is slower than the native implementation. As the existing smartcard and Java card do not support the work-sharing mechanism proposed in ISO/IEC 24787, extra Java functions were implemented to simulate the work-sharing mechanism in the Java card that incurred extra processing time. The average matching time on the Sharp Java card (16-bit CPU, 25MHz) is approximately 1.2 seconds and the Oberthur card (8-bit Java card, 30 MHz) is approximately 2.5 seconds.

Figure 7 FVC 2000 DB1 receiver operating curve (see online version for colours)

48 T-P. Chen et al.






Table 3 EERs of respective databases

Database Equal error rate (%)

FVC 2000 DB1 4.25

FVC 2000 DB2 4.15

FVC 2000 DB3 8.09

FVC 2002 DB1 2.65

FVC 2002 DB2 2.32

Table 4 shows the accuracy and match time of the top four participants of FVC2000 DB1, DB2 and DB3. The details of respective participants can be found in the FVC2000 report. All participants were using proprietary minutiae template generator whereas our algorithm used NIST minutiae generator to generate template in compact size format. Our algorithm can achieve average EER 5.49% which is ranked number 4 among the others. The match time is listed in the table for reference. Our implementation can achieve 1.2 s average match time using the Sharp Java card. For FVC2002, our algorithm is ranked at 17 and 14 with DB1 and DB2 respectively. In DB1, the average match time of the best algorithm is 2.47 s with EER = 0.1% and the match time of the fastest algorithm is 0.2 s with EER = 2.15% at rank 14. Hence, the accuracy and the match time of our implementation are comparable to the PC version of fingerprint matcher. Table 4 Performance comparison among top participants of FVC2000

Average equal error rate (%) Match time

Sag1 1.64 1.37 s Sag2 2.00 1.25 s Cspn 5.24 0.23 s Our algorithm 5.49 1.2 s (Sharp Java card) Cetp 5.99 1.18 s

50 T-P. Chen et al.

6 Conclusions

The ISO/IEC 24787 on-card biometric comparison standard published on 15th December 2010 provides a comprehensive specification for developer to implement biometrics on smartcard which allows interoperability with different vendors. This standard contains 6 normative references that cover three main areas including smartcard operation, biometric sample quality and data structure for template storage. There are certain constraints in the data format such that low-cost smartcard may not have sufficient computing power to perform biometric comparison within an ideal period of verification time. Hence, the ISO/IEC 24787 specifies a work-sharing mechanism which allows the external biometric verification to work with the smartcard to process biometric comparison. To illustrate this useful mechanism to enable the implementation of biometric comparison on low-cost smartcard, we proposed a novel approach for on-card fingerprint comparison with work-sharing mechanism based on the ISO/IEC 24787. The purposed algorithm splits the comparison process into two separate sections: off-card alignment and on-card comparison of minutiae templates. The enrolment template is separated into an open portion and a secure portion. The open portion consists of relative minutiae information that can only be used to find the optimal set of reference positions between the enrolment template and the query template for subsequent alignment process. It is difficult to use relative minutiae information alone to reverse engineer the original minutiae template. The secure portion of biometric reference data which consists of the Cartesian coordinates, ridges directions and types of minutiae, as specified in ISO/IEC 19794-2 will never be sent out to any external device and only be used for on-card comparison process. A new PMSS scheme is presented to estimate the optimal reference minutiae by searching a cluster of connected matched relative minutiae information structures between the query template Tq_open and the enrolment template Te_open. Finally, the algorithm for on-card alignment of the enrolment template and the on-card template comparison are also introduced. A prototype of the proposed algorithm was implemented on Java card. The average EER of the sample implementation is approximately 4.3% using FVC 2000 and FVC 2002 databases. The average matching time with 8-bit Java card running at 30 MHz and 16-bit Java card running at 25 MHz are approximately 2.5 s and 1.2 s respectively. In conclusion, by using the work-sharing scheme, it is possible to implement an on-card biometric comparison with low-cost smartcard that has limited processing power. Even though the current open portion is still in proprietary format, it is possible to purpose those features mentioned in this paper to be part of the mandatory standard to SC17 WG11 in the future for work-sharing fingerprint matching. Moreover, the work-sharing mechanism allows the BVT to assist the smartcard to process calculation. It is possible to use additional features to further enhance the performance of fingerprint matching with the aid of this mechanism for on-card fingerprint comparison.

References Allah, M.M.A. (2005) ‘A fast and memory efficient approach for fingerprint authentication

system’, IEEE Conf. on Advanced Video and Signal Based Surveillance, pp.259–263. Bistarelli, S., Santini, F. and Vaccarelli, A. (2005) ‘An asymmetric fingerprint matching algorithm

for Java CardTM’, Audio- and Video-Based Biometric Person Authentication LNCS, Vol. 3546, pp.279–288.


Cao, K., Yang, X., Tian, J., Zhang, Y., Li, P. and Tao, X. (2009) ‘Fingerprint matching based on neighboring information and penalized logistic regression’, Proc. Int. Conf. Biometrics, Alghero, Italy, June, pp.617–626.

Chung, Y., Kim, K., Kim, M., Pan, S. and Park, N. (2005) ‘A hardware implementation for fingerprint retrieval’, Knowledge-Based Intelligent Information and Engineering Systems LNCS, Springer, New York, USA, pp.374–380.

Feng, J. (2008) ‘Combining minutiae descriptors for fingerprint matching’, Pattern Recognition, Vol. 41, No. 1, pp.342–352.

Feng, J., Ouyang, Z. and Cai, A. (2006) ‘Fingerprint matching using ridges’, Pattern Recognition, Vol. 39, No. 11, pp.2131–2140.

Germain, R.S., Califano, A. and Colville, S. (1997) ‘Fingerprint matching using transformation parameter clustering’, IEEE Computational Science & Engineering, Vol. 4, No. 4, pp.42–49.

He, Y., Tian, J., Li, L., Chen, H. and Yang, X. (2006) ‘Fingerprint matching based on global comprehensive similarity’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 28, No. 6, pp.850–862.

Huvanandana, S., Kim, C. and Hwang, J.N. (2000) ‘Reliable and fast fingerprint identification for security applications’, Proc. Int. Conf. on Image Processing, Vol. 2, pp.503–506.

ISO/IEC 19785-1 (2006) Information Technology – Common Biometric Exchange Formats Framework – Part 1: Data Element Specification, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 19785-3 (2007) Information Technology – Common Biometric Exchange Formats Framework – Part 3: Patron Format Specifications, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 19794-2 (2005) Information Technology – Biometric Data Interchange Format – Finger Minutiae Data, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 24787 (2010) Information Technology – Identification Cards – On-card Biometric Comparison, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 29794-1 (2009) Information Technology – Biometric Sample Quality – Part 1: Framework, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 7816-11 (2006) Identification Cards – Integrated Circuit Cards – Part 11: Personal Verification through Biometric Methods, International Organization for Standardization and International Electrotechnical Commission.

ISO/IEC 7816-4 (2005) Identification Cards – Integrated Circuit Cards – Part 4: Organization, Security and Commands for Interchange, International Organization for Standardization and International Electrotechnical Commission.

Jain, A.K., Hong, L. and Bolle, R. (1997) ‘On-line fingerprint verification’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 19, No. 4, pp.302–313.

Jiang, X. and Yau, W.Y. (2000) ‘Fingerprint minutiae matching based on the local and global structures’, Proc. Int. Conf. Pattern Recog., Vol. 2, pp.1042–1045.

Krivec, V., Birchhauer, J.A., Marius, W. and Bischof, H. (2003) ‘A hybrid fingerprint matcher in memory constrained environments’, Proceedings of the 3rd International Symposium on Image and Signal Processing and Analysis, pp.617–620.

Maltoni, D., Maio, D., Jain, A.K. and Prabhakar, S. (2003) Handbook of Fingerprint Recognition, Springer, New York, USA.

Mimura, M., Ishida, S. and Seto, Y. (2002) ‘Fingerprint verification system on smart card’, International Conference on Consumer Electronics, pp.182–183.

52 T-P. Chen et al.

Pan, S.B., Gil, Y.H., Moon, D., Chung, Y. and Park, C.H. (2003) ‘A memory-efficient fingerprint verification algorithm using a multi-resolution accumulator array’, ETRI Journal, Vol. 25, No. 3, pp.179–186.

Rikin, A.S., Li, D., Isshiki, T. and Kunied, A.H. (2005) ‘A fingerprint matching using minutia ridge shape for low cost match-on-card systems’, IEICE Trans. Fundamentals, Vol. E88–A, No. 5, pp.1305–1312.

Sanchez-Reillo, R. and Sanchez-Avila, C. (2002) ‘Fingerprint verification using smart cards for access control systems’, IEEE Aerospace and Electronics Systems Magazine, Vol. 17, No. 9, pp.12–17.

Sanchez-Reillo, R., Mengibar-Pozo, L. and Sanchez-Avila, C. (2003) ‘Microprocessor smart cards with fingerprint user authentication’, IEEE Aerospace and Electronics Systems, Magazine, Vol. 18, No. 3, pp.22–24.

X.680 (2002) Information Technology – Abstract Syntax Notation One (ASN.1): Specification of Basic Notation, International Telecommunication Union – Telecommunication.

iso/iec standards for on-card biometric comparison · iso/iec standards for on-card biometric...

Documents