1、 INCITS/ISO/IEC TR 19795-3-2009 Information technology Biometric performance testing and reporting Part 3: Modality-specific testing Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-
2、,-,-INCITS/ISO/IEC TR 19795-3-2009 PDF disclaimer This PDF file may contain embedded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the
3、editing. In downloading this file, parties accept therein the responsibility of not infringing Adobes licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can
4、 be found in the General Info relative to the file; the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat
5、 at the address given below. Adopted by INCITS (InterNational Committee for Information Technology Standards) as an American National Standard. Date of ANSI Approval: 12/10/2009 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2009 by Inform
6、ation Technology Industry Council (ITI). All rights reserved. These materials are subject to copyright claims of International Standardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Counci
7、l (ITI). Not for resale. No part of this publication may be reproduced in any form, including an electronic retrieval system, without the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1250 Eye Street NW, Washington, DC 20005. Printed in the Uni
8、ted States of America ii ITIC 2009 All rights reserved Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 ITIC 2009 All rights reserved iiiContents P
9、age Foreword iv Introduction.v 1 Scope1 2 Normative references1 3 Terms and definitions .1 4 Symbols and abbreviated terms 2 5 Modality-specific test design .2 5.1 Development flow2 5.2 Modality-specific factors that may impact performance (Step 1) 3 5.3 Modality-specific policies pertaining to Test
10、 Subjects (Step 2) .3 5.4 Modality-specific policies pertaining to data collection (Step 3) .5 5.5 Modality-specific policies in impostor transactions (Step 4) .7 5.6 Modality-specific reporting policies (Step 5)9 6 Evaluation of modality-specific influential factors 10 6.1 Robustness tests.10 6.2 B
11、ase example: face .11 6.3 Other examples: fingerprint, iris, vein, and voice13 7 Test design principles on “new” modalities.18 Bibliography19 Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license
12、from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 iv ITIC 2009 All rights reservedForeword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or
13、IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, go
14、vernmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives,
15、 Part 2. The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the natio
16、nal bodies casting a vote. In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report of one of the following types: type 1, when the required support cannot be obtained for the publication of an International Standard, despite repeated efforts; typ
17、e 2, when the subject is still under technical devel opment or where for any other reason there is the future but not immediate possibility of an agreement on an International Standard; type 3, when the joint technical committee has coll ected data of a different kind from that which is normally pub
18、lished as an International Standard (“state of the art”, for example). Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to be reviewe
19、d until the data they provide are considered to be no longer valid or useful. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. ISO/IEC TR 1979
20、5-3, which is a Technical Report of type 1/2/3, was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics. ISO/IEC TR 19795 consists of the following parts, under the general title Information technology Biometric performance testing and reportin
21、g: Part 1: Principles and framework Part 2: Testing methodologies for technology and scenario evaluation Part 3: Modality-specific testing Technical Report Performance and interoperability testing of data interchange formats will form the subject of a future Part 4. Copyright American National Stand
22、ards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 ITIC 2009 All rights reserved vIntroduction In biometric performance testing and reporting, careful consideration should be given t
23、o the characteristic differences of each modality (fingerprint, face, iris, etc.). These differences naturally require variations within the general methodology defined in ISO/IEC 19795-1. Since there are many modality-dependent variations, it is desirable that each modality be specified with specif
24、ic testing and reporting methods. This ensures that not only specialists of biometrics but also non-specialists can carry out reasonably accurate testing. The characteristics that affect evaluation methodology can at least be discussed from the following viewpoints: characteristics of users, includi
25、ng the definition of impostors; restrictions that come from practical situations in which that biometric modality is used in applications. Characteristics of impostors: There are two factors to consider regarding the definition of impostors: (1) multiple biometric data from one person, and (2) impos
26、tor attempts for behaviour-based modalities, such as voice or signature. For modalities in which multiple biometric data can be collected from one person, e.g. finger (ten fingerprints from one person) and iris (two iris-images from one person), a rule for permitting or prohibiting use of these data
27、 as impostor attempts needs to be clearly defined. In the case of behaviour-based modalities, testing results regarding impostor attempts false match rate (FMR) or false acceptance rate (FAR) can be influenced depending on whether (or how much) an impostor tries to imitate an authorized users behavi
28、our or not. For instance, the case in which an impostor physically traces an authorized users signature that the impostor obtained differs significantly in FMR or FAR from the case where the impostor only looks at the signature and imitates it. For these modalities, a criterion regarding impostor at
29、tempts needs to be defined. Characteristics of modality specific to applications In general, almost all modalities of biometrics are used for user authentication. However, some modalities are expected to be used in different classes of applications; for example, face-based identification is widely u
30、sed in surveillance applications. While we can expect a users cooperation in the former, we cannot expect it in the latter case. Thus variation of testing methodologies needs to be considered depending on the way the modality is used in real applications. These restrictions can be divided into two c
31、lassifications: factors relating to users, such as facial expression s that affect the countenance of the face, wearing eye-glasses or contact lenses for the iris; factors relating to external environments that are uncontrollable by the algorithm or system, such as illumination change for face or ba
32、ckground noise for voice. These factors naturally affect the performance, and the types and number of factors are different in each modality. These modality-dependent variations need to be considered in testing and reporting. In addition, a concept of robustness testing needs to be introduced to eva
33、luate the sensitivity or robustness of the technology toward environmental factors, in case the variation of the factors strongly influences the observed performance. Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking p
34、ermitted without license from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 vi ITIC 2009 All rights reservedThis part of ISO/IEC 19795 is intended to describe the variations in methodologies relating to these modality-dependent characteristics. It presents and defines methods for determining, given a speci
35、fic biometric modality, how to develop a technical performance test. This part of ISO/IEC 19795 has been developed based on six technical reports 1-6, which are the outputs of standardization activities in Japan for biometric testing and reporting, and have been published by the Japanese Standards A
36、ssociation as JIS-TRs (Japanese Industrial Standards Technical Reports). They have been prepared through discussions by experts of respective modalities, and have extensive considerations particular to different procedures specific to each modality. They are intended to define detailed procedures fo
37、r testing, including specifications of a test database and how to collect data. These documents can be used as reference when designing specific evaluation procedures. Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking
38、permitted without license from IHS-,-,-TECHNICAL REPORT INCITS/ISO/IEC TR 19795-3-2009 ITIC 2009 All rights reserved 1Information technology Biometric performance testing and reporting Part 3: Modality-specific testing 1 Scope In biometric performance testing and reporting, careful consideration nee
39、ds to be given to the characteristic differences of each modality (fingerprint, face, iris, etc.). These differences naturally require variations within the general methodology defined in ISO/IEC 19795-1. This part of ISO/IEC 19795 describes the methodologies relating to these modality-dependent var
40、iations. It presents and defines methods for determining, given a specific biometric modality, how to develop a technical performance test. 2 Normative references The following referenced documents are indispensable for the application of this document. For dated references, only the edition cited a
41、pplies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO/IEC 19795-1:2006, Information technology Biometric performance testing and reporting Part 1: Principles and framework 3 Terms and definitions For the purposes of this document, the t
42、erms and definitions given in ISO/IEC 19795-1:2006 and the following apply. 3.1 influencing factors factors that have influence on performance 3.2 robustness test test to evaluate how much a certain influential factor affects biometric performance 3.3 active forgery attempt impostor attempt in which
43、 an individual tries to match the stored template of a different individual by presenting a simulated or reproduced biometric sample, or by intentionally modifying his/her own biometric characteristics 3.4 forgery type type of method of impostor forgery attempts Copyright American National Standards
44、 Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 2 ITIC 2009 All rights reserved3.5 forgery level difficulty level of impostor forgery attempts 4 Symbols and abbreviated terms ROC rece
45、iver operating characteristic CMR cumulative matching rate CMC cumulative match characteristic FNMR false non-match rate FMR false match rate 5 Modality-specific test design 5.1 Development flow When designing performance evaluation tests, the following test planning sequence should be used to ensur
46、e that the primary modality-dependent factors are accounted for (see Figure 1): Step 1: Identify and analyze “influencing factors“ that may impact performance Step 2: Consider and establish policy on test subjects Step 3: Consider and establish policy on data collection Step 4: Consider and establis
47、h policy on impostor transactions Step 5: Consider and establish policy on performance reporting test subjects data collection impostor transactions performance reporting Consider and establish policy Identify and analyze “influencing factors“ Figure 1 Development flow This sequence assumes that the
48、 modality and test type (i.e. technology / scenario / operational) have been defined. Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS/ISO/IEC TR 19795-3-2009 ITIC 2009 All rights reserved 35.2 Modality-specific factors that may impact performance (Step 1) Factors that influence performance (“influencing factors”) should be identified and anal
