1、 INCITS/ISO/IEC 19795-5:2011 2011 ISO/IEC 19795-5:2011 Information technology Biometric performance testing and reporting Part 5: Access control scenario and grading scheme INCITS/ISO/IEC 19795-5:2011 2011 PDF disclaimer This PDF file may contain embedded typefaces. In accordance with Adobes licensi
2、ng policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, parties accept therein the responsibility of not infringing Adobes licensing policy. The ISO C
3、entral Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters were optimized for printing. Every care has been
4、taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. Adopted by INCITS (InterNational Committee for Information Technology Standards) as an American Nati
5、onal Standard. Date of ANSI Approval: 6/28/2011 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2011 by Information Technology Industry Council (ITI). All rights reserved. These materials are subject to copyright claims of International Sta
6、ndardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Council (ITI). Not for resale. No part of this publication may be reproduced in any form, including an electronic retrieval system, with
7、out the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1250 Eye Street NW, Washington, DC 20005. Printed in the United States of America ii ITIC 2011 All rights reserved ISO/IEC 19795-5:2011(E) ISO/IEC 2011 All rights reserved iiiContents Page F
8、oreword .v Introductionvi 1 Scope1 2 Conformance .2 3 Normative references2 4 Terms and definitions .2 5 Definition of testing scenario.3 5.1 Overview.3 5.2 Relationship of biometric system / subsystem to access control system3 5.3 Evaluation metrics overview 5 5.4 Evaluation approach .5 5.4.1 Tests
9、 .5 5.4.2 Universality of the test6 5.4.3 Levels of effort and decision policies .6 5.4.4 Controlled Indoor Environment .6 5.5 Crew characteristics and management.7 5.5.1 Crew demographics 7 5.5.2 Crew size 8 5.5.3 Test crew selection .8 5.5.4 Test crew training9 5.5.5 Operator - crew member interac
10、tion .9 5.5.6 Habituation.9 5.6 Privacy9 5.6.1 General .9 5.6.2 Crew identity protection .9 5.6.3 Data protection 10 5.6.4 Proprietary information.10 6 Testing approach and conduct 10 6.1 Planning .10 6.1.1 General .10 6.1.2 Test objectives.10 6.1.3 Inputs to and outputs from the test process10 6.1.
11、4 Concept of operations 10 6.1.5 Adherence to native system operations .11 6.2 General test approach.11 6.2.1 General .11 6.2.2 Pre-test activities.12 6.2.3 System operability verification 14 6.2.4 Data collection .14 6.2.5 Problem reporting and tracking.15 6.2.6 Post-test briefing.16 6.3 Testing me
12、thodology 16 6.3.1 Introduction16 6.3.2 Enrolment transactions and results generation 17 6.3.3 Verification attempts, transactions, and results generation.17 6.3.4 Enrolment and verification temporal separation .18 6.3.5 Impostor tests20 6.4 Errors and exception cases .20 6.5 Incremental performance
13、 evaluations.21 ISO/IEC 19795-5:2011(E) iv ISO/IEC 2011 All rights reserved7 Grading and reporting 21 7.1 Grading 21 7.1.1 Data analysis. 21 7.1.2 Using statistical analysis methods. 21 7.1.3 Performance measures 21 7.1.4 Grading of matching performance illustration 25 7.1.5 Uses (of grading) . 25 7
14、.2 Documentation requirements and control . 26 7.2.1 General. 26 7.2.2 Test control . 26 7.3 Reporting performance results . 27 7.3.1 Reporting requirements . 27 7.3.2 Report structure 28 Annex A (informative) Grading information 29 A.1 Equivalence of tests . 29 A.2 Comparison of test results 29 A.3
15、 Grading values for enrolment performance. 29 A.4 Grading values for matching performance 30 A.5 Grading illustration shown in Figure A.1 . 30 A.6 Grading values for transaction time performance 31 A.7 Defining system requirements as in Table 7 . 31 Annex B (normative) Statistical methods for estima
16、tion of confidence bounds graded test metrics 33 B.1 Correlated binary method 33 B.2 Beta distribution method . 34 B.3 Z-statistic. 35 Bibliography. 36 ISO/IEC 19795-5:2011(E) ISO/IEC 2011 All rights reserved vForeword ISO (the International Organization for Standardization) and IEC (the Internation
17、al Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields o
18、f technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint
19、 technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee
20、are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not
21、 be held responsible for identifying any or all such patent rights. ISO/IEC 19795-5 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics. ISO/IEC 19795 consists of the following parts, under the general title Information technology Biometric
22、 performance testing and reporting: Part 1: Principles and framework Part 2: Testing methodologies for technology and scenario evaluation Part 3: Modality-specific testing Technical report Part 4: Interoperability performance testing Part 5: Access control scenario and grading scheme Part 6: Testing
23、 methodologies for operational evaluation Part 7: Testing of on-card biometric comparison algorithms ISO/IEC 19795-5:2011(E) vi ISO/IEC 2011 All rights reservedIntroduction This part of ISO/IEC 19795 is concerned solely with the scientific “technical performance testing” of biometric systems and sub
24、systems to be used for access control. Technical performance testing seeks to determine error rates and transaction times with the goal of understanding and predicting the real-world error and transaction times of a biometric system. The error rates include false accept rate, and false reject rate,
25、as well as failure to enrol (FTE) and failure to acquire (FTA) rates across the test population. These measures are generally applicable to all access control systems that contain a biometric verification subsystem. This part of ISO/IEC 19795 defines a testing framework with the following fundamenta
26、l aspects. This part of ISO/IEC 19795 was conceived to be a framework for a general- or multi-purpose test: “one size fits many (but not all)”. The focus is limited to access control applications. The framework is suitable as both a requ irements statement and an evaluation report. The general-purpo
27、se nature of th is part of ISO/IEC 19795 is centred on the common access control application requirements, and acknowledges the fact that this framework will not be suitable for specialized applications (very high levels of protection, specialized user populations like the elderly, students, etc.).
28、Specialized applications will warrant specialized testing processes. The perceived benefit of the general- or multi-purpos e test is economy. The supplier can submit to one testing process, and many potential customers can utilize the results, interpreting the suitability of the device (based on the
29、 results) for their application. This testing framework assigns grades representing the tested level of performance, and these grades include a statistical confidence taking the conservative approach, that is, the performance of the system is at least as good as the grade indicated (at the 90% confi
30、dence level). Using the grading scheme to specify a required performance level of a system needs to take into account this conservative approach. It is acknowledged that technical performance testing is only one form of biometric testing. Other types of testing not considered in this part of ISO/IEC
31、 19795 include the following: reliability, availability and maintainability; security, including vulnerability; human factors, including user acceptance; environmental; safety; cost/benefit; privacy regulation compliance. Methods and philosophies for these other types of tests are currently being co
32、nsidered internationally by a broad range of groups. The purpose of this part of ISO/IEC 19795 is to capture the current understanding by the biometrics community of requirements and best scientific practices for conducting performance testing towards the end of providing consistent, structured eval
33、uations of biometric systems intended for use in access control applications. The framework defined in this part of ISO/IEC 19795 has utility as a method for defining user requirements, for specifying the extent of performance evaluation, for conducting and for reporting. INTERNATIONAL STANDARD ISO/
34、IEC 19795-5:2011(E) ISO/IEC 2011 All rights reserved 1Information technology Biometric performance testing and reporting Part 5: Access control scenario and grading scheme 1 Scope This part of ISO/IEC 19795: defines a common biometric access control scenar io for use in scenario evaluation of biomet
35、ric verification systems; provides a grading scheme for expressing quantitat ive biometric system requirements and performance levels; provides a common basis for conducting scenario eval uations to demonstrate that specified performance grades are being achieved which is adaptable to particular tes
36、ting facilities and to specific biometric systems. This part of ISO/IEC 19795 is applicable to performance testing of biometric systems without detailed knowledge of the comparison algorithms or of the underlying distribution of biometric characteristics in the population of interest. The minimum fa
37、lse accept rate (FAR) tested by this part of ISO/IEC 19795 is 0.1%. If a lower FAR is required, customized testing (outside the scope of this part of ISO/IEC 19795) might be appropriate, and needs to be fully compliant with ISO/IEC 19795-2. This part of ISO/IEC 19795 addresses testing a biometric sy
38、stem for physical access control, and the suitability of the testing for logical access devices needs to be determined on a case-by-case basis. Not within the scope of this part of ISO/IEC 19795 is the measurement of error and throughput rates for people deliberately trying to circumvent correct rec
39、ognition by the biometric system (i.e. active impostors). In addition, this part of ISO/IEC 19795 does not assess the following: reliability, availability and maintainability; security, including vulnerability; human factors, including user acceptance; environmental impacts; safety; cost/benefit/sui
40、tability; privacy regulation compliance. These assessments are the responsibility of the procuring authority. ISO/IEC 19795-5:2011(E) 2 ISO/IEC 2011 All rights reserved2 Conformance A test conforms to this part of ISO/IEC 19795 if the scenario used (including test crew demographics, environmental co
41、ntrols, time separation between enrolment and revisit, numbers of attempts and transactions), test conduct, and test reporting all conform to the mandatory requirements in Clauses 5 through 7. 3 Normative references The following referenced documents are indispensable for the application of this doc
42、ument. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO/IEC 19795-1:2006, Information technology Biometric performance testing and reporting Part 1: Principles and framework ISO/IEC 197
43、95-2:2007, Information technology Biometric performance testing and reporting Part 2: Testing methodologies for technology and scenario evaluation ISO/IEC TR 19795-3, Information technology Biometric performance testing and reporting Part 3: Modality-specific testing 4 Terms and definitions For the
44、purposes of this document, the terms and definitions given in ISO/IEC 19795-1 and the following apply. 4.1 access control system ACS entire electro-mechanical suite that performs the granting or denying of access at controlled entry points of a facility 4.2 biometric subsystem portion of a biometric
45、 system that is present at each access entry point, including the biometric sensor or sampling subsystem 4.3 grade levels measurement associated with the quantified levels of biometric subsystem performance NOTE Grade levels are defined, ranging from 0 to 3, or 0 to 6. It is possible that additional
46、 grade levels above these values will be defined at a future date. 4.4 FAR level scale for the relative level of resistance to false accepts in a form associated with three specific false accept rate (FAR) values 4.5 transaction time time required for the biometric system portion of an access contro
47、l transaction NOTE Transaction time is measured in seconds. ISO/IEC 19795-5:2011(E) ISO/IEC 2011 All rights reserved 35 Definition of testing scenario 5.1 Overview The goal of testing and evaluating biometric access control systems against the standard set of criteria documented in this part of ISO/
48、IEC 19795 is to ensure that the technical performance of every biometric access control system is evaluated fairly, accurately and equivalently. Testing shall be performed in a consistent, unbiased manner under conditions that are well understood and documented. Test controls shall be applied to ens
49、ure reproducible test results to the most practical extent possible (considering the involvement of human crew members). To accomplish this, every candidate biometric access control system shall be tested in accordance with the same general test protocol. The procedures to be used shall be based upon a “framework” consisting of specific metrics extracted from biometric system
