1、American National StandardDeveloped byfor Information Technology Biometric Performance Testingand Reporting Part 2: Technology Testingand ReportingANSI INCITS 409.2-2005ANSIINCITS 409.2-2005Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reprod
2、uction or networking permitted without license from IHS-,-,-Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-ANSIINCITS 409.2-2005American National Standardfor Information Technol
3、ogy Biometric Performance Testingand Reporting Part 2: Technology Testing and ReportingSecretariatInformation Technology Industry CouncilApproved October 25, 2005American National Standards Institute, Inc.AbstractThis standard specifies procedures for conducting offline tests of the performance of b
4、iometric technologies.Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Approval of an American National Standard requires review by ANSI that therequirements for due process, cons
5、ensus, and other criteria for approval havebeen met by the standards developer.Consensus is established when, in the judgement of the ANSI Board ofStandards Review, substantial agreement has been reached by directly andmaterially affected interests. Substantial agreement means much more thana simple
6、 majority, but not necessarily unanimity. Consensus requires that allviews and objections be considered, and that a concerted effort be madetowards their resolution.The use of American National Standards is completely voluntary; theirexistence does not in any respect preclude anyone, whether he has
7、approvedthe standards or not, from manufacturing, marketing, purchasing, or usingproducts, processes, or procedures not conforming to the standards.The American National Standards Institute does not develop standards andwill in no circumstances give an interpretation of any American NationalStandard
8、. Moreover, no person shall have the right or authority to issue aninterpretation of an American National Standard in the name of the AmericanNational Standards Institute. Requests for interpretations should beaddressed to the secretariat or sponsor whose name appears on the titlepage of this standa
9、rd.CAUTION NOTICE: This American National Standard may be revised orwithdrawn at any time. The procedures of the American National StandardsInstitute require that action be taken periodically to reaffirm, revise, orwithdraw this standard. Purchasers of American National Standards mayreceive current
10、information on all standards by calling or writing the AmericanNational Standards Institute.American National StandardPublished byAmerican National Standards Institute, Inc.25 West 43rd Street, New York, NY 10036Copyright 2005 by Information Technology Industry Council (ITI)All rights reserved.No pa
11、rt of this publication may be reproduced in anyform, in an electronic retrieval system or otherwise,without prior written permission of ITI, 1250 Eye Street NW, Washington, DC 20005. Printed in the United States of AmericaCAUTION: The developers of this standard have requested that holders of patent
12、s that may berequired for the implementation of the standard disclose such patents to the publisher. However,neither the developers nor the publisher have undertaken a patent search in order to identifywhich, if any, patents may apply to this standard. As of the date of publication of this standarda
13、nd following calls for the identification of patents that may be required for the implementation ofthe standard, no such claims have been made. No further patent search is conducted by the de-veloper or publisher in respect to any standard it processes. No representation is made or impliedthat licen
14、ses are not required to avoid infringement in the use of this standard.Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Copyright American National Standards Institute Provided by
15、 IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-ContentsPageForeword . iiiIntroduction vii1 Scope. 12 Conformance . 13 Normative References . 14 Terms and Definitions 25 Principles . 45.1 General Aspects 45.2 Application Realism . 45.3 D
16、ata . 55.4 Tester Passivity 65.5 Fairness . 75.6 Noninterchangeable identification and verification attempts 75.7 Acknowledgment of Models. 75.8 Significance of Results. 85.9 Sample Processing Failure 85.10 Throughput Performance. 106 SDK Tests 116.1 Mode of Operation . 116.2 Execution Environment 1
17、16.3 Test Taxonomy 126.4 Planning. 146.5 Functional Requirements. 156.6 Reporting . 176.7 Legal Issues. 186.8 Pre-test Procedures. 186.9 Verification Tests . 196.10 Identification Tests. 266.11 Specific Tests. 34Tables1 Categorization of Technology Tests 132 Test Planning Procedure. 14i3 Procedures
18、for Conducting a Simple Verification Test 204 Procedures for Conducting a Multiple Enrollee Verification Test 22Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Page5 Procedures f
19、or Conducting a Multiple Enrollee True Impostor Verification Test 236 Procedures for Conducting a Jack-Knife Based Verification Test. 257 Procedures for Conducting Yes/No Verification Test 268 Outputs of an Identification Test . 279 Identification False Non-match Rate Measures 2810 Candidate List Re
20、striction Criteria. 2911 Cumulative Match Characteristic Computation. 3012 Procedure for Simple Closed-Set Identification Test 3113 Procedure for Simple Open-Set Identification Test. 32AnnexesA Sample Test Plan . 35B Interface . 38C Client-Server Testing 42D Identification DET Characteristics 43E Bi
21、bliography 45ii.2-2005.)Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Foreword (This foreword is not part of American National Standard ANSI INCITS 409This standard specifies m
22、ethods for the offline testing of biometric technologies.INCITS (The International Committee for Information Technology Standards) is theANSI recognized Standards Development Organization for information technologywithin the United States of America. Members of INCITS are drawn from Govern-ment, Cor
23、porations, Academia and other organizations with a material interest in thework of INCITS and its Technical Committees. INCITS does not restrict membershipand attracts participants in its technical work from 13 different countries, and oper-ates under the rules of the American National Standards Ins
24、titute. In the field of Biometrics, INCITS has established the Technical Committee M1. Stan-dards developed by this Technical Committee have reached consensus throughoutthe development process and have been thoroughly reviewed through several PublicReview processes. In addition, this American Nation
25、al Standard has been approvedby the INCITS Executive Board and ANSI Board of Standards Review for Publicationas an ANSI INCITS Standard. The ANSI INCITS 409 series of standards was prepared by Technical CommitteeINCITS/M1, Biometrics, Task Group 5, Performance Testing and Reporting. TheANSI INCITS 4
26、09 series of standards consists of the following parts, under the gen-eral title Biometric Performance Testing and Reporting:Part 1: Principles and Framework (ANSI INCITS 409.1);Part 2: Technology Testing and Reporting (ANSI INCITS 409.2);Part 3: Scenario Testing and Reporting (ANSI INCITS 409.3);Pa
27、rt 4: Operational Testing and Reporting (ANSI INCITS 403.4 in development);Part 5: Framework for Testing and Evaluation of Biometric Devices for Access Control (ANSI INCITS 403.5 in development)This document contains five annexes, one normative (A) and four informative (B, C,D, and E).Requests for i
28、nterpretation, suggestions for improvement or addenda, or defect re-ports are welcome. They should be sent to InterNational Committee for InformationTechnology Standards (INCITS), ITI, 1250 Eye Street, NW, Suite 200, Washington,DC 20005.This standard was processed and approved for submittal to ANSI
29、by INCITS. Com-mittee approval of this standard does not necessarily imply that all committee mem-bers voted for its approval. At the time it approved this standard, INCITS had thefollowing members:Karen Higginbottom, ChairJennifer Garner, SecretaryOrganization Represented Name of RepresentativeAppl
30、e Computer, Inc. David Michael Electronic Industries Alliance Edward Mikoski, Jr. Henry Cuschieri (Alt.)EMC Corporation Gary Robinson Farance, Inc. Frank Farance GS1 US Frank Sharkey James Chronowski (Alt.)Mary Wilson (Alt.)iiiCopyright American National Standards Institute Provided by IHS under lic
31、ense with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Organization Represented Name of RepresentativeHewlett-Packard Company. Karen Higginbottom Steve Mills (Alt.)Scott Jameson (Alt.)IBM Corporation . Ronald F. Silletti Institute for Certification of Comput
32、er Professionals Kenneth M. Zemrowski Thomas Kurihara (Alt.)IEEE . Judith Gorman Richard Holleman (Alt.)Robert Pritchard (Alt.)Intel. Philip Wennblom Dave Thewlis (Alt.)Jesse Walker (Alt.)Lexmark International. Don Wright Dwight Lewis (Alt.)Paul Menard (Alt.)Microsoft Corporation . Isabelle Valet-Ha
33、rper Don Stanwyck (Alt.)Mike Ksar (Alt.)National Institute of Standards multisample (three views of a face, or four fingerprints) or any hybrid thereof.6. Multialgorithmic testing: If multiple, suitably encapsulated, recognizers can becombined offline testing can be used on the black-box result.Rela
34、tion of Technology, Scenario, and Operational TestingThis document describes only offline technology tests. However, if the data are ac-quired to mimic a particular scenario, then the test can be considered a scenario test.Similarly, if the data to be used in the test is itself operational data, or
35、is otherwiseclosely representative of data that will be acquired in deployment, then the test canvii-,-,-Copyright American National Standards Institute Provided by IHS under license with ANSINot for ResaleNo reproduction or networking permitted without license from IHSbe considered an operational t
36、est. In some applications, for instance when the sam-ples are to be acquired covertly, the human-system interaction is particularly simple,and a technology test becomes a reasonable approximation to the operational mode.Further, because operational testing is usually conducted on site with live subj
37、ects,there can be no certainty of the impostor identities. In such cases, either scenario ortechnology testing should be conducted to assess false match performance.Technology testing is often construed as an evaluation of a modular component thatembeds certain technologies. The offline testing meth
38、ods documented here presumethe existence of a suitable archived dataset and, as such, are capable of testing allmodules subsequent to the human-sensor interface. These may include a qualitycontrol and feedback module, (iterative phases of) image (or signal) processing (res-toration, enhancement, ana
39、lysis), image fusion (when multimodal or multisample bio-metrics are available) feature extraction and normalization, feature-level fusion,similarity score computation and fusion, and score normalization.viii-,-,-AMERICAN NATIONAL STANDARD ANSI INCITS 409.2-20051 American National Standard for Infor
40、mation Technology Biometric Performance Testing and Reporting Part 2: Technology Testing and Reporting 1 Scope This standard specifies methods for performance testing of biometric systems and devices. It constitutes a specialization of a biometric testing framework standard in that it is concerned o
41、nly with the offline use of stored (i.e., previously captured) biometric samples, and not the interaction of human subjects with a biometric sensor. The standard covers: Comparative or absolute testing of performance of biometric algorithms, components, or systems; Comparison of biometric data sets;
42、 Prediction of elements of deployed online performance; Assessment of performance available from complex data samples including repeated sample and multimodal data. This standard does not cover: All aspects of the human-sensor interaction, except as they manifest themselves in the output samples. 2
43、Conformance A test shall conform to this standard if it implements one of the entries of Table 1, conforms to all normative subclauses of clauses 5 and 6. 3 Normative References The following standards contain provisions which, through reference in this text, constitute provisions of this American N
44、ational Standard. At the time of publication, the editions indicated were valid. All standards are subject to revision, and parties to agreements based on this American National Standard are encouraged to investigate the possibility of applying the most recent editions of the standards indicated bel
45、ow. ANSI INCITS 409.1-2005, Information technology Biometric performance testing and reporting Part 1: Principles and framework ISO/IEC 17025, General requirements for the competence of testing and calibration laboratories Copyright American National Standards Institute Provided by IHS under license
46、 with ANSINot for ResaleNo reproduction or networking permitted without license from IHS-,-,-ANSI INCITS 409.2-2005 2 4 Terms and Definitions For the purposes of this document, the terms and definitions given in the remaining parts of this standard and the following apply: 4.1 candidate data element
47、 output by a biometric system or device that includes, at least, the identity of an enrollee and its respective similarity score 4.2 candidate list set of candidates produced in an identification or verification attempt NOTE 1: For an impostor transaction, the list should ideally (if no errors occur
48、) have length zero. NOTE 2: For a genuine transaction, the list should ideally (if no errors occur) have length one, containing just the identity of the user. 4.3 enrollment the process of collecting biometric samples from a person and the subsequent preparation and storage of biometric reference templates representing that persons identity 4.4 enrollment template feature data extracted from, or used to represent, a sample used in an enrollment 4.5 enrollment attempt creation and storage of an enrollment template from a sample 4.6 enrollment te
