1、American National StandardDeveloped byfor Information Technology Biometric Performance Testingand Reporting Part 5: Framework for Testingand Evaluation of Biometric System(s)for Access ControlINCITS 409.5-2011INCITS 409.5-2011INCITS 409.5-2011American National Standardfor Information Technology Biom
2、etric Performance Testingand Reporting Part 5: Framework for Testing andEvaluation of Biometric System(s)for Access ControlSecretariatInformation Technology Industry CouncilApproved December 5, 2011American National Standards Institute, Inc.AbstractThis standard specifies a framework for testing and
3、 reporting of biometric system(s) used in ap-plications supporting access control. It specifies the environment that testing will be performedin, the evaluation metrics, demographic controls, and the means by which testing will be per-formed and how the graded results will be reported. The intent of
4、 this testing framework is to pro-vide a “one size fits many” approach to re-usable test results, centered on typical access controlrequirementsApproval of an American National Standard requires review by ANSI that therequirements for due process, consensus, and other criteria for approval havebeen
5、met by the standards developer.Consensus is established when, in the judgement of the ANSI Board ofStandards Review, substantial agreement has been reached by directly andmaterially affected interests. Substantial agreement means much more thana simple majority, but not necessarily unanimity. Consen
6、sus requires that allviews and objections be considered, and that a concerted effort be madetowards their resolution.The use of American National Standards is completely voluntary; theirexistence does not in any respect preclude anyone, whether he has approvedthe standards or not, from manufacturing
7、, marketing, purchasing, or usingproducts, processes, or procedures not conforming to the standards.The American National Standards Institute does not develop standards andwill in no circumstances give an interpretation of any American NationalStandard. Moreover, no person shall have the right or au
8、thority to issue aninterpretation of an American National Standard in the name of the AmericanNational Standards Institute. Requests for interpretations should beaddressed to the secretariat or sponsor whose name appears on the titlepage of this standard.CAUTION NOTICE: This American National Standa
9、rd may be revised orwithdrawn at any time. The procedures of the American National StandardsInstitute require that action be taken periodically to reaffirm, revise, orwithdraw this standard. Purchasers of American National Standards mayreceive current information on all standards by calling or writi
10、ng the AmericanNational Standards Institute.American National StandardPublished byAmerican National Standards Institute, Inc.25 West 43rd Street, New York, NY 10036Copyright 2011 by Information Technology Industry Council (ITI)All rights reserved.No part of this publication may be reproduced in anyf
11、orm, in an electronic retrieval system or otherwise,without prior written permission of ITI, 1101 K Street NW, Suite 610, Washington, DC 20005. Printed in the United States of AmericaCAUTION: The developers of this standard have requested that holders of patents that may berequired for the implement
12、ation of the standard disclose such patents to the publisher. However,neither the developers nor the publisher have undertaken a patent search in order to identifywhich, if any, patents may apply to this standard. As of the date of publication of this standardand following calls for the identificati
13、on of patents that may be required for the implementation ofthe standard, no such claims have been made. No further patent search is conducted by the de-veloper or publisher in respect to any standard it processes. No representation is made or impliedthat licenses are not required to avoid infringem
14、ent in the use of this standard.Contents i Page 1. Scope4 2. Normative references4 3. Conformance.4 4. Terms and Definitions .5 5. Test Methodology7 5.1. Overview 7 5.2. Evaluation metrics and grades.7 5.2.1. Overview.7 5.2.2. FAR levels and comparison error rates 8 5.2.3. Failure-to-enroll rate .8
15、5.2.4. Transaction time .9 5.2.5. Multi-biometric system(s)10 5.2.6. Evaluation metrics as a summary.10 5.3. Methodology.11 5.3.1. Introduction.11 5.3.2. Test processes .11 5.3.2.1. Biometric subsystem(s) test settings 11 5.3.2.2. Enrollment transactions and results generation12 5.3.2.3. Biometric v
16、erification attempts, transactions, and results generation .12 5.4. Revisit testing.14 5.4.1. Overview.14 5.4.2. Single-revisit testing .14 5.4.3. Multiple-revisit testing .15 5.5. Grading 15 5.5.1. Plotting grades on DET curves.15 5.5.2. Universality of the test 16 5.5.3. Comparability across test
17、alternatives16 5.6. Statistical analysis of test results .17 5.7. Graded test metrics17 5.8. Crew demographics .17 5.8.1. General.17 5.8.2. Age .17 5.8.3. Gender18 5.9. Verification attempts embedded within enrollment transactions 18 5.9.1. Overview.18 5.9.2. Disengagement between enrollment and sam
18、e-day verification transactions18 5.9.3. Disengagement between verification transactions .18 5.10. Impostor tests .18 5.10.1. General .18 5.10.2. Crew composition .19 5.10.3. Method of analysis 19 5.11. Crew size19 6. Test requirements .20 6.1. Planning .20 6.1.1. General.20 6.1.2. Concept of operat
19、ions 20 6.1.3. Supplier responsibilities20 6.2. Test schedule.20 6.3. Test interfaces21 6.4. Fidelity to native system(s) operations.21 6.5. General test approach21 ii 6.5.1. General.21 6.5.2. Configuration management 21 6.5.3. System(s) operability verification22 6.6. Test crew selection 22 6.6.1.
20、General.22 6.6.2. Pre-test activities 22 6.6.2.1. General .22 6.6.2.2. Pre-test briefing.22 6.6.2.3. Configuration audit23 6.6.2.4. Test readiness review .23 6.7. Data collection23 6.8. Problem reporting and tracking 24 6.9. Post-test briefing 25 6.10. Data analysis 25 6.11. Privacy26 6.11.1. Genera
21、l .26 6.11.2. Crew identity protection 26 6.11.3. Data protection26 6.11.4. Proprietary information26 6.12. Inspection .26 6.12.1. General .26 6.12.2. Physical layout of test environment 26 6.12.3. Specifications27 6.12.4. Architecture.27 6.12.5. Implementation .27 6.13. Operator - crew member inter
22、action.27 iiiForeword (This foreword is not part of American National Standard INCITS 409.5-2011.)This American National Standard defines a general-purpose test methodology forscenario evaluation of biometric access control system(s) and subsystem(s). Thestandard specifies test planning, execution,
23、and reporting requirements. The stan-dard establishes grade levels as functions of observed false reject rates at each ofthree separate false accept rates, failure to enroll rate and transaction time.The general-purpose nature of the standard applies to most common access controlapplication requirem
24、ents, such that results are applicable to many but not all accesscontrol applications. the framework is not suitable for highly specialized access con-trol applications (e.g., those requiring very high levels of protection or with special-ized user populations such as the elderly). highly specialize
25、d access controlapplication warrant test process are beyond the scope of this standard.This standard contains three informative annexes, which are no considered part ofthe standard.Requests for interpretation, suggestions for improvement or addenda, or defect re-ports are welcome. They should be sen
26、t to the InterNational Committee for Informa-tion Technology Standards (INCITS), ITI, 1101 K Street, NW, Suite 610, Washington,DC 20005.This standard was processed and approved for submittal to ANSI by INCITS. Com-mittee approval of this standard does not necessarily imply that all committee mem-ber
27、s voted for its approval. At the time it approved this standard, INCITS had thefollowing members:Don Wright, ChairJennifer Garner, SecretaryOrganization Represented Name of RepresentativeAdobe Systems, Inc Scott Foshee Steve Zilles (Alt.)AIM Global, Inc. . Steve HallidayApple Computer, Inc. . Helene
28、 WorkmanDavid Singer (Alt.)Distributed Management Task Force John Crandall Jeff Hilland (Alt.)Electronic Industries Alliance . Edward Mikoski, Jr. Henry Cuschieri (Alt.)EMC Corporation . Gary RobinsonFarance, Inc Frank FaranceTimothy Schoechle (Alt.)GS1 US . Frank SharkeyCharles Biss (Alt.)Hewlett-P
29、ackard Company Karen Higginbottom Paul Jeran (Alt.)IBM Corporation Gerald Lane Robert Weir (Alt.)Arnaud Le Hors (Alt.)Debra Boland (Alt.)Steve Holbrook (Alt.)Alexander Tarpinian (Alt.)IEEE . Terry deCourcelleJodie Haasz (Alt.)Bob Labelle (Alt.)Joan Woolery (Alt.)Intel Philip Wennblom Grace Wei (Alt.
30、)Stephen Balogh (Alt.)ivOrganization Represented Name of RepresentativeLexmark International. Don Wright Dwight Lewis (Alt.)Paul Menard (Alt.)Jerry Thrasher (Alt.)Microsoft Corporation . Jim Hughes Dick Brackney (Alt.)John Calhoun (Alt.)National Institute of Standards a multiple-revisit test include
31、s multiple revisits. Results from single-revisit tests are not comparable to results from multiple-revisit tests due to the potential effect of habituation on grade levels. Biometric system(s) can be, for instance: A verification system(s) with centralized biometric template storage; A verification
32、system(s) with decentralized biometric template storage in the biometric subsystem(s); A verification system(s) with decentralized biometric template storage (e.g. on an ID card); An identification system(s) used for the purpose of verification (e.g. “PIN-less verification“); Multi-biometric fused s
33、ystem(s). Figure 1 illustrates the components and information flows in a generic access control system(s) that includes a biometric system(s). Real deployed system(s) may vary from this general model. Explanation is described in Table 1. INCITS 409.5-2011 Figure 1 Generic Biometric Access Control Sy
34、stem(s) 2 INCITS 409.5-2011 Table 1 Components and Descriptions of Figure 1 Letter Component Information flow A Token (ID card) Any form of machine-readable credential presented by the user to the ID reader to claim an identity. B (flow between ID reader and the panel) User identity code (ID number,
35、 card number, ACS ID) read from the token by the ID reader and sent to the panel for the ACS to determine access privilege. This flow is part of a typical legacy ACS. C Lock control Electrical signal from the panel used to command the door electro-mechanical locking mechanisms. This flow may also in
36、clude other signals such as door-open indicators, emergency lock override, etc. This flow is part of a typical legacy ACS. D ACS network (Physical) communication channel (Ethernet, RS485, etc.) enabling data interchange between the panel, ACS processor, and ACS database. The ACS network (logically)
37、depends upon site-specific implementation, and includes a user identity code from panel and user access authorization from ACS processor. E Biometric characteristic (trait) The body part or human behavior presented by the applicant to the biometric sensor during enrollment (e.g. fingerprint, iris, v
38、oice, signature). This flow may also include any interactions between applicant and sensor such as audio and visual feedback. NOTE: An applicant becomes a user only after the enrollment process is completed and access privileges are granted by the access control authority. F (flow between the ID rea
39、der and biometric processor) Biometric template data from enrollment database to biometric processor (for implementations using server-stored templates). This flow is architecture-specific, may be per user transaction or periodic pre-loads. G Biographical information Applicant-supplied information (
40、name, address, etc.) obtained during ACS enrollment via the ACS Processor. This flow is part of a typical legacy ACS. 3 INCITS 409.5-2011 1. Scope This American National Standard defines a general-purpose test methodology for scenario evaluation of biometric access control system(s) and subsystem(s)
41、. The standard specifies test planning, execution, and reporting requirements. The standard establishes grade levels as functions of observed false reject rates at each of three separate false accept rates, failure to enroll rate and transaction time. The general purpose nature of the standard appli
42、es to most common access control application requirements, such that results are applicable to many but not all access control applications. The framework is not suitable for highly specialized access control applications (e.g., those requiring very high levels of protection or with specialized user
43、 populations such as the elderly). Highly specialized access control application warrant test processes beyond the scope of this standard. The following types of tests are not in the scope of this standard: Active impostor testing; Environmental; Human factors, including user acceptance; Identificat
44、ion performance metrics; Reliability, availability and maintainability; Safety; Security, including vulnerability. 2. Normative references The following standards contain provisions which, through reference in this text, constitute provisions of this American National Standard. All standards are sub
45、ject to revision, and parties to agreements based on this American National Standard are encouraged to investigate the possibility of applying the most recent editions of the standards indicated below. The following referenced documents are indispensable for the application of this part of INCITS 16
46、02D (or 409). For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO/IEC 17025, General requirements for the competence of testing and calibration laboratories ISO/IEC 19795-1 (2005), Informa
47、tion Technology Biometric Performance Testing and Reporting Part 1: Principles and framework ISO/IEC 19795-2 (2007), Information Technology Biometric Performance Testing and Reporting Part 2: Testing methodologies for technology and scenario evaluation; Clause 7: Scenario Testing 3. Conformance A si
48、ngle-revisit test conforms to this standard if it satisfies all normative requirements other than those in 5.4.2. A multiple-revisit test conforms to this standard if it satisfies all normative requirements other than those in 5.4.3. 4 INCITS 409.5-2011 4. Terms and Definitions For the purposes of t
49、his document, the terms and definitions given in ISO/IEC 19795 parts 1 and 2 and the following apply. 4.1 access control system(s) (ACS) entire electro-mechanical suite that performs the granting or denying of access at controlled entry points of a facility 4.2 access policy level framework provides for evaluation of error rates at two access policy levels: single attempt and multiple-attempt (also referred to as “transaction-level”). The multiple-attempt policy allows up to three attempts per transaction for identity confirmation. Multiple-atte
