1、American National StandardDeveloped byfor Information Technology Biometric Performance Testingand Reporting Part 5: Framework for Testingand Evaluation of Biometric System(s)for Access ControlINCITS 409.5-2011INCITS 409.5-2011Copyright American National Standards Institute Provided by IHS under lice
2、nse with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS 409.5-2011American Nation
3、al Standardfor Information Technology Biometric Performance Testingand Reporting Part 5: Framework for Testing andEvaluation of Biometric System(s)for Access ControlSecretariatInformation Technology Industry CouncilApproved December 5, 2011American National Standards Institute, Inc.AbstractThis stan
4、dard specifies a framework for testing and reporting of biometric system(s) used in ap-plications supporting access control. It specifies the environment that testing will be performedin, the evaluation metrics, demographic controls, and the means by which testing will be per-formed and how the grad
5、ed results will be reported. The intent of this testing framework is to pro-vide a “one size fits many” approach to re-usable test results, centered on typical access controlrequirementsCopyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduct
6、ion or networking permitted without license from IHS-,-,-Approval of an American National Standard requires review by ANSI that therequirements for due process, consensus, and other criteria for approval havebeen met by the standards developer.Consensus is established when, in the judgement of the A
7、NSI Board ofStandards Review, substantial agreement has been reached by directly andmaterially affected interests. Substantial agreement means much more thana simple majority, but not necessarily unanimity. Consensus requires that allviews and objections be considered, and that a concerted effort be
8、 madetowards their resolution.The use of American National Standards is completely voluntary; theirexistence does not in any respect preclude anyone, whether he has approvedthe standards or not, from manufacturing, marketing, purchasing, or usingproducts, processes, or procedures not conforming to t
9、he standards.The American National Standards Institute does not develop standards andwill in no circumstances give an interpretation of any American NationalStandard. Moreover, no person shall have the right or authority to issue aninterpretation of an American National Standard in the name of the A
10、mericanNational Standards Institute. Requests for interpretations should beaddressed to the secretariat or sponsor whose name appears on the titlepage of this standard.CAUTION NOTICE: This American National Standard may be revised orwithdrawn at any time. The procedures of the American National Stan
11、dardsInstitute require that action be taken periodically to reaffirm, revise, orwithdraw this standard. Purchasers of American National Standards mayreceive current information on all standards by calling or writing the AmericanNational Standards Institute.American National StandardPublished byAmeri
12、can National Standards Institute, Inc.25 West 43rd Street, New York, NY 10036Copyright 2011 by Information Technology Industry Council (ITI)All rights reserved.No part of this publication may be reproduced in anyform, in an electronic retrieval system or otherwise,without prior written permission of
13、 ITI, 1101 K Street NW, Suite 610, Washington, DC 20005. Printed in the United States of AmericaCAUTION: The developers of this standard have requested that holders of patents that may berequired for the implementation of the standard disclose such patents to the publisher. However,neither the devel
14、opers nor the publisher have undertaken a patent search in order to identifywhich, if any, patents may apply to this standard. As of the date of publication of this standardand following calls for the identification of patents that may be required for the implementation ofthe standard, no such claim
15、s have been made. No further patent search is conducted by the de-veloper or publisher in respect to any standard it processes. No representation is made or impliedthat licenses are not required to avoid infringement in the use of this standard.Copyright American National Standards Institute Provide
16、d by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-Contents i Page 1. Scope4 2. Normative references4 3. Conformance.4 4. Terms and Definitions .5 5. Test Methodology7 5.1. Overview 7 5.2. Evaluation metrics and grades.7 5.2.1. Overvie
17、w.7 5.2.2. FAR levels and comparison error rates 8 5.2.3. Failure-to-enroll rate .8 5.2.4. Transaction time .9 5.2.5. Multi-biometric system(s)10 5.2.6. Evaluation metrics as a summary.10 5.3. Methodology.11 5.3.1. Introduction.11 5.3.2. Test processes .11 5.3.2.1. Biometric subsystem(s) test settin
18、gs 11 5.3.2.2. Enrollment transactions and results generation12 5.3.2.3. Biometric verification attempts, transactions, and results generation .12 5.4. Revisit testing.14 5.4.1. Overview.14 5.4.2. Single-revisit testing .14 5.4.3. Multiple-revisit testing .15 5.5. Grading 15 5.5.1. Plotting grades o
19、n DET curves.15 5.5.2. Universality of the test 16 5.5.3. Comparability across test alternatives16 5.6. Statistical analysis of test results .17 5.7. Graded test metrics17 5.8. Crew demographics .17 5.8.1. General.17 5.8.2. Age .17 5.8.3. Gender18 5.9. Verification attempts embedded within enrollmen
20、t transactions 18 5.9.1. Overview.18 5.9.2. Disengagement between enrollment and same-day verification transactions18 5.9.3. Disengagement between verification transactions .18 5.10. Impostor tests .18 5.10.1. General .18 5.10.2. Crew composition .19 5.10.3. Method of analysis 19 5.11. Crew size19 6
21、. Test requirements .20 6.1. Planning .20 6.1.1. General.20 6.1.2. Concept of operations 20 6.1.3. Supplier responsibilities20 6.2. Test schedule.20 6.3. Test interfaces21 6.4. Fidelity to native system(s) operations.21 6.5. General test approach21 Copyright American National Standards Institute Pro
22、vided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-ii 6.5.1. General.21 6.5.2. Configuration management 21 6.5.3. System(s) operability verification22 6.6. Test crew selection 22 6.6.1. General.22 6.6.2. Pre-test activities 22 6.6.
23、2.1. General .22 6.6.2.2. Pre-test briefing.22 6.6.2.3. Configuration audit23 6.6.2.4. Test readiness review .23 6.7. Data collection23 6.8. Problem reporting and tracking 24 6.9. Post-test briefing 25 6.10. Data analysis 25 6.11. Privacy26 6.11.1. General .26 6.11.2. Crew identity protection 26 6.1
24、1.3. Data protection26 6.11.4. Proprietary information26 6.12. Inspection .26 6.12.1. General .26 6.12.2. Physical layout of test environment 26 6.12.3. Specifications27 6.12.4. Architecture.27 6.12.5. Implementation .27 6.13. Operator - crew member interaction.27 Copyright American National Standar
25、ds Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-iiiForeword (This foreword is not part of American National Standard INCITS 409.5-2011.)This American National Standard defines a general-purpose test methodology f
26、orscenario evaluation of biometric access control system(s) and subsystem(s). Thestandard specifies test planning, execution, and reporting requirements. The stan-dard establishes grade levels as functions of observed false reject rates at each ofthree separate false accept rates, failure to enroll
27、rate and transaction time.The general-purpose nature of the standard applies to most common access controlapplication requirements, such that results are applicable to many but not all accesscontrol applications. the framework is not suitable for highly specialized access con-trol applications (e.g.
28、, those requiring very high levels of protection or with special-ized user populations such as the elderly). highly specialized access controlapplication warrant test process are beyond the scope of this standard.This standard contains three informative annexes, which are no considered part ofthe st
29、andard.Requests for interpretation, suggestions for improvement or addenda, or defect re-ports are welcome. They should be sent to the InterNational Committee for Informa-tion Technology Standards (INCITS), ITI, 1101 K Street, NW, Suite 610, Washington,DC 20005.This standard was processed and approv
30、ed for submittal to ANSI by INCITS. Com-mittee approval of this standard does not necessarily imply that all committee mem-bers voted for its approval. At the time it approved this standard, INCITS had thefollowing members:Don Wright, ChairJennifer Garner, SecretaryOrganization Represented Name of R
31、epresentativeAdobe Systems, Inc Scott Foshee Steve Zilles (Alt.)AIM Global, Inc. . Steve HallidayApple Computer, Inc. . Helene WorkmanDavid Singer (Alt.)Distributed Management Task Force John Crandall Jeff Hilland (Alt.)Electronic Industries Alliance . Edward Mikoski, Jr. Henry Cuschieri (Alt.)EMC C
32、orporation . Gary RobinsonFarance, Inc Frank FaranceTimothy Schoechle (Alt.)GS1 US . Frank SharkeyCharles Biss (Alt.)Hewlett-Packard Company Karen Higginbottom Paul Jeran (Alt.)IBM Corporation Gerald Lane Robert Weir (Alt.)Arnaud Le Hors (Alt.)Debra Boland (Alt.)Steve Holbrook (Alt.)Alexander Tarpin
33、ian (Alt.)IEEE . Terry deCourcelleJodie Haasz (Alt.)Bob Labelle (Alt.)Joan Woolery (Alt.)Intel Philip Wennblom Grace Wei (Alt.)Stephen Balogh (Alt.)Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted without li
34、cense from IHS-,-,-ivOrganization Represented Name of RepresentativeLexmark International. Don Wright Dwight Lewis (Alt.)Paul Menard (Alt.)Jerry Thrasher (Alt.)Microsoft Corporation . Jim Hughes Dick Brackney (Alt.)John Calhoun (Alt.)National Institute of Standards a multiple-revisit test includes m
35、ultiple revisits. Results from single-revisit tests are not comparable to results from multiple-revisit tests due to the potential effect of habituation on grade levels. Biometric system(s) can be, for instance: A verification system(s) with centralized biometric template storage; A verification sys
36、tem(s) with decentralized biometric template storage in the biometric subsystem(s); A verification system(s) with decentralized biometric template storage (e.g. on an ID card); An identification system(s) used for the purpose of verification (e.g. “PIN-less verification“); Multi-biometric fused syst
37、em(s). Figure 1 illustrates the components and information flows in a generic access control system(s) that includes a biometric system(s). Real deployed system(s) may vary from this general model. Explanation is described in Table 1. Copyright American National Standards Institute Provided by IHS u
38、nder license with ANSI Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-INCITS 409.5-2011 Figure 1 Generic Biometric Access Control System(s) 2 Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or n
39、etworking permitted without license from IHS-,-,-INCITS 409.5-2011 Table 1 Components and Descriptions of Figure 1 Letter Component Information flow A Token (ID card) Any form of machine-readable credential presented by the user to the ID reader to claim an identity. B (flow between ID reader and th
40、e panel) User identity code (ID number, card number, ACS ID) read from the token by the ID reader and sent to the panel for the ACS to determine access privilege. This flow is part of a typical legacy ACS. C Lock control Electrical signal from the panel used to command the door electro-mechanical lo
41、cking mechanisms. This flow may also include other signals such as door-open indicators, emergency lock override, etc. This flow is part of a typical legacy ACS. D ACS network (Physical) communication channel (Ethernet, RS485, etc.) enabling data interchange between the panel, ACS processor, and ACS
42、 database. The ACS network (logically) depends upon site-specific implementation, and includes a user identity code from panel and user access authorization from ACS processor. E Biometric characteristic (trait) The body part or human behavior presented by the applicant to the biometric sensor durin
43、g enrollment (e.g. fingerprint, iris, voice, signature). This flow may also include any interactions between applicant and sensor such as audio and visual feedback. NOTE: An applicant becomes a user only after the enrollment process is completed and access privileges are granted by the access contro
44、l authority. F (flow between the ID reader and biometric processor) Biometric template data from enrollment database to biometric processor (for implementations using server-stored templates). This flow is architecture-specific, may be per user transaction or periodic pre-loads. G Biographical infor
45、mation Applicant-supplied information (name, address, etc.) obtained during ACS enrollment via the ACS Processor. This flow is part of a typical legacy ACS. 3 Copyright American National Standards Institute Provided by IHS under license with ANSI Not for ResaleNo reproduction or networking permitted
46、 without license from IHS-,-,-INCITS 409.5-2011 1. Scope This American National Standard defines a general-purpose test methodology for scenario evaluation of biometric access control system(s) and subsystem(s). The standard specifies test planning, execution, and reporting requirements. The standar
47、d establishes grade levels as functions of observed false reject rates at each of three separate false accept rates, failure to enroll rate and transaction time. The general purpose nature of the standard applies to most common access control application requirements, such that results are applicabl
48、e to many but not all access control applications. The framework is not suitable for highly specialized access control applications (e.g., those requiring very high levels of protection or with specialized user populations such as the elderly). Highly specialized access control application warrant test processes beyond the scope of this standard. The following types of tests are not in the scope of this standard: Active impostor testing; Environmental; Human factors, including user acceptance; Identification performance metrics; Reliability, availability and maintainabilit
