ANSI INCITS ISO IEC TR 29198-2013 Information technology - Biometrics - Characterization and measurement of difficulty for fingerprint databases for technology evaluation (Technica.pdf

1、 INCITS/ISO/IEC TR 29198:2013 2015 (ISO/IEC TR 29198:2013, IDT) Information technology - Biometrics - Characterization and measurement of difficulty for fingerprint databases for technology evaluation (Technical Report) INCITS/ISO/IEC TR 29198:2013 2015 PDF disclaimer This PDF file may contain embed

2、ded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, parties accept therein the responsibility of n

3、ot infringing Adobes licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters w

4、ere optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. Registered by INCITS (InterNational Committee for In

5、formation Technology Standards) as an American National Standard. Date of Registration: 7/26/2015 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2015 by Information Technology Industry Council (ITI). All rights reserved. These materials ar

6、e subject to copyright claims of International Standardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Council (ITI). Not for resale. No part of this publication may be reproduced in any fo

7、rm, including an electronic retrieval system, without the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1101 K Street NW, Suite 610, Washington DC 20005. Printed in the United States of America ii ITIC 2015 All rights reserved ISO/IEC 2013Infor

8、mation technology Biometrics Characterization and measurement of difficulty for fingerprint databases for technology evaluationTechnologies de linformation Biomtrie Caractrisation et mesure de difficult pour bases de donnes dempreintes digitales pour valuation de technologieTECHNICAL REPORTISO/IECTR

9、29198First edition2013-12-15Reference numberISO/IEC TR 29198:2013(E)ISO/IEC TR 29198:2013(E)ii ISO/IEC 2013 All rights reservedCOPYRIGHT PROTECTED DOCUMENT ISO/IEC 2013All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form or

10、by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below or ISOs member body in the country of the requester.ISO copyright officeCase postale 56 CH-121

11、1 Geneva 20Tel. + 41 22 749 01 11Fax + 41 22 749 09 47E-mail copyrightiso.orgWeb www.iso.orgPublished in SwitzerlandISO/IEC TR 29198:2013(E) ISO/IEC 2013 All rights reserved iiiContents PageForeword ivIntroduction v1 Scope . 12 Terms and definitions . 13 Symbols and abbreviated terms . 44 Differenti

12、al factors in fingerprint samples . 44.1 General . 44.2 Common area . 64.3 Relative deformation 114.4 Relative sample quality .164.5 Calculating LOD of a dataset .165 Analysis of mated pair data characteristics based on comparison results 205.1 General 205.2 Matchability 215.3 Building datasets of d

13、ifferent levels of difficulty 23Bibliography .28ISO/IEC TR 29198:2013(E)ForewordISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC p

14、articipate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governm

15、ental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part

16、2.The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bod

17、ies casting a vote.In exceptional circumstances, when the joint technical committee has collected data of a different kind from that which is normally published as an International Standard (“state of the art”, for example), it may decide to publish a Technical Report. A Technical Report is entirely

18、 informative in nature and shall be subject to review every five years in the same manner as an International Standard.Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any o

19、r all such patent rights.ISO/IEC TR 29198 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics.iv ISO/IEC 2013 All rights reservedISO/IEC TR 29198:2013(E)IntroductionRecently, there have been worldwide increasing activities in testing and ev

20、aluating the performance of fingerprint recognition systems or algorithms. Testing activities occur in public sector, private sector, and academic entities, typically using datasets exclusive to a given entity. This complicates comparison of test results from different entities. Methodologies for as

21、sessing the level of difficulty of test datasets should improve the comparability of performance evaluation results over different fingerprint datasets.ISO/IEC 19795-1:2006, 5.5.311states:“In a technology evaluation, testing of all algorithms is carried out on a standardized corpus, ideally collecte

22、d by a “universal” sensor (i.e. a sensor that collects samples equally suitable for all algorithms tested). Nonetheless, performance against this corpus will depend on both the environment and the population in which it is collected.”Comparison of evaluation results based on testing against differen

23、t corpora may be misleading. Further, policies for inclusion or removal of low-quality data in a corpus may vary from organization to organization, such that the same algorithm tested against the same corpus may generate different results. There are also certain difficulties when trying to compare m

24、ultiple evaluation results derived from different corpora. Currently there is no established methodology for characterizing the level of difficulty of datasets used in performance evaluation. The ability to characterize a datasets level of difficulty should support predictions of operational accurac

25、y when processing data known to be of equivalent difficulty.The purpose of this Technical Report is to provide guidance on predicting how “challenging“ or “stressing“ a fingerprint dataset is for recognition, based on factors such as relative sample quality, relative rotation, deformation, and overl

26、ap between impressions. The provided guidance can be used for characterizing and measuring the relative difficulty levels of fingerprint datasets used in technology evaluation.Following the guidance in this Technical Report, users and system evaluators in different organizations will be able to comp

27、are and place into context the performance evaluation results of the other organizations according to the level of difficulty of its dataset.This Technical Report proposes dataset generation methods based on analysis of comparison results or scores from multiple fingerprint recognition algorithms. T

28、hese dataset generation methods support creation of datasets with specific levels of difficulty and creation of datasets for use in interoperability evaluations.ISO/IEC TR 29794-416defines methods for expressing the quality score of a single fingerprint image. Such quality scores are typically predi

29、ctive of matching accuracy. This Technical Report, by contrast, is concerned with differences in rotation, deformation, and common area between reference and probe samples.NOTE Other modalities can be considered in the future as more information becomes available about standardized quality measureme

30、nts that are suitable for predicting the performance of other biometric systems. ISO/IEC 2013 All rights reserved vInformation technology Biometrics Characterization and measurement of difficulty for fingerprint databases for technology evaluation1 ScopeThis Technical Report provides guidance on est

31、imating how “challenging“ or “stressing“ is an evaluation dataset for fingerprint recognition, based on relative sample quality, relative rotation, deformation, and overlap between impressions. In addition, this Technical Report establishes a method for construction of datasets of different levels o

32、f difficulty. This Technical Report defines the relative level of difficulty of a fingerprint dataset used in technology evaluation of fingerprint recognition algorithms. Level of difficulty is based on differences between reference and probe samples in the aformentioned factors. This Technical Repo

33、rt addresses such issues as: characterizing level of difficulty attributable to differences between samples acquired from the same finger, developing statistical methodologies for representing the level of difficulty of a fingerprint dataset by aggregating influencing factors, comparing the level of

34、 difficulty of different fingerprint datasets, defining procedures for testing and reporting the level of difficulty of fingerprint datasets collected for technology evaluation, analysing mated pair data characteristics based on comparison scores, describing the archived data selection methodology f

35、or building a dataset for evaluation.This Technical Report provides guidelines for comparing the relative level of difficulty of fingerprint datasets.Outside the scope of this Technical Report are: defining the quality of individual fingerprint images, defining the methodologies or explicit measures

36、 for evaluating or predicting the performance of fingerprint recognition algorithms.2 Terms and definitionsFor the purposes of this document, the following terms and definitions apply.2.1raw biometric sampleinformation obtained from a biometric sensor, either directly or after further processing2.2b

37、iometric referenceone or more stored biometric samples, biometric templates or biometric models attributed to a biometric data subject and used as the object of comparisonEXAMPLE Face image stored digitally on a passport; Fingerprint minutiae template on a National ID card; Gaussian Mixture Model fo

38、r speaker recognition, in a dataset.TECHNICAL REPORT ISO/IEC TR 29198:2013(E) ISO/IEC 2013 All rights reserved 1ISO/IEC TR 29198:2013(E)Note 1 to entry: A biometric reference may be created with implicit or explicit use of auxiliary data, such as Universal Background Models.Note 2 to entry: The subj

39、ect/object labelling in a comparison might be arbitrary. In some comparisons a biometric reference might be used as the subject of the comparison with other biometric references or incoming samples used as the objects of the comparisons. For example, in a duplicate enrolment check a biometric refere

40、nce will be used as the subject for comparison against all other biometric references in the dataset.2.3biometric probebiometric data input to an algorithm for comparison to a biometric reference(s)2.4technology evaluationoffline evaluation of one or more algorithms for the same biometric modality u

41、sing a pre-existing or specially collected corpus of samples2.5failure-to-enrol rateFTEproportion of the population for whom the system fails to complete the enrolment processNote 1 to entry: The observed failure-to-enrol rate is measured on test crew enrolments. The predicted/expected failure-to-en

42、rol rate will apply to the entire target population.2.6failure-to-acquire rateFTAproportion of verification or identification attempts for which the system fails to capture or locate an image or signal of sufficient qualityNote 1 to entry: The observed failure-to-acquire rate is distinct from the pr

43、edicted/expected failure-to-acquire rate (the former may be used to estimate the latter).2.7false non-match rateFNMRproportion of genuine attempt samples falsely declared not to match the biometric reference of the same characteristic from the same subject supplying the sample2.8false match rateFMRp

44、roportion of zero-effort impostor attempt samples falsely declared to match the compared non-self templateNote 1 to entry: The measured/observed false match rate is distinct from the predicted/expected false match rate (the former may be used to estimate the latter).2.9false reject rateFRRproportion

45、 of verification transactions with truthful claims of identity that are incorrectly denied2.10false accept rateFARproportion of verification transactions with wrongful claims of identity that are incorrectly confirmed2 ISO/IEC 2013 All rights reservedISO/IEC TR 29198:2013(E)2.11receiver operating ch

46、aracteristic curveROC curveplot of the rate of false positives (i.e. impostor attempts accepted) on the x-axis against the corresponding rate of true positives (i.e. genuine attempts accepted) on the y-axis plotted parametrically as a function of the decision threshold2.12detection error trade-off c

47、urveDET curvemodified ROC curve which plots error rates on both axes (false positives on the x-axis and false negatives on the y-axis)2.13performancecapability in terms of error rates and throughput rates2.14qualitydegree to which a biometric sample fulfils specified requirements for a targeted appl

48、icationNote 1 to entry: Specified quality requirements may address aspects of quality such as focus, resolution, etc. Implicit quality requirements address the likelihood of achieving a correct matching result.2.15quality scorequantitative expression of quality2.16matchabilitydegree to which two mat

49、ed fingerprint samples can be successfully compared through multiple comparison algorithms2.17mated pairset of two samples of the same biometric characteristics captured from the same source, where one is used for the reference and the other used for the test2.18level of difficultymeasure of a biometric dataset which represents how challenging or stressing the fingerprint dataset is for recognition relative to other datasetsNote 1 to ent