BS ISO IEC 19795-4-2008 Information technology Biometric performance testing and reporting Interoperability performance testing《信息技术 生物计量性能试验和报告 互通性性能试验》.pdf

1、BRITISH STANDARD BS ISO/IEC 19795-4:2008 Information technology Biometric performance testing and reporting Part 4: Interoperability performance testing ICS 35.040 BS ISO/IEC 19795-4:2008 This British Standard was published under the authority of the Standards Policy and Strategy Committee on 31 Jul

2、y 2008 BSI 2008 ISBN 978 0 580 54244 2 National foreword This British Standard is the UK implementation of ISO/IEC 19795-4:2008. The UK participation in its preparation was entrusted to Technical Committee IST/44, Biometrics. A list of organizations represented on this committee can be obtained on r

3、equest to its secretary. This publication does not purport to include all the necessary provisions of a contract. Users are responsible for its correct application. Compliance with a British Standard cannot confer immunity from legal obligations. Amendments/corrigenda issued since publication Date C

4、omments Reference number ISO/IEC 19795-4:2008(E)INTERNATIONAL STANDARD ISO/IEC 19795-4 First edition 2008-06-01 Information technology Biometric performance testing and reporting Part 4: Interoperability performance testing Technologies de linformation Essais et rapports de performances biomtriques

5、Partie 4: Essais de performances dinteroprabilit BS ISO/IEC 19795-4:2008ii iiiContents Page Foreword vi Introduction.vii 1 Scope1 2 Conformance .1 3 Normative references2 4 Terms and definitions .2 5 Abbreviated terms .4 6 Goals.5 6.1 Coverage 5 6.2 Target application8 6.2.1 Biometric application8 6

6、.2.2 Interoperable application9 6.3 Purpose 10 6.3.1 Interoperability testing10 6.3.2 Sufficiency testing.11 7 Metrics 12 7.1 General .12 7.2 Figures of merit .12 7.2.1 Recognition performance figure of merit12 7.2.2 Measuring component failure 13 7.3 Interoperability matrices.14 7.3.1 General .14 7

7、.3.2 Interoperability with sBDB generators14 7.3.3 Interoperability with sBDB generators15 7.3.4 Fixed operating point interoperability.16 7.3.5 Reporting failure of sBDB generators.16 7.4 Proprietary performance.16 8 Conducting a test 17 8.1 Structure of test.17 8.2 Sample data .17 8.2.1 Acquisitio

8、n .17 8.2.2 Representative data 18 8.2.3 Collection of ancillary data.18 8.2.4 Corpus size 18 8.2.5 Removal of subject-specific metadata 18 8.2.6 Removal of unrepresentative metadata 18 8.2.7 Origin of samples 19 8.2.8 Untainted samples.19 8.2.9 Sequestered data.19 8.3 Conformance testing.19 8.3.1 C

9、onformance .19 8.3.2 Executing conformance tests 19 8.3.3 Reporting20 8.4 Constraints on the sBDBs20 8.4.1 Optional encodings .20 8.4.2 Optional encodings from profile standards20 8.4.3 Deviation from the base standard .20 8.4.4 Data encapsulation20 BS ISO/IEC 19795-4:2008iv 8.5 Components.21 8.5.1

10、Components for sufficiency testing 21 8.5.2 Establishing modularity requirements 21 8.5.3 Components for interoperability testing .21 8.5.4 Underlying algorithms.21 8.5.5 Capture device user interfaces 21 8.5.6 Multimodal components .22 8.5.7 Component variability .22 8.5.8 Component reporting requi

11、rements 22 8.6 Planning decisions 22 8.6.1 Computational intensity22 8.6.2 Supplier recruitment23 8.6.3 Provision of samples to suppliers .23 8.6.4 Equivalency of generator resources23 8.6.5 Handling violations of test requirements24 8.6.6 Comparison subsystem output data encapsulation24 8.6.7 Funda

12、mental generator requirement.24 8.6.8 Fundamental comparison subsystem requirement .25 8.6.9 General requirements on software implementations.25 8.7 Prevention and detection of gaming26 8.7.1 General aspects .26 8.7.2 Modes of gaming .26 8.7.3 Prevention and detection of gaming28 8.8 Test procedure.

13、29 8.8.1 Primary test 29 8.8.2 Uncertainty measurement.30 8.8.3 Variance estimation.30 8.8.4 Remedial testing 30 8.8.5 Survey of configurable parameters .30 9 Interpretation of the interoperability matrix30 9.1 Determination of interoperable subsystems 30 9.1.1 General30 9.1.2 Identifying interopera

14、ble combinations of subsystems31 9.1.3 Acceptable numbers of interoperable subsystems .33 9.1.4 Combinatorial search for maximum interoperability-classes.33 9.1.5 Multiple interoperable subgroups34 9.1.6 Statistical stability of the test result 34 9.2 Interoperability with previously certified produ

15、cts35 9.2.1 Decertification considerations .35 9.2.2 Continuity of testing35 9.2.3 Interoperability with previously certified generators.35 9.2.4 Interoperability with previously certified comparison subsystems.36 9.2.5 Treatment of systematic effects.36 9.2.6 Retroactive exclusion from analysis .37

16、 9.3 Overall sufficiency.37 Annex A (informative) Procedures for conducting a test of sufficiency and/or interoperability38 Annex B (informative) Example Interoperability Test42 Bibliography 45 Figure 1 General biometric interoperability 6 Figure 2 Specific interoperability: enrolment BDB is standar

17、dized .6 Figure 3 Specific interoperability: enrolment BDB is proprietary.7 Figure 4 Offline interoperability testing.7 Figure 5 Biometric capture device interoperability 8 BS ISO/IEC 19795-4:2008vFigure 6 Cells of an example interoperability space10 Figure 7 Sufficiency testing: proprietary vs. sta

18、ndard interchange formats 12 Figure 8 Cross-generator performance matrix .15 Figure 9 Example performance matrix 15 Figure 10 Proprietary performance matrix16 Table 1 Conformity with ISO/IEC 19795-2 .1 Table 2 Sample size adjustment of error rate requirement.31 Table 3 Confidence levels of the stand

19、ard Normal distribution.32 Table A.1 Interoperability test procedure, phase 1: planning.38 Table A.2 Interoperability test procedure, phase 2: setup39 Table A.3 Interoperability test procedure, phase 3: sBDB and pBDB generation39 Table A.4 Interoperability test procedure, phase 4: verification 40 Ta

20、ble A.5 Interoperability test procedure, phase 5: identification .40 Table A.6 Interoperability test procedure, phase 6: reporting41 Table A.7 Interoperability test procedure, phase 7: variance estimation41 BS ISO/IEC 19795-4:2008vi Foreword ISO (the International Organization for Standardization) a

21、nd IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal w

22、ith particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC h

23、ave established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joi

24、nt technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights

25、. ISO and IEC shall not be held responsible for identifying any or all such patent rights. ISO/IEC 19795-4 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics. ISO/IEC 19795 consists of the following parts, under the general title Informati

26、on technology Biometric performance testing and reporting: Part 1: Principles and framework Part 2: Testing methodologies for technology and scenario evaluation Part 3: Modality-specific testing Technical Report Part 4: Interoperability performance testing Part 6: Testing methodologies for operation

27、al evaluation is under preparation. BS ISO/IEC 19795-4:2008viiIntroduction The multi-part biometric data interchange format standard, ISO/IEC 19794, has been developed to foster interoperable exchange of biometric data. By defining open containers for image, signal and feature data, and constraining

28、 some of the properties of the samples, the standards enhance interoperability by requiring implementers to be able to handle a restricted set of all possible biometric samples. Examples of this are the template standards of ISO/IEC 19794-2 and ISO/IEC 19794-8 which embed compact processed data from

29、 fingerprint images. Only samples of the same format type (several of which can be defined in the same part of ISO/IEC 19794) are intended to be interchangeable. One common assertion prior to SC 37s formulation of data interchange standards was that proprietary templates offer greater recognition pe

30、rformance than any likely standard on the grounds that the proprietary instances are the product of processes that embed considerable, private, intellectual property. The question of whether the emerging standards are sufficient then arises: that is, do they code data (feature, image, etc.) represen

31、tations that allow matching with accuracy comparable to that available from the proprietary solutions? A second issue, interoperability, arises in those applications where standardized data are generated and matched by different institutions and systems. If a companys feature extraction subsystem pr

32、ocesses acquired samples to produce ISO/IEC 19794-x compliant instances, then can other companies comparison subsystems attain performance comparable with that obtained from the originators own comparison subsystem? A further question is then whether a third company can successfully recognize enrolm

33、ent and user samples from two different sources. This part of ISO/IEC 19795 defines tests to specifically address absolute performance, sufficiency, and interoperability available from biometric data formatted to comply with established standards, particularly those developed in the various parts of

34、 ISO/IEC 19794. However, because this part of ISO/IEC 19795 references interchange formats generically, by referencing only their black box generation and use, it also applies to other open standards. One consequence of this approach is that the success of a test is predicated on the correctness and

35、 appropriateness of lower-level data elements and values, i.e. conformance to the respective standards. Therefore, the approach here is to require conformance testing as an integral part of the test. This is achieved by referencing formal published conformance tests or profiles of standards. For ins

36、tance, an interoperability test of the ISO/IEC 19794-5 face format might reference an application profile of its Token image, which in turn might rely on ISO/IEC 15444-1 (JPEG 2000 core coding system). This part of ISO/IEC 19795 conceives of the following three kinds of tests: online: a scenario tes

37、t in which a volunteer population enrols on suppliers products and subsequently uses suppliers verification or identification implementations to make genuine and impostor attempts; offline: a technology test in which an archived corpus of captured samples, not necessarily collected with any intent t

38、o simulate the operational conditions of a particular application, is used as input to suppliers enrolment, verification or identification products to make genuine and impostor attempts; hybrid: a test in which the sample corpus is collected online under conditions which attempt to simulate the oper

39、ational conditions of a particular application, and is then processed offline. In each case, an interoperability test needs to embed multi-supplier generation, exchange, and comparison of samples of the standard interchange format. Online collection from a live population is appropriate when the bio

40、metric capture device, and/or the subject interaction with the biometric capture device, is considered to have a material effect on the interoperable performance of the intended application. An offline test is appropriate when a representative corpus of samples is already available (for example pass

41、port photographs to be converted into Token instances of ISO/IEC 19794-5). An offline test may be appropriate when the collection of representative data is neither practical nor necessary to determine the interoperable performance of specific subsystems, such as feature extraction and/or comparison.

42、 BS ISO/IEC 19795-4:2008viii In all cases, an interoperability test must enrol subjects on one or more products and verify or identify on one or more others. This should involve subjects making transactions as themselves (genuine trials) and as one or more other people (impostor trials). If a large

43、enough population is available, a disjoint impostor population can be used. Since online tests can become onerous on the test population when many products and impostor attempts are needed, hybrid and offline testing allow execution of many zero-effort impostor attempts. In an interoperability perfo

44、rmance test, J generators of standardized biometric data blocks (BDBs) are applied to the samples assembled as part of a hybrid or offline test. By applying K comparison subsystems to the standard BDBs, up to KJ 2verification or identification trials are conducted, each following ISO/IEC 19795-2. Th

45、e BDB may be an image or signal, or a standardized template. Optional encodings allowed by the standard interchange format should be fully specified. This might be achieved by normatively referencing one of the ISO/IEC 24713-x profiles. If the format in question is an image, a subsequent internal (u

46、sually proprietary) template would be used, but its existence here is subsumed by the notion of a black-box comparison of two instances of the given format. The test advanced by this part of ISO/IEC 19795 demarcates the generic aspects of interoperability from the meaning associated with each partic

47、ular biometric format of ISO/IEC 19794-x. BS ISO/IEC 19795-4:2008 1Information technology Biometric performance testing and reporting Part 4: Interoperability performance testing 1 Scope This part of ISO/IEC 19795 prescribes methods for technology and scenario evaluations of multi-supplier biometric

48、 systems that use biometric data conforming to biometric data interchange format standards. It specifies requirements needed to assess performance available from samples formatted according to a standard interchange format (SIF), performance available when samples formatted according to a SIF are ex

49、changed, performance available from samples formatted according to a SIF, relative to proprietary data formats, SIF interoperability, by quantifying cross-product performance relative to single-product performance, performance available from multi-sample and multimodal data formatted according to one or more SIFs, and performance interoperability of biometric capture devices. In addition