IEEE 2003-1997 en Information Technology - Requirements and Guidelines for Test Methods Specifications and Test Method Implementations for Measuring Conformance.pdf

1、IEEE Std 2003-1997IEEE Standard for Information TechnologyRequirements and Guidelines forTest Methods Specifications andTest Method Implementations forMeasuring Conformance to POSIX StandardsSponsorPortable Applications Standards Committeeof theIEEE Computer SocietyAbstract: This standard defines th

2、e requirements and guidelines for test methodspecifications and test method implementations for measuring conformance toPOSIX standards. Test specification standard developers for other ApplicationProgramming Interface (API) standards are encouraged to use this standard.This document is aimed primar

3、ily at developers and users of test methodspecifications and implementations.Keywords: assertion, assertion test, implementation under test, option, confor-mance document, conformance test procedure, conformance test software, testmethod implementation, test method specification, test result codePOS

4、IX is a registered trademark of the Institute of Electrical and Electronics Engineers, Inc.ulululululululululululululululululululululululululululululululululululululululululululululululululululululululululThe Institute of Electrical and Electronics Engineers, Inc.345 East 47th Street, New York, NY 1

5、0017-2394, USACopyright 1998 by theInstitute of Electrical and Electronics Engineers, Inc.All rights reserved. Published 1998.Printed in the United States of America.ISBN 1-55937-895-6No part of this publication may be reproduced in any form,in an electronic retrieval system or otherwise,without the

6、 prior written permission of the publisher.ulululululululululululululululululululululululululululululululululululululululululululululululululululululululululbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbrbr3ISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E)ContentsPAGESection 1: Genera

7、l 11.1 Scope 11.2 References 21.3 Conformance Criteria 21.4 IUT Conformance Assessment 4Section 2: Definitions and General Requirements . 72.1 Conventions 72.2 Definitions 9Section 3: Assertion Definitions, Types, Syntax, and Constructs 153.1 Introduction 153.2 Generic Assertion Structure 163.3 Asse

8、rtion Types and Constructs . 183.4 Macros 243.5 Summary . 25Section 4: Test Result Codes . 274.1 Introduction 274.2 Test Method Implementations 274.3 Test Method Specifications . 30Section 5: Test Report . 315.1 Test Report 315.2 CD Audit . 31Section 6: Profiles 336.1 Definition . 336.2 Conformance

9、to a Profile . 336.3 Conformance Assessment . 34Section 7: Guidelines for Testing and Complexity Levels . 397.1 Introduction 397.2 Testing Levels . 397.3 Complexity Levels 417.4 Conclusion 41Section 8: Guidelines for Writing Assertions . 438.1 Introduction 438.2 Identifying Preconditions . 488.3 Wri

10、ting the . 518.4 Other Assertion Types 538.5 Macros 56iiISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E)Section 9: Comprehensive Examples 599.1 Specification of Allowable Test Result Codes . 59Annex A (informative) Bibliography 71A.1 Related Standards 71Annex B (informative) Rationale and Notes . 73B.1

11、 General . 73B.2 Definitions and General Requirements . 74B.3 Assertion Definitions, Types, Syntax, and Constructs 76B.4 Test Result Codes 77B.5 Test Report 78B.6 Profiles . 79B.7 Guidelines for Testing and Complexity Levels 79B.8 Guidelines for Writing Assertions 80Alphabetic Topical Index 73FIGURE

12、SFigure 1-1 - Single Base Standard . 5Figure 3-1 - Generic Assertion Structure . 16Figure 3-2 - Assertion Types . 19Figure 3-3 - Basic Assertion Structure 20Figure 3-4 - General Assertion Structure . 20Figure 3-5 - Assertion Derived from a General Assertion . 21Figure 3-6 - Reference Assertion Struc

13、ture 21Figure 3-7 - Documentation Assertion Structure 22Figure 3-8 - General Documentation Assertion Structure . 23Figure 3-9 - Documentation Assertion from a General DocumentationAssertion 23Figure 3-10 - Examples of Unused Assertion Identifiers 24Figure 4-1 - Entity versus Allowable Test Result Co

14、de 30Figure 6-1 - Profile Standard from Multiple Base Standards 36Figure 6-2 - Profile Standard from Multiple Profiles . 37Figure 8-1 - Terms . 46Figure 9-1 - Entity versus Allowable Test Result Code 60TABLESTable 2-1 - Typographical Conventions 8Table 8-1 - C Language Limits 52Table 8-2 - Phrases D

15、enoting Allowable Test Result Codes . 56Table 8-3 - Features Denoting Allowable Test Result Codes 56iiiISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E)ivIEEE Std 2003-1997 (E)vParticipantsIEEE Std 2003-1997 was prepared by the 2003 Working Group, sponsored by the Portable ApplicationsStandards Committe

16、e of the IEEE Computer Society. At the time this standard was approved, the member-ship of the 2003 Working Group was as follows:Portable Applications Standards Committee (POSIX)Chair: Lowell JohnsonVice-Chair: Joseph GwinnSecretary: Nick StoughtonFunctional Chair of Balloting: Jay AshfordFunctional

17、 Chair of Interpretations: Andrew JoseyFunctional Chair of Logistics: Curtis Royster2003 Working Group OfcialsChair: Barry Hedquist*Roger Martin (1995)Lowell Johnson (1992-1995)Vice-Chair: Barry HedquistEditor: Anthony Cincotta*Secretary: Keith Stobie (1992-1994)Working GroupDon Cragun Eric Lewine K

18、rys SuppleeKevin Dodson Kathy Liburdy Ken ThomasShiela Frankel Glenn McPherson Andrew TwiggerKen Harvey Jerry Powell* Bruce WeinerDavid Hollenbeck Tom Robinson Fred ZlotnickWilliam SudmanIn the preceding list, those individuals identied with asterisks (*) served during the balloting period asTechnic

19、al Reviewers for resolving comments and objections to designated portions of the standard.The following persons were members of the 2003 Balloting Group that approved the standard for submis-sion to the IEEE Standards Board:Andy R. Bihain James F. Leathrum William R. Smith, JrAnthony Cincotta Kevin

20、Lewis Keith StobieDonald Cragun Kathy Liburdy James G. TannerMichel Gien Shane P. McCarron Kenneth G. ThomasBarry Hedquist John S. Meckley Mark-Rene UchidaLowell Johnson Dave Plauger Bruce WeinerLawrence J. Kilgallen Gerald Powell Alex WhiteMartin J. Kirk Paul Rabin X/OPEN CO LTDThomas M. Kurihara T

21、homas Shem Oren YuenISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E)When the IEEE Standards Board approved this standard on 9 December 1997, ithad the following membership:Donald C. Loughry, Chair Richard J. Holleman, Vice ChairAndrew G. Salem, SecretaryClyde R. Camp Lowell Johnson Louis-Francois PauSt

22、ephen L. Diamond Robert Kennelly Gerald H. PetersonHarold E. Epstein E. G. Al Kiener John W. PopeDonald C. Fleckenstein Joseph L. Koepfinger* Jose R. RamosJay Forster* Stephen R. Lambert Ronald H. ReimerThomas F. Garrity Lawrence V. McCall Ingo RuschDonald N. Heirman L. Bruce McClung John S. RyanJim

23、 Isaak Marco W. Migliaro Chee Kiow TanBen C. Johnson Howard L. Wolfman* Member EmeritusAlso included are the following nonvoting IEEE Standards Board liaisons:Satish K. AggarwalAlan H. CooksonNoelle HumenickIEEE Standards Project Editorvirururururururururururururururururururururururururururururururu

24、rurururururururururururururururururururururururururururururururururururururururururururururuINTERNATIONAL STANDARD ISO/IEC 13210: 1998 (E)rurururururururururururururururururururururururururururururururururururururururururururururururururururururururururururururururururururururuInformation technology

25、Requirements andGuidelines for Test Methods Specificationsand Test Method Implementations forMeasuring Conformance to POSIX StandardsSection 1: General1.1 ScopeThis International Standard is applicable to the development and use of confor-mance test method specifications for POSIX standards and may

26、be applicable toother application programming interface specifications. This International Stan-dard is intended for developers and users of test method specifications and testmethod implementations.The users of this standard include Assertion Writers: to format assertions Assertion Test Writers: to

27、 write assertion tests Test Suite or System Procurers: to interpret the results of test suitesThe purpose of this standard is to define requirements and guidelines for develop-ing assertions and related test methods for measuring conformance of an imple-mentation under test (IUT)toPOSIX standards. T

28、est method implementationsmay include Conformance Test Software (CTS), POSIX Conformance Test Pro-cedures (CTP), and audits of Conformance Documents (CD).Testing conformance of an implementation to a standard includes testing thecapabilities and behavior of the implementation with respect to the con

29、formancerequirements of the standard. Test methods are intended to provide a reasonable,practical assurance that the implementation conforms to the standard. Use of1.1 Scope 1ISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E) TEST METHODS SPECIFICATIONS AND IMPLEMENTATIONS FORthese test methods will not

30、guarantee conformance of an implementation to thestandard; that normally would require exhaustive testing (see 7.2.1), which isimpractical for both technical and economic reasons.1.2 References1.2.1 Normative ReferencesThe following standards contain provisions that, through references in this text,

31、constitute provisions of this standard. At the time of publication, the editionsindicated were valid. All standards are subject to revision, and parties to agree-ments based on this standard are encouraged to investigate the possibility ofapplying the most recent editions of the standards listed bel

32、ow.1 IEEE Std 729-1983,1)IEEE Glossary of Software Engineering Terminology(ANSI).1.2.2 Informative ReferencesSeveral of the terms defined in 2.2.2, General Terms, have corresponding counter-parts that are used in the international community for conformity assessmentpurposes. The references cited her

33、e use terminology and concepts that correlateto some of the terminology used in this standard.2 ISO/IEC 9646-1:1994, Information technologyOpen Systems Interconnec-tionConformance testing methodology and frameworkPart 1, Generalconcepts.3 ISO/IEC Guide 25:1990, General Requirements for the Competenc

34、e and Cali-bration of Testing Laboratories.1.3 Conformance CriteriaTest method specifications and implementations claiming to conform to this stan-dard shall conform to all the requirements contained in this standard. Materialsidentified in this standard as guidelines or as methodology are suggestio

35、ns oroptions, as are all examples, notes, and footnotes. All statements containing theterm shall are requirements, as are statements where shall is obviously implied.rurururururururururururururururu1) IEEE documents can be obtained from the The Institute of Electrical and Electronics Engineers,Inc.,

36、 445 Hoes Lane, PO Box 1331, Piscataway, New Jersey 08855-1331, USA.2 1 GeneralISO/IEC 13210: 1998 (E)MEASURING CONFORMANCE TO POSIX STANDARDS IEEE Std 2003-1998 (E)1.3.1 Test Method Specification Conformance CriteriaA test method specification conforming to this standard shall include all the fol-l

37、owing criteria: It shall use the required assertion definitions, types, syntax, and constructsspecified in this standard as applicable (see Section 3). It shall use the test result codes specified in this standard for test resultsdefined by this standard (see Section 4). It shall define any test met

38、hod specification structures and test result codesit uses in addition to those defined by this standard. It shall specify a list of conforming test result codes that indicate confor-mance to the specifications being tested (see 4.3.1).A conforming test method specification may specify other construc

39、ts not specifiedin this standard.1.3.2 Test Method Implementation Conformance CriteriaTest method implementations that conform to this standard shall include all thefollowing criteria: A test method implementation conforming to this standard shall documentthat it conforms to this standard and shall

40、document any other test methodspecifications to which it claims to conform. The test method implementation shall test each instance for an assertionthat specifies a set of instances often separated by the word or. The CTS shall be automated to the maximum extent possible. The documentation of the CT

41、S shall include the following: All information necessary to install, configure, and execute the CTS The edition of the standard being tested An overview of the CTS and the optional base standard featurestested by the CTS Deviances from the recommendations defined within this standard Known limitatio

42、ns of the CTS Release-specific changes Configuration-specific parameters Development system hardware requirements such as memory, diskspace, terminal, and printer needs Development system software requirements Hardware and software requirements necessary to execute the CTS A description of any CTS t

43、rouble-clearing procedures1.3 Conformance Criteria 3ISO/IEC 13210: 1998 (E)IEEE Std 2003-1998 (E) TEST METHODS SPECIFICATIONS AND IMPLEMENTATIONS FOR The documentation of the test method implementations shall include, ifneeded, instructions for performing the CTP and an audit of the CD. The document

44、ation of the test method implementations shall describe howto gather and interpret the results. Additional criteria may be specified in applicable standards.When no test method specification exists, test method implementations shallderive the test method specifications from the base standards and th

45、e test methodimplementation shall then meet all the criteria listed above.1.4 IUT Conformance AssessmentFigure 1-1 depicts the basic model for conformance assessment for an IUT where atest method standard is derived from an existing base standard by using thisstandard. The test method standard shall

46、 identify the test result codes that arerequired for conformance and provides the assertions from which test methodimplementations are derived. When a test method implementation is executed, itprovides intermediate test result codes that must be resolved to final test resultcodes. When the final tes

47、t result codes match the conformance test result codes,the implementation is judged to be conforming.Note: This process does not include every potential step that may be required bya certifying body, such as a conformance document audit. It addresses only therelationship between the conforming test

48、result codes identified in the testmethod standard and the final test result codes derived from the test methodimplementation.4 1 GeneralMEASURING CONFORMANCE TO POSIX STANDARDS IEEE Std 2003-19981.4 IUT Conformance Assessment5Figure 1-1 Single Base StandardISO/IEC 13210: 1998 (E)IEEE Std 2003-1998

49、E)Section 2: Definitions and General Requirements2.1 ConventionsThis International Standard uses the following typographical conventions:(1) The italic font is used for The initial appearances of defined terms Cross-references to defined terms in 2.2.1, 2.2.2, and 2.2.3 Parameters (option arguments and operands) that generally are sub-stituted with real values by the application C language data types and function names Global external variable names The elements trailing base standard subclause reference(2) The bold font