1、STD*ITU-T RECMN INTERNAT ONAL 1997 48b32593 Db44702 858 = TELECOMMUN ITU=T TELECOMMUNICATION STANDARDIZATION SECTOR OF TU CATION UNION 2.500 (05197) SERIES Z: PROGRAMMING LANGUAGES Methods for validation and testing - Framework on formal methods in conformance testing ITU-T Recommendation 2.500 (Pre
2、viously CCITT Recommendation) ITU-T Z-SERIES RECOMMENDATIONS PROGRAMMING LANGUAGES FORMAL DESCRIPTION TECHNIQUES (FDT) Specification and Description Language (SDL) Application of Formal Description Techniques Message Sequence Chart PROGRAMMING LANGUAGES CHILL: The ITU-T high level language MAN-MACHI
3、NE LANGUAGE General principles Basic syntax and dialogue procedures Extended MML for visual display terminals Specification of the man-machine interface QUALITY OF TELECOMMUNICATION SOFTWARE METHODS FOR VALIDATION AND TESTING 2.100-2. 199 2.100-2. 109 2. 1 10-2.1 19 2. 120-2.129 2.200-2.299 2.200-2.
4、209 2.300-2.499 2.300-2.309 2.3 10-2.3 19 2.320-2.329 2.330-2.399 2.400-2.499 2.500-2.599 For further details, please refer to ITU-T List of Recommendations STD-ITU-T RECMN Z.5O-ENGL 1977 m 48b259L Ub1i4704 b20 ITU-T RECOMMENDATION 2.500 FRAMEWORK ON FORMAL METHODS IN CONFORMANCE TESTING Source ITU-
5、T Recommendation 2.500 was prepared by ITU-T Study Group 10 (1997-2000) and was approved under the WTSC Resolution No. 1 procedure on the 6th of May 1997. 1 Recommendation 2.500 (05/97) - FOREWORD IT (International Telecommunication Union) is the United Nations Specialized Agency in the field of tel
6、ecommuni- cations. The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of the ITU. The IT-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The Wo
7、rld Telecommunication Standardization Conference (WTSC), which meets every four years, establishes the topics for study by the ITU-T Study Groups which, in their turn, produce Recommendations on these topics, The approval of Recommendations by the Members of the ITU-T is covered by the procedure lai
8、d down in WTSC Resolution No. 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with IS0 and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommu
9、nication administration and a recognized operating agency. INTELLECTUAL PROPERTY RIGHTS The ITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. The ITU takes no position concerning the evidenc
10、e, validity or applicability of claimed Intellectual Property Rights, whether asserted by I?Tu members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, the ITU had not received notice of intellectual property, protected by patents, which
11、 may be required to implement this Recommendation. However, implementors are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database. O ITU 1998 All rights reserved. No part of this publication may be reproduced or utilized in
12、any form.or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from the ITU. 11 Recommendation 2.500 (05197) STD-ITU-T RECMN Z.500-ENGL 1997 48b257L bYu7Ub 4T3 Sl 1 2 3 4 5 6 7 8 9 CONTENTS Scope 2.1 Conformance testing . 2.2 Formal descriptio
13、n techniques Definitions 3.1 Terms from other related standards 3.2 Terms defined in this Recommendation Abbreviations . Mathematical concepts and notation conventions 5.1 Sets 5.2 Logic . 5.3 Relations . 5.4 Functions The meaning of conformance . 6.1 Introduction 6.2 Specifications 6.3 Implementati
14、ons Conformance of an implementation to a formal specification Normative references . 6.4 Testing concepts . 7.1 Introduction 7.2 Test architecture 7.3 Formal model of the test architecture . . 7.4 Test execution . Conformance testing 8.1 Introduction 8.2 Definition of conformance testing 8.3 Test g
15、eneration 8.4 Test suite size reduction . : 8.5 Fault coverage . 8.6 Test suite cost . Compliance 9.1 9.2 9.3 9.4 Annex A . A . 1 A.? A.3 A.4 A.5 A . 6 A.7 A.8 Introduction Compliance with clause 6: The meaning of conformance Compliance with clause 7: Testing concepts Compliance with clause 8: Confo
16、rmance testing . Specifications . Implementation options and instantiated specifications Implementations and models of implementations . Conformance by implementation relations . Conformance by requirements Test architecture Specifications of tests . . . References Recommendation 2.500 (05/97) Page
17、1 1 1 2 2 2 3 4 5 5 5 6 6 6 6 6 7 8 10 10 10 11 12 14 14 14 14 15 16 16 17 17 17 18 18 18 18 23 28 30 33 34 35 42 . 111 Introduction Many protocol and service specifications are nowadays described in formal notations called Formal Description Techniques (FDTs). Examples of standardized FDTs are SDL,
18、 LOTOS, Estelle and ASN. 1, There also exists a formal notation for test suite specifications: TTCN. The use of FDTs has the following advantages: - they describe the formats and behavious in an unambiguous way; - they give a basis for rigorous validation including conformance testing. Conformance t
19、o a communication protocol or service standard is considered to be a prerequisite for the correct interoperability of open systems. Conformance testing, i.e. the assessment by means of testing whether a product conforms to its specification, is an important issue in product development, because it i
20、ncreases the confidence in correct interoperability . This Recommendation “Framework on Formal Methods in Conformance Testing“ (FMCT) defines the meaning of conformance if formal methods are used for the specification of a communication protocol or service. It is also meant to guide computer aided t
21、est generation. This Recommendation defines a framework for the use of formal methods in conformance testing. It is intended for implementers, testers, and specifiers involved in conformance testing to guide in defining conformance and the testing process of an implementation with respect to a speci
22、fication that is given as a formal description. iv Recommendation 2.500 (05/97) - - STD*ITU-T RECMN Z-500-ENGL 1777 9 48b257L 0b44708 27b Recommendation 2.500 FRAMEWORK ON FORMAL METHODS IN CONFORMANCE TESTING (Geneva, 1997) 1 Scope This Recommendation is applicable where a formal specification of a
23、 communication protocol or service exists, from which a conformance test suite shall be developed. It can guide the manual process as well as the development of tools for computer aided test case generation. This Recommendation defines a framework and does not prescribe any particular test case gene
24、ration method, nor does it prescribe any specific conformance relation between a formal specification and an implementation. It is supplementary to the joint ITU-TDSO standard “Conformance Testing Methodology and Framework“ (CTMF) ISODEC 96461, which is applicable to a wide range of products and spe
25、cifications, including specifications in natural language. FMCT interprets conformance testing concepts in a formal context. 2 Normative references The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recomme
26、ndation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; all users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other refe
27、rences listed below. A list of the currently valid ITU-T Recommendations is regularly published. 2.1 Conformance testing NOTE - The following set of references will be referred to as CTMF is this Recommendation. - ITU-T Recommendation X.290 (1995) (equivalent to ISO/IEC 9646-1 :1994), OSI conformanc
28、e testing methodology andframework for protocol Recommendations for ITU-T applications - General concepts. - ITU-T Recommendation X.291 (1995) (equivalent to ISOIEC 9646-2: 1994), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications - Abstract test su
29、ite specifications. - CCITT Recommendation X.292 (1992) (equivalent to ISO/IEC 9646-3: 1992), OSI conformance testing methodology andjiamework for protocol Recommendations for CCITT applications - The Tree and Tabular Combined Notaion (TTGV). - ITU-T Recommendation X.291 (1995) (equivalent to ISO/IE
30、C 9646-4: 1994), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications - Test realization. - ITU-T Recommendation X.294 (1995) (equivalent to ISO/IEC 9646-5: 1994), OSi conformance testing methodology and jiamework for protocol Recommendations for ITU-
31、T applications - Requirements on test laboratories and clients for the conformance assessment process. - ITU-T Recommendation X.295 (1995) (equivalent to ISOIEC 9646-6: 1994), OSI conformance testing methodology and Jiamework for protocol Recornmendations for ITU-T applications - Protocol projle tes
32、t specijcation. - ITU-T Recommendation X.296 (1995) (equivalent to ISOIEC 9646-7: 19953, OSZ conformance testing methodology and framework for protocol Recommendations for TU-T applications - Implementation conformance statements. Recommendation 2.500 (05/97) 1 STD-ITU-T RECMN Z.SOO-ENGL 1777 qBb259
33、L 0b114707 LO2 2.2 Formal description techniques - ITU-T Recommendation Z. 1 00 (1 993), KITT Specrfication and Description Language (SDL). - ITU-T Recommendation Z. 120 (1 996), Message Sequence Charts (MSC). - ISOIIEC 8807:1989, information processing systems - Open Systems Interconnection - LOTOS
34、 - A formal description technique based on the temporal ordering of observational behaviour. - ISOAEC 9074: 1989, informaiion processing systems - Open Systems Interconnection - Estelle: A formal description technique based on an extended state transition model. 3 Definitions 3.1 NOTE -Although the
35、following definitions are given in ITU-T X.290 and ISOAEC 9646-1, they are repeated here because the meanings of these terms are important for the formal interpretations in this Recommendation. Terms from other related standards . 3.1.1 abstract test method: The definition of how an IUT is to be tes
36、ted, given at an appropriate level of abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable tests to be specified for this test method. 3.1.2 perform dynamic conformance testing for one or more protocols. conformance tes
37、t suite: A complete set of test cases, possibly combined into nested test groups, that is selected to 3.1.3 conformance testing: Testing the extent to which an IUT is a conforming implementation. 3.1.4 consistent with the capabilities stated in the ICS(s). conforming implementation: An IUT which sat
38、isfies both static and dynamic conformance requirements, 3.1.5 behaviour is permitted by the relevant specification( s) in instances of communication. dynamic conformance requirement: All those requirements (and options) which determine what observable 3.1.6 fail (verdict): A test verdict given when
39、 the observed test outcome either demonstrates non-conformance with respect to (at least one of) the conformance requirements on which the test purpose of the test case is focused, or contains at least one invalid test event, with respect to the relevant specification(s). 3.1.7 implementation confor
40、mance statement (ICs): A statement made by the supplier of an implementation or system claimed to conform to a given specification, stating which capabilities have been implemented. The ICs can take several forms: protocol ICs, profile ICs, protocol specific ICs, and information object ICs. 3.1.8 re
41、lationship, being that part of the real open system which is to be studied by testing. implementation under test (IUT): An implementation of one or more protocols in an adjacent wedprovider 3.1.9 implementation extra information for testing (IXIT): A statement made by the supplier or implementor of
42、an IUT which contains or references all of the information (in addition to that given in the ICs) related to the IUT and its testing environment, which will enable the testing laboratory to run an appropriate test suite against the IUT. An MT can take several forms: protocol WT, profile EUT, profile
43、 specific IXIT, and information object WT, TMP implementation statement. 3.1.10 means of testing (MOT): The combination of equipment and procedures that can perform the derivation, selection, parameterization and execution of test cases, in conformance with a reference standardized ATS, and can prod
44、uce a conformance log. 2 Recommendation 2.500 (05197) STD-ITU-T RECMN Z.500-ENGL 1997 - 48b259L Ob4471O 924 m 3.1.11 supplied with values in accordance with the appropriate ICS(s) and iXiT(s). 3.1.12 pass (verdict): A test verdict given when the observed test outcome gives evidence of conformance to
45、 the conformance requirement(s) on which the test purpose of the test case is focused, and when no invalid test event has been detected. 3.1.13 is to be controlled and observed, as defined in an Abstract Test Method. 3.1.14 realized. 3.1.15 implemented capabilities permitted in a real open system wh
46、ich is claimed to conform to the relevant specification(s). 3.1.16 requirement or a set of related conformance requirements as specified in the appropriate specification. 3.1.17 conformance of an TUT with respect to that test case when it is executed. parameterized abstract test suite: A selected ab
47、stract test suite in which all relevant parameters have been point of control and observation (PCO): A point within a testing environment where the occurrence of test events reference standardized abstract test suite (ATS): The standardized ATS for which a Means of Testing is static conformance requ
48、irement: One of the requirements that specify the limitations on the combinations of test purpose: A prose description of a well defined objective of testing, focusmg on a single conformance test verdict: A statement of “pass“, “fail“, or “inconclusive“, as specified in an abstract test case, concer
49、ning . 3.2 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5 3.2.6 3.2.7 3.2.8 3.2.9 3.2.10 3.2.11 3.2.12 3.2.13 3.2.14 3.2.15 3.2.16 3.2.17 3.2.18 3.2.19 3.2.20 3.2.21 Terms defined in this Recommendation compatibility: See 6.4.3. complete (test suite): See 8.2. conformance: See 6.4. conformance testing: See 8.2. dynamic conformance: See 6.4.2 and 6.4.3. exhaustive (test suite): See 8.2. fault model: See 8.4.1. fault coverage: See 8.5. formal specification: See 6.2. implementation: See 6.3. implementation access point: See 7.2. implementation access point, (formal model of): See 7.3. impl
