1、 International Telecommunication Union ITU-T Series XTELECOMMUNICATION STANDARDIZATION SECTOR OF ITU Supplement 5(09/2008) SERIES X: DATA NETWORKS, OPEN SYSTEM COMMUNICATIONS AND SECURITY ITU-T X.290-series Supplement on interoperability testing framework and methodology ITU-T X-series Recommendatio
2、ns Supplement 5 ITU-T X-SERIES RECOMMENDATIONS DATA NETWORKS, OPEN SYSTEM COMMUNICATIONS AND SECURITY PUBLIC DATA NETWORKS X.1X.199 OPEN SYSTEMS INTERCONNECTION X.200X.299 INTERWORKING BETWEEN NETWORKS X.300X.399 MESSAGE HANDLING SYSTEMS X.400X.499DIRECTORY X.500X.599 OSI NETWORKING AND SYSTEM ASPEC
3、TS X.600X.699 OSI MANAGEMENT X.700X.799 SECURITY X.800X.849 OSI APPLICATIONS X.850X.899 OPEN DISTRIBUTED PROCESSING X.900X.999 INFORMATION AND NETWORK SECURITY General security aspects X.1000X.1029 Network security X.1030X.1049 Security management X.1050X.1069 Telebiometrics X.1080X.1099 SECURE APPL
4、ICATIONS AND SERVICES Multicast security X.1100X.1109 Home network security X.1110X.1119 Mobile security X.1120X.1139 Web security X.1140X.1149 Security protocols X.1150X.1159 Peer-to-peer security X.1160X.1169 Networked ID security X.1170X.1179 IPTV security X.1180X.1199 CYBERSPACE SECURITY Cyberse
5、curity X.1200X.1229 Countering spam X.1230X.1249 Identity management X.1250X.1279 SECURE APPLICATIONS AND SERVICES Emergency communications X.1300X.1309 Ubiquitous sensor network security X.1310X.1339 For further details, please refer to the list of ITU-T Recommendations. X series Supplement 5 (09/2
6、008) i Supplement 5 to ITU-T X-series Recommendations ITU-T X.290-series Supplement on interoperability testing framework and methodology Summary Supplement 5 to ITU-T X.290-series Recommendations defines the principles, methodology and architectures to serve as a foundation for interoperability tes
7、ting and the development of interoperability test suites. It is applicable to interoperability testing of protocols and also to interoperability testing of all areas of software. Source Supplement 5 to ITU-T X-series Recommendations was agreed on 19 September 2008 by ITU-T Study Group 17 (2005-2008)
8、. ii X series Supplement 5 (09/2008) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ
9、of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the top
10、ics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are pre
11、pared on a collaborative basis with ISO and IEC. NOTE In this publication, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this publication is voluntary. However, the publication may contai
12、n certain mandatory provisions (to ensure e.g., interoperability or applicability) and compliance with the publication is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requi
13、rements. The use of such words does not suggest that compliance with the publication is required of any party. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of this publication may involve the use of a claimed Intellectual Property Right. ITU
14、 takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the publication development process. As of the date of approval of this publication, ITU had not received notice of intellectual propert
15、y, protected by patents, which may be required to implement this publication. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2009 All rights reserved. No p
16、art of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. X series Supplement 5 (09/2008) iii CONTENTS Page 1 Scope 1 2 References. 1 3 Definitions 1 4 Abbreviations and acronyms 2 5 General 3 5.1 Interoperability . 3 5.2 Interoperability testin
17、g 4 5.3 Overview of the interoperability testing framework and methodology . 4 6 Interoperability testing framework . 4 6.1 Interface, entity, association . 4 6.2 Interoperation architecture 6 6.3 Interoperability test architecture. 7 6.4 Interoperability test case. 8 6.5 Interoperability test suite
18、 9 6.6 Test execution and verdict assignment. 9 7 Interoperability testing methodology 10 7.1 Interoperability test suite development 10 7.2 Interoperability testing procedure. 11 Bibliography. 13 iv X series Supplement 5 (09/2008) Introduction The need for interoperability testing arises when it is
19、 required to demonstrate that two or more systems or system components are capable of communicating with each other. Conformance testing (CT) checks whether an implementation is implemented correctly with respect to the requirements stated in the relevant standard or specification. A framework and m
20、ethodology for conformance testing is specified in the ITU-T X.290-series Recommendations. By itself, it has two important limitations in ensuring interoperation of communicating systems. First, specifications may contain non-interoperable components such as optional features. Second, complete confo
21、rmance testing may be impractical due to the large number of tests that may be required to cover all the requirements. Therefore, interoperability testing (IOPT) on multiple implementations is needed to complement conformance testing. This supplement is intended to assist ITU-T Study Groups responsi
22、ble for developing protocol specifications, conformance test suites and interoperability test suites by defining a testing methodology that includes a framework for test purposes, a test architecture and test methods for use in interoperability testing. It is also intended as a guide to the develope
23、rs of interoperability test suites, test tool developers and test operators, with the objective of achieving a common framework for testing. Specification writers would understand non-interoperation phenomena and their causes better so that they can be more informed and prepared to write interoperab
24、le specifications. System developers would be helped in developing interoperable systems and services. Interoperability test suites developers would know how to write practically complete interoperability test suites (i.e., they would know what needs to be tested and would also know the gap between
25、what should be tested and what can be tested). Test operators would get help in understanding the causes of interoperability problems and in finding out how to fix problems. This supplement is complementary to, and may be used in conjunction with, testing methodologies and test specification languag
26、es defined in related Recommendations such as: ITU-T X.290 through ITU-T X.296, and ITU-T Z.161 through ITU-T Z.163, where the X.290-X.296 series addresses conformance and provides the methodology and framework for conformance, and the Z.161-Z.163 series defines a general test scenarios description
27、language for conformance testing and interoperability testing. A particularly related document in terms of its technical contents is Supplement 4 to ITU-T X-series Recommendations: ITU-T X.290-series: Supplement on generic approach to interoperability testing, which provides general guidance on the
28、specification and execution of interoperability tests for communication systems in next generation networks (NGN). It is meant to be applied to communication equipment interoperability testing and in many places gives specific guidance on how test suites can be written and interoperability testing c
29、an be carried out. On the other hand, this Supplement 5 to the ITU-T X.290-series of Recommendations targets at more foundational principles and issues that underlie specific interoperability testing approaches and on the basis of which interoperability testing approaches can be carried out and asse
30、ssed. In addition, Recommendations concerning ITU-T formal languages such as ASN.1, MSC, SDL and UML can be used in relation with this supplement, for test data description, system behaviours and test scenarios descriptions, communication system modelling, and software modelling, respectively. X ser
31、ies Supplement 5 (09/2008) 1 Supplement 5 to ITU-T X-series Recommendations ITU-T X.290-series Supplement on interoperability testing framework and methodology 1 Scope This supplement provides a generic framework and methodology for interoperability testing of interconnected communication systems. T
32、he framework is complementary to, and may be used in conjunction with, testing methodologies and test specification languages defined in related Recommendations. This supplement is also intended to assist study groups responsible for developing protocol specifications and conformance test suites. Th
33、is methodology is applicable to interoperability testing of protocols and also to interoperability testing of all areas of software. It is also intended as a guide to the developers of interoperability test suites, test tool developers and test operators, with the objective of achieving a common fra
34、mework for testing. 2 References ITU-T X.290 Recommendation ITU-T X.290 (1995), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications General concepts. ITU-T X.291 Recommendation ITU-T X.291 (1995), OSI conformance testing methodology and framework for
35、 protocol Recommendations for ITU-T applications Abstract test suite specification. ITU-T X.292 Recommendation ITU-T X.292 (2002), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications The Tree and Tabular Combined Notation (TTCN). ITU-T X.293 Recommen
36、dation ITU-T X.293 (1995), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications Test realization. ITU-T X.294 Recommendation ITU-T X.294 (1995), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications Requ
37、irements on test laboratories and clients for the conformance assessment process. ITU-T X.295 Recommendation ITU-T X.295 (1995), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications Protocol profile test specification. ITU-T X.296 Recommendation ITU-T
38、 X.296 (1995), OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications Implementation conformance statements. 3 Definitions This supplement defines the following terms: 3.1 application programming interface (API): Allows a software application to make us
39、e of another piece of software through well-defined software functions and procedures with fully specified parameters. 2 X series Supplement 5 (09/2008) 3.2 conformance testing (CT): A service, offered to clients by a test laboratory, to perform the conformance assessment process for one or more pro
40、tocols, with a choice of test methods sufficient to make the service applicable to all real systems that claim to implement the specified protocols. 3.3 interoperability (IOP): The degree to which a set of entities can collaborate using a predetermined set of functions. 3.4 implementation under test
41、 (IUT): The subdivision of a system under test that is to be tested, such as, the specific layer(s) of protocol within a set of layers of protocol. 3.5 interoperability of communication systems: Ability of systems to interoperate using the relevant communication standard(s). 3.6 interoperability tes
42、ting (IOPT): Activity of demonstrating that two or more entities, such as systems, equipment and software components, interoperate. 3.7 IOPT test case (IOPT-TC): A test case that tests interoperability of a set of entities. 3.8 IOPT test report (IOPT-TR): A document produced at the end of an interop
43、erability testing carried out using a particular test suite. It provides the details of the test results by listing the test cases of the test suite that were executed, their execution scenarios and the verdicts assigned as the results of execution. 3.9 IOPT test suite (IOPT-TS): A set of interopera
44、bility test cases. 3.10 protocol data unit (PDU): An atomic unit of data specified in an (N)-layer protocol, consisting of (N)-protocol information and possibly (N)-user data. 3.11 protocol implementation conformance statement (PICS): A statement made by the supplier of an implementation or system,
45、stating which capabilities have been implemented for a given protocol. 3.12 service access point (SAP): A physical interface of a component through which services provided at the interface are utilized. 3.13 test campaign: The process of executing an interoperability test suite against two IUTs. 3.1
46、4 test purpose (TP): A concise statement of the purpose of an interoperability test to be conducted on a pair of PUTs (or IUTs), in terms of the functionality of the relevant OSI application (profile or protocol), that is also independent of the PUTs (or IUTs) themselves. 3.15 test report (TR): A do
47、cument which uniquely and fully identifies the conditions of an interoperability test campaign and the identity of the tests executed and presents the verdicts obtained for each test. 3.16 test specification: An amplification of a test purpose defining the test in more detail in terms of the OSI ope
48、rations to be performed. The specification includes information on initial conditions for the test, expected behaviour and final conditions, together with criteria for judging Pass, Fail and Inconclusive test verdicts. 4 Abbreviations and acronyms This supplement uses the following abbreviations and
49、 acronyms: API Application Programming Interface ATM Asynchronous Transfer Mode CT Conformance Testing CT-TS CT Test Suite CTMF Conformance Testing Methodology and Framework X series Supplement 5 (09/2008) 3 FSM Finite State Machine ICS Implementation Conformance Statement IOP Interoperability IOPT Interoperability Testing IOPT-TC IOPT Test Case IOPT-TR IOPT Test Report IOPT-TS IOPT Test Suite IUT Implementation Under Test IXIT Implementation eXtra Information for Testing LT Lower Tester MOT Means
