ETSI TR 102 789-2010 Methods for Testing and Specification (MTS) Automated Interoperability Testing Summary of ETSI experiences about using automated interoperability testing tools_1.pdf

1、 ETSI TR 102 789 V1.1.1 (2010-01)Technical Report Methods for Testing and Specification (MTS);Automated Interoperability Testing;Summary of ETSI experiences about using automated interoperability testing toolsETSI ETSI TR 102 789 V1.1.1 (2010-01) 2Reference DTR/MTS-00117 AutoIOP_Expe Keywords intero

2、perability, methodology, testing ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice Individual cop

3、ies of the present document can be downloaded from: http:/www.etsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Format

4、(PDF). In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current stat

5、us of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced excep

6、t as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2010. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTM, TIPHONTM, the TIPHON logo and the ETSI logo are Trade Marks of ETSI register

7、ed for the benefit of its Members. 3GPPTM is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. LTE is a Trade Mark of ETSI currently being registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are

8、 Trade Marks registered and owned by the GSM Association. ETSI ETSI TR 102 789 V1.1.1 (2010-01) 3Contents Intellectual Property Rights 4g3Foreword . 4g31 Scope 5g32 References 5g32.1 Normative references . 5g32.2 Informative references 5g33 Abbreviations . 6g34 Automated interoperability testing . 6

9、g34.1 ETSI Plugtests 7g35 Third ETSI IMS Plugtest Event . 7g35.1 IMS core network NNI interoperability testing 7g35.2 Event Summary 8g35.3 IMS Interoperability Test Automation . 9g35.3.1 IMS Interoperability Test Tool Implementation . 9g35.3.2 Use of the Test Tool 9g35.3.3 Event Feedback on Automate

10、d Interoperability Test Tool . 9g3History 11g3ETSI ETSI TR 102 789 V1.1.1 (2010-01) 4Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI

11、 members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/weba

12、pp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may becom

13、e, essential to the present document. Foreword This Technical Report (TR) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS). ETSI ETSI TR 102 789 V1.1.1 (2010-01) 51 Scope The present document presents a summary of experiences collected from the use automate i

14、nteroperability testing in real-world interoperability testing, e.g. ETSI IMS Plugtests. More specifically it addresses the use of test systems which have been developed based on the methodology and framework for automated interoperability testing for distributed systems i.1. 2 References References

15、 are either specific (identified by date of publication and/or edition number or version number) or non-specific. For a specific reference, subsequent revisions do not apply. Non-specific reference may be made only to a complete document or a part thereof and only in the following cases: - if it is

16、accepted that it will be possible to use all future changes of the referenced document for the purposes of the referring document; - for informative references. Referenced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Referenc

17、e. NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity. 2.1 Normative references The following referenced documents are indispensable for the application of the present document. For dated references, only the editio

18、n cited applies. For non-specific references, the latest edition of the referenced document (including any amendments) applies. Not applicable. 2.2 Informative references The following referenced documents are not essential to the use of the present document but they assist the user with regard to a

19、 particular subject area. For non-specific references, the latest version of the referenced document (including any amendments) applies. i.1 ETSI EG 202 810: “Methods for Testing and Specification (MTS); Automated Interoperability Testing; Methodology and Framework“. i.2 ETSI TS 124 229 (V7.15.0): “

20、Digital cellular telecommunications system (Phase 2+); Universal Mobile Telecommunications System (UMTS); LTE; Internet Protocol (IP) multimedia call control protocol based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP); Stage 3 (3GPP TS 24.229 version 7.15.0 Release 7)“

21、. i.3 IETF RFC 3261: “Session Initiation Protocol“. i.4 ETSI TS 186 011-2 (V2.3.1): “Technical Committee for IMS Network Testing (INT); IMS NNI Interworking Test Specifications; Part 2: Test descriptions for IMS NNI Interworking“. i.5 ETSI ES 201 873-1: “Methods for Testing and Specification (MTS);

22、The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language“. ETSI ETSI TR 102 789 V1.1.1 (2010-01) 6i.6 ETSI EG 202 237: “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); Generic approach to interoperability testing“. i.7 ETSI TR 102 788: “Methods for

23、Testing and Specification (MTS); Automated Interoperability Testing; Specific Architectures“. i.8 ETSI ES 201 873-5: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI)“. i.9 ETSI ES 201 873-6: “Methods for Testing and

24、 Specification (MTS); The Testing and Test Control Notation version 3; Part 6: TTCN-3 Control Interface (TCI)“. i.10 T3DevKit User Manual. NOTE: See http:/t3devkit.gforge.inria.fr/doc/userref/ fetched on 9.11.2009. 3 Abbreviations For the purposes of the present document, the following abbreviations

25、 apply: AS Application Server CSCF Call Session Control Function DNS Domain Name Server EUT Equipment Under Test HSS Home Subscriber Server (HSS) IBCF Interconnection Border Control Function IMS IP Multimedia Subsystem IP Internet Protocol MMTEL Multi-Media Telephony NNI Network-to-Network Interface

26、 PCAP Packet CAPture SIP Session Initiation protocol SUT System Under Test TCI TTCN-3 Control Interface TRI TTCN-3 Runtime Interface TTCN-3 Testing and Test Control Notation 3 UE User Equipment VoIP Voice over IP (VoIP) 4 Automated interoperability testing Interoperability is a key factor in the wid

27、espread commercial success of any given technology in the telecommunication sector. Interoperability fosters diversity as well as competition in a market. Vendors can achieve interoperability of their products only if they agree and implement a common set of open standards. However, standardization

28、does not necessarily lead to interoperability. Standards have to be engineered for interoperability. In its efforts to formalize interoperability testing of distributed systems, ETSI has produced a generic approach to interoperability testing i.6 which is targeted for certification of products by th

29、e means of interoperability testing, as well as a framework and methodology for automating interoperability testing in general i.1, e.g. in the context of ETSI PlugtestsTM. The methodology for automated interoperability testing has been derived from i.6 and defines a means for interoperability testi

30、ng as well as a System Under Test (SUT) based on concepts such as Equipment Under Test (EUT), an interconnecting network, application support nodes, a test coordinator, a test oracle, interface monitors and equipment users. It has been designed to be independent of the technology under test, as well

31、 as the test language used to implement the means of interoperability testing. The document on this methodology also discusses topics such as limitations and feasibility of automation, verdict handling, and controllability of EUT interfaces. More information as well as a detailed explanation of thes

32、e concepts can be found in i.1. ETSI ETSI TR 102 789 V1.1.1 (2010-01) 7The present document presents a summary of the experiences and the knowledge gained during the application of the methodology for automated interoperability testing in an interoperability event organized by ETSI. 4.1 ETSI Plugtes

33、ts PlugtestsTMare interoperability testing events that are organized by ETSI to offer its members as well as non-members a means to assess interoperability between systems and the maturity of standards. These events are attended by vendors that implement systems based on standards and depending on t

34、he event also by observers that include customers of these vendors or research partners. A previously agreed interoperability test specification is the basis for each event. Tests are executed in parallel test sessions where implementations from different vendors are paired with each other and attem

35、pt to execute as many applicable tests as possible in the given test session time limit. The main goal of ETSI Plugtests is the validation of standards: Interoperability issues as well as deviations from standards observed at these events are reported to relevant ETSI technical committees which use

36、this feedback to further improve standards and their interoperability. 5 Third ETSI IMS Plugtest Event 5.1 IMS core network NNI interoperability testing The Internet Protocol (IP) Multimedia Subsystem (IMS) i.2 is an architectural framework for delivering IP multimedia to fixed as well as mobile use

37、rs. It was originally designed by the wireless standards body 3rd Generation Partnership Project (3GPP). It is a major step in the evolution of telecommunication networks beyond Global System for Mobile Communications (GSM) and a convergence with the Internet. Contrary to other conventional telecomm

38、unication frameworks, IMS is based on IP, i.e. the Session Initiation Protocol (SIP) i.3, and access network independent. IMS does not standardise specific applications, but aids the access of multimedia and voice applications across wireless and wireline terminals, i.e. a form of fixed and mobile c

39、onvergence. It isolates application services from the access networks and provides a horizontal control layer for them. IMS core network interoperability testing focuses on the assessment of interoperability of basic services such as basic Voice over IP (VoIP) call and instant messaging between two

40、distinct IMS networks as well as more sophisticated services that require the use of an Application Server (AS), e.g. Multi-Media Telephony (MMTel) and presence. Each IMS core networks consists of Proxy Call Session Control Function (P-CSCF), I(nterrogating)-CSCF, S(erving)-CSCF, Interconnection Bor

41、der Control Function (IBCF), and Home Subscriber Server (HSS) components. In terms of the automated interoperability testing methodology each IMS core network by itself constitutes an Equipment Under Test (EUT) and two connected IMS networks constitute the System Under Test (SUT). External entities

42、such as IMS User Equipment (UE), Application Servers, and Domain Name Servers (DNS) or simulations of such are used to stimulate and observe the interoperability of IMS core networks. While performing interoperability tests traffic on standardized interfaces such as Gm, Mw, Ic, and ISC is captured a

43、nd used to determine if each IMS core network follows the IMS standard when realizing IMS services. The focus in this analysis is the inspection of interfaces between different network elements - also called Network-to-Network Interfaces (NNI) - i.e. Mw, Ic, and ISC interfaces. Figure 1 shows an exa

44、mple test configuration with IMS core networks A and B. Let us assume that the user of UE A has an account in IMS A (i.e. the HSS of IMS A) and the user of UE B has an account in IMS B. An example use case is that the first user - after registering in the network of IMS A - calls with his UE A the s

45、econd user who is already registered with his UE B in the network of IMS B. In this case, initial (IN) SIP requests are exchanged via the S-CSCF of IMS A and the I-CSCF of IMS B. Subsequent (SU) messages are then exchanged between the S-CSCF of IMS A and the S-CSCF of IMS B. Note that although the c

46、onnection is (physically) different, the (logical) IMS NNI reference point is the same, i.e. Mw. ETSI ETSI TR 102 789 V1.1.1 (2010-01) 8Figure 1: Example IMS Core Network NNI test configuration 5.2 Event Summary The third ETSI IMS interoperability event held in Lannion, France from October 16thto 23

47、rd2009 at the Ursulines center. The main focus was the assessment of the interoperability as well as conformance of IMS core networks (composed of P/I/S-CSCF, IBCF, AS (telephony and presence), DNS and HSS) which are implemented on the basis of TS 124 229 i.2 at their NNI. The tests executed at the

48、event were related to basic IMS call functionality, messaging, IMS roaming, topology hiding, MMTEL supplementary services, and the presence service and were taken from the ETSI IMS NNI interoperability test specification TS 186 011-2 i.4. Eight IMS core network vendors participated at this event. Du

49、ring the event 495 of 2 805 potential IMS NNI tests were executed out of which 317 were automatically analyzed for conformance. Overall results show a very high level of interoperability (89 %) of IMS core networks but a lower level of overall conformance to the 3GPP base standard (55 %) in the tests executed. Also note that 13 % of all potential tests could not be executed due to issues outside of the IMS core networks caused, e.g. by a lack of the support of a feature by a participating IMS core network. Tests were executed at matc