1、 ETSI TR 103 168 V1.1.1 (2011-02)Technical Report Methods for Testing and Specifications (MTS);Application of Model-Based Testing in the Telecoms DomainETSI ETSI TR 103 168 V1.1.1 (2011-02) 2Reference DTR/MTS-00125 MBT Keywords application, methodology, report, testing ETSI 650 Route des Lucioles F-
2、06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice Individual copies of the present document can be downloaded from: http:/www.e
3、tsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF). In case of dispute, the reference shall be the printing
4、on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/porta
5、l.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced except as authorized by written permission. The copyright and the fo
6、regoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2011. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTM, TIPHONTM, the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM is a Trade Mark of ET
7、SI registered for the benefit of its Members and of the 3GPP Organizational Partners. LTE is a Trade Mark of ETSI currently being registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI
8、ETSI TR 103 168 V1.1.1 (2011-02) 3Contents Intellectual Property Rights 4g3Foreword . 4g31 Scope 5g32 References 5g32.1 Normative references . 5g32.2 Informative references 5g33 Abbreviations . 6g34 Case study introduction 7g34.1 System Under Test . 7g34.2 Requirements 8g34.3 Process and tool-chain
9、8g35 Modelling the system . 9g35.1 Modelling competence . 9g35.1.1 Use of system models . 10g35.2 Tool support . 10g36 Test generation . 11g37 Test execution 11g38 Test analysis . 12g39 Tool integration 12g39.1 Lack of common exchange format . 12g39.2 On-line mode 13g39.3 Offline mode 13g310 Special
10、 issues 14g310.1 Model-based testing and telecommunication standards . 14g310.2 Coverage. 14g3History 15g3ETSI ETSI TR 103 168 V1.1.1 (2011-02) 4Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to
11、these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Lat
12、est updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the update
13、s on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Report (TR) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS). ETSI ETSI TR 103 168 V1.1.1 (2011-02) 51 Scope The present document reports
14、 on the application of model-based testing in the telecommunication domain. A relevant case study is briefly described in terms of system under test, applied tool chain, together with an overview of the technical requirements. The case study was conducted as part of ITEA2 i.1 D-MINT project i.2. The
15、 document concentrates on the results and conclusions from this work, giving an insight into how applicable such methods are today for testing and indicating the current strengths and weaknesses. 2 References References are either specific (identified by date of publication and/or edition number or
16、version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be
17、found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity. 2.1 Normative references The following referenced documents are necessary for the application of the present document. No
18、t applicable. 2.2 Informative references The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ITEA2 web site. NOTE: Available at http:/www.itea2.org; August 2010. i.2 D-MINT web site. N
19、OTE: Available at http:/www.d-mint.org; August 2010. i.3 Object Management Group; Systems Modeling Language; Version 1.1; November 2008. i.4 Object Management Group; Unified Modeling Language (UML) Infrastructure; Version 2.1.2; November 2007. i.5 ETSI TS 123 002: “Digital cellular telecommunication
20、s system (Phase 2+); Universal Mobile Telecommunications System (UMTS); Network architecture (3GPP TS 23.002 version 6.10.0 Release 6)“. i.6 Heikki Kaaranen, Ari Ahtiainen, Lauri Laitinen, Siamk Naghian, and Valtteri Niemi, editors. UMTS Networks, 2nd Edition. John Wiley Object Constraint Language (
21、OCL); Version 2.0; May 2006. i.8 S. R. Dalal, A. Jain, N. Karunanithi, J. M. Leaton, C. M. Lott, G. C. Patton, and B. M. Horowitz. Model-based testing in practice. In ICSE 99: Proceedings of the 21st international conference on Software engineering, pages 285-294. IEEE Computer Society Press, 1999.
22、ETSI ETSI TR 103 168 V1.1.1 (2011-02) 6i.9 Wolfgang Prenninger, Mohammad El-Ramly, and Marc Horstmann. Model-Based Testing of Reactive Systems, chapter Case Studies. Number 3472 in Advance Lectures of Computer Science. Springer, 2005. i.10 ETSI TS 124 008: “Digital cellular telecommunications system
23、 (Phase 2+); Universal Mobile Telecommunications System (UMTS); Mobile radio interface Layer 3 specification; Core network protocols; Stage 3 (3GPP TS 24.008 version 6.10.0 Release 6)“. i.11 ISO/IEC 9646-1: 1984, Information technology - Open Systems Interconnection - Conformance testing methodology
24、 and framework - Part 1: General concepts. i.12 ETSI ES 201 873-5: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI)“. i.13 Malik, Q.A.; Jaaskelainen, A.; Virtanen, H.; Katara, M.; Abbors, F.; Truscan, D.; Lilius, J.
25、; Model-Based Testing Using System vs. Test Models - What Is the Difference?; 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS); 2010; 2010; pages: 291 - 299. i.14 Abbors, F.; Backlund, A.; Truscan, D.; MATERA - An Integrated Framework for Model-Based T
26、esting; 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS); 2010; pages: 321 - 328. i.15 Abbors Johan; Increasing the Quality of UML Models Used for Automatic Test Generation; Embedded Systems Laboratory, Faculty of Technology, bo Akademi University; Mas
27、ters Thesis; 2009. i.16 ITU-T RECOMMENDATION X.680; Information technology - Abstract Syntax Notation One (ASN.1): Specification of basic notation. i.17 ETSI ES 201 873-7: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 7: Using ASN.1 with TTCN-3“.
28、 i.18 Thomas Dei, Andreas J. Nyberg, Stephan Schulz, Risto Teittinen, Colin Willcock; Industrial Deployment of the TTCN-3 Testing Technology; IEEE Software, July/August 2006, vol. 23, no. 4; pages 48 - 54. i.19 OMG Model Driven Architecture Guide; Version 1.0.1; June 2003. i.20 OMG Meta-Object Facil
29、ity (MOF) Core Specification; Version 2.0; 2006. i.21 Eclipse Modeling Framework project web page. NOTE: Available at http:/www.eclipse.org/modeling/emf/. August 2010. i.22 Tuomas Pjrvi; Generating Input for a Test Design Tool from UML Design Models; Embedded Systems Laboratory; Faculty of Technolog
30、y; bo Akademi University; Masters Thesis; 2009. 3 Abbreviations For the purposes of the present document, the following abbreviations apply: ASN.1 Abstract Syntax Notation, number 1 BSS Base Station Subsystem EMF Eclipse Modelling Framework HLR Home Location Register MBT Model-Based Testing MDA Mode
31、l Driven Architecture MGW Media Gateway MOF Meta-Object Facility MSC Message Sequence Charts OCL Object Constraint LanguageRNS Radio Network Subsystem ETSI ETSI TR 103 168 V1.1.1 (2011-02) 7SDL System Description Language SUT System Under Test SysML Systems Modeling Language TTCN-3 Testing and Test
32、Control Notation version 3 UML Unified Modeling Language 4 Case study introduction The case study is introduced by describing three major aspects. First, the System Under Test (SUT) is described in clause 4.1. Second, the case study requirements are elaborated in clause 4.2. Third, the related proce
33、sses and tools are introduced in clause 4.3. 4.1 System Under Test The System Under Test in the case study is the Mobile Services Switching Centre Server (MSC Server) of 2ndand 3rdgeneration mobile networks. An example configuration of the network architecture is depicted in Figure 1. The MSC Server
34、 is connected via standardized interfaces to a 2ndgeneration, GSM, Base Station Subsystem (BSS), a 3rdgeneration, UMTS, Radio Network Subsystem (RNS), the Home Location Register (HLR) for subscriber data, and the Media Gateway (MGW) transporting the actual user data. The mobile is connected to the B
35、SS or RNS, connection to the MSC Server it using logical links only. The details of the network architecture are specified in TS 123 002 i.5. Evolution from 2nd generation GSM systems to 3rd generation UMTS networks and detailed description of the latter technology are provided by Kaaranen et al. in
36、 i.6. Figure 1: An example configuration of network elements using in the case study ETSI ETSI TR 103 168 V1.1.1 (2011-02) 84.2 Requirements Prior to the case study it was already known that model-based testing was used successfully in several projects in industry. For example Dalal et al. list a nu
37、mber of case studies i.8 and more recent results are explained by Prenninger et al. in i.9. Hence, in this case study focus was on automation aspects on the model-based testing (MBT) and applying MBT in telecommunication domain. The automation aspects were covered by the process description and the
38、tool-chain implementation described in clause 4.3. This aspect was considered also from legacy perspective, i.e. how MBT can be integrated to the existing processes and tools used in telecom product development. Many test generation tools used in MBT domain takes test models as input. In contrast to
39、 such an approach system models were chosen to model behaviour of the system in this case study. Difference of these approaches are explained by Malik et al. in i.13. The selection was made to exploit system models produced earlier in software development process. In addition, the intention was to i
40、nvestigate are the generated tests meaningful and efficient for product development in telecom domain. Yet another aspect set for the case study was to investigate reuse of existing material available in the telecommunication standards, e.g. use of Message Sequence Charts (MSCs), ASN.1 (Abstract Syn
41、tax Notation, number 1) definitions. In first glance automatic test generation may look like the perfect solution for testing as the test generation is able to produce lots of test cases. However, if there are too many test cases, test generation will take too much time. Therefore, it was a signific
42、ant evaluation aspect to find out how long test generation takes, how to control the amount of test cases produced by the test generation, and how long it takes to execute the generated test cases. Finally, use of test generation differs from non-MBT testing due to fact that the generated tests may
43、change significantly after the models are modified and tests are regenerated. In fact, tests may not be comparable between different test generation rounds. This required investigations how to trace progress and coverage of testing. 4.3 Process and tool-chain The SUT is tested using a process depict
44、ed in Figure 2. Four major phases can be identified from the process. First, requirements are processed and described using Systems Modeling Language (SysML) i.3. In addition, the system is described using Unified Modeling Language (UML) i.4 models including references to the SysML requirements. The
45、 models are validated using a set of validation rules in order to improve the quality. Second, tests are generated from the models. The test generation phase produces executable test scripts. Third, the test scripts are executed with help of a test execution system. The execution phase produces test
46、 logs that are used for further analysis. Fourth, tests are analyzed in case of failures and requirement coverage tracing is performed. ValidationreportRequirementsTest reportLogsModellingModel validationRequirement analysisTest evaluationTest generationTest executionSystem Under TestTest casesModel
47、sFigure 2: An overview of the case study process ETSI ETSI TR 103 168 V1.1.1 (2011-02) 9The process is supported by a tool-chain developed in the case study. The tool-chain is depicted in Figure 3. UML models are edited with a model editor. A model validation tool was used to ensure custom rules imp
48、lemented using Object Constraint Language (OCL) i.7. The UML models are transformed with the model validation tool into models in a proprietary UML based modelling language and given as input to the test generation tool. The generation tool produced test scripts that are executed a protocol simulato
49、r. The protocol simulator had the role of a test system. It executed test cases by sending and receiving messages, i.e. using asynchronous communication. The test logs produced by the protocol simulator were analyzed and evaluated against the original models using the test evaluation tool i.14. Figure 3: Description of the case study tool-chain 5 Modelling the system 5.1 Modelling competence Lack of modelling competence prior to the case study did not cause any major issues regarding modelling as suc
