1、 ETSI TR 102 840 V1.2.1 (2011-02)Technical Report Methods for Testing and Specifications (MTS);Model-based testing in standardisationETSI ETSI TR 102 840 V1.2.1 (2011-02) 2Reference RTR/MTS-00106ed121-ModDrivTe Keywords conformance, interoperability, methodology, testing ETSI 650 Route des Lucioles
2、F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice Individual copies of the present document can be downloaded from: http:/www
3、.etsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF). In case of dispute, the reference shall be the printin
4、g on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/por
5、tal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced except as authorized by written permission. The copyright and the
6、foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2011. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTM, TIPHONTM, the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM is a Trade Mark of
7、ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. LTE is a Trade Mark of ETSI currently being registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETS
8、I ETSI TR 102 840 V1.2.1 (2011-02) 3Contents Intellectual Property Rights 5g3Foreword . 5g3Introduction 5g31 Scope 6g32 References 6g32.1 Normative references . 6g32.2 Informative references 6g33 Definitions and abbreviations . 6g33.1 Definitions 6g33.2 Abbreviations . 7g34 Standardized test specifi
9、cation development at ETSI 7g35 Test generation from system models 9g35.1 Modelling . 10g35.2 Test case construction 10g35.3 Test asset generation 10g35.4 Requirements traceability . 11g36 Recommendations on languages and tools . 11g36.1 Modelling language support . 11g36.1.1 Standardized languages.
10、 11g36.1.2 Widely used languages . 11g36.1.3 Model exchange 11g36.1.4 Availability of multiple tools 11g36.2 Modelling language features 12g36.2.1 Expressivity 12g36.2.2 Modelling of underspecified features . 12g36.2.3 Modelling of optional and conditional features 12g36.2.4 Expressing test configur
11、ations 12g36.2.5 Requirement annotations 13g36.3 General tool requirements 13g36.3.1 General availability. 13g36.3.2 Platform support . 13g36.3.3 Model import and export 13g36.4 Test case generation, selection and organization 13g36.4.1 Requirements traceability . 13g36.4.2 Minimal coverage of new r
12、equirements per test case . 14g36.4.3 Identification of preambles and postambles 14g36.4.4 Preamble and postamble selection 14g36.4.5 Test case dependencies . 14g36.4.6 Test case ordering . 14g36.4.7 Test cases ending in the starting state . 14g36.4.8 Support for testing underspecified behaviour . 1
13、5g36.4.9 Support for generating undefined inputs . 15g36.4.10 Conformance testing . 15g36.4.11 Interoperability testing 15g36.5 Generation of test descriptions . 15g36.6 Generation of executable TTCN-3 tests . 16g36.6.1 Syntax . 16g36.6.2 Structure 17g36.6.3 Behaviour 17g36.6.4 Configurability and l
14、ate binding . 17g37 Recommended methodology 18g3ETSI ETSI TR 102 840 V1.2.1 (2011-02) 47.1 Using model-based testing in a test design project . 18g37.2 Requirements traceability . 18g37.3 What to generate . 18g37.4 Selection of tools 18g37.5 Required skill set 18g37.6 Normative and non-normative mod
15、els . 19g37.7 Normative and non-normative tests 19g37.8 Reviewing procedures 19g37.9 Maintenance . 19g37.10 Taking model-based testing into use 20g37.11 Ceasing use of model-based testing 20g38 Conclusions 20g3Annex A (informative): Bibliography . 21g3History 22g3ETSI ETSI TR 102 840 V1.2.1 (2011-02
16、) 5Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Prope
17、rty Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investigation, incl
18、uding IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Report (TR) has been
19、 produced by ETSI Technical Committee Methods for Testing and Specification (MTS). Introduction Carried out both within STFs and the Center for Testing and Interoperability (CTI), test specification and test creation for standardized systems is an important part of ETSIs mission today. In order to m
20、ake the best tools and methods available to the Members, the technical committee on Methods for Testing and Specification (MTS) is continuously aiming, among others, to provide methodologies for the specification of standardized tests including formal definition languages as well as the generation,
21、processing and verification of test suites. As part of this pursuit, MTS has created a work item to investigate the use of model-based testing as a complementary method for test design and test creation as well as to collect requirements on and recommendations related to this methodology, especially
22、 pertaining to the standardization context. The rationale for this work item is that test creation is a resource-intensive and costly process, and that it is in the best interest of the Members to streamline and improve this process if possible. Model-based testing, a methodology and family of techn
23、ologies for the automatic derivation of test descriptions and test cases from system models, has been taken successfully into use by early movers in industry verticals such as data- and telecommunications infrastructure, and has been reported to have realistic potential for significantly reducing th
24、e cost of test design and increasing the quality of test specifications. ETSI has a long history in using various formal and semi-formal languages for modelling and specifying both systems and test systems, including but not limited to SDL, TTCN-3, and TPLan. Whereas the model-based testing methodol
25、ogy is not tied to any particular modelling notation (such as SDL or UML), nor to any output format (English, MSCs, TPLan or TTCN-3), this history provides a solid foundation for potential deployment of model-based test generation in the future. The present document describes shortly in clause 4 the
26、 present manual process for test specification, and gives in clause 5 an overview on the model-based testing with system models. Clause 6 lists then requirements that stem from the standardization context, but in the form of recommendations, for modelling languages and tool chains used to potentiall
27、y enable model-based testing from system models in the ETSI context. Clause 7 provides (early) recommendations for process-level pragmatic methodology that have been derived from industrial experience and adapted to the ETSI context. The present document is not normative and is provided for informat
28、ional purposes only, with the hope that it provides a useful foundation for discussion, and hopefully in the future also for the definition and adoption of a methodology that will positively add to ETSIs capability to serve its Members in the test specification sphere in the early 21stcentury. ETSI
29、ETSI TR 102 840 V1.2.1 (2011-02) 61 Scope The present document presents a collection of recommendations for applying system model-based test generation in a standardization context, especially within ETSI. 2 References References are either specific (identified by date of publication and/or edition
30、number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location
31、 might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity. 2.1 Normative references The following referenced documents are necessary for the application of the present do
32、cument. Not applicable. 2.2 Informative references The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ETSI EG 202 237: “Methods for Testing and Specification (MTS); Internet Protocol
33、Testing (IPT); Generic approach to interoperability testing“. i.2 ISO/IEC 9646-1: “Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 1: General concepts“. 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present docume
34、nt, the following terms and definitions apply: Implementation Conformance Statement (ICS): statement made by the supplier of an IUT claimed to conform to a given specification, stating which capabilities have been implemented Implementation eXtra Information for Testing (IXIT): statement made by a s
35、upplier of an IUT which contains or references all of the information related to the IUT and its testing environment, which will enable the test laboratory to run an appropriate test suite against the IUT Implementation Under Test (IUT): See ISO/IEC 9646-1 i.2. model-based testing: umbrella of appro
36、aches that generate tests from models system model: computer-readable behavioural model that describes the intended external operational characteristics of a system, i.e. how the system being modelled interacts with its environment System Under Test (SUT): See ISO/IEC 9646-1 i.2. ETSI ETSI TR 102 84
37、0 V1.2.1 (2011-02) 73.2 Abbreviations For the purposes of the present document, the following abbreviations apply: CSCF Call Session Control Function CTI Center for Testing and Interoperability DNS Domain Name Server ENUM E.164 Number Mapping ICS Implementation Conformance Statement I-CSCF Interroga
38、ting-CSCF IFS Interoperable Functions Statement IMS IP Media Subsystem IP Internet Protocol IUT Implementation Under Test IXIT Implementation eXtra Information for Testing MTS Methods for Testing and Specification P-CSCF Proxy-CSCF PDU Protocol Data Unit S-CSCF Serving-CSCF SDL Specification and Des
39、cription Language SIP Session Initiation Protocol SUT System Under Test TC Test Cases TCI TTCN-3 Control Interface TCL Tool Command Language TD Test Descriptions TP Test PurposesTPLan Test Purpose Language TRI TTCN-3 Runtime Interface TSS Test Group Structure TTCN Testing and Test Control Notation U
40、ML Unified Modelling Language URI Uniform Record Identifier XML eXtensible Markup Language 4 Standardized test specification development at ETSI Next to producing standardized base specifications for a variety of technologies, ETSI also has a strong reputation in producing standardized test specific
41、ations. As specified in EG 202 237 i.1, these test specifications can serve one of two purposes: they can either help to assess if an implementation conforms to a standard, i.e. conformance test specifications, or they can be used to assess if two or more implementations interoperate properly with e
42、ach other, i.e. interoperability test specifications. ETSI ETSI TR 102 840 V1.2.1 (2011-02) 8In the development of both conformance and interoperability test specifications, ETSI has traditionally followed a stepwise approach based on the methodology defined in ISO/IEC 9646-1 i.2 and resulting in a
43、number of different test specification deliverables. Figure 1 illustrates this approach. These steps can be understood as different levels of abstraction that bridge the large intellectual gap between a base specification and the final executable test suite. They not only form an essential framework
44、 for test specification but also enable a common understanding of the complete test specification for different target audiences, e.g. standardization experts, technology experts, managers, and test engineers. Figure 1: ETSI (Test) Specification Development In the first step, requirements are identi
45、fied from one or more base specifications. These may be catalogued and published in a Requirements Catalogue. Then the Implementation Conformance or Interoperable Functions Statement (ICS/IFS) is constructed. These both are essentially high level check lists for features and capabilities in a standa
46、rd. The check lists are filled out by system vendors, according to which features they implement in their products. This information can help to determine if two implementations of the same standard have potential to interoperate. In the next step one or more test purposes are specified for each tes
47、table identified requirement, either in English prose, or in the TPLan notation TPLan. A test purpose formulates (an aspect of) a requirement as a set of IUT pre-conditions, stimuli, and responses, and specifies test verdict criteria. The testability of a requirement is affected by the type of testi
48、ng to be carried out, e.g. requirements related to error management cannot be assessed using interoperability tests because many error conditions cannot be triggered by conforming implementations. After the definition of test purposes, an informal test description can be specified in English prose,
49、as tables, or as message sequences covering usually one, but sometimes multiple test purposes. Test descriptions extend test purposes by providing more detailed information of preambles and postambles. Test descriptions are by definition not executable. The preambles and postambles in test descriptions are not conceptually the focus of testing, even though they in practice depend on identified requirements in the same way as test bodies do. Therefore, conventionally preambles tend to be specified to invoke behaviour that is most likely to be correctly implem
