ETSI TS 102 950-3-2017 Methods for Testing and Specification (MTS) TTCN-3 Conformance Test Suite Part 3 Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (_1.pdf

1、 ETSI TS 102 950-3 V1.6.1 (2017-08) Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Part 3: Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (IXIT) floppy3TECHNICAL SPECIFICATION ETSI ETSI TS 102 950-3 V1.6.1 (2017-08)2 Reference RTS/MTS-1029505-

2、3ed161 IXIT Keywords ATS, conformance, testing, TTCN ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important n

3、otice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written author

4、ization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should b

5、e aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the followi

6、ng services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF ver

7、sion shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. ETSI 2017. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPT

8、M and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. oneM2M logo is protected for the benefit of its Members GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 102 950-3 V1.6.1 (2017-08)3 Cont

9、ents Intellectual Property Rights 4g3Foreword . 4g3Modal verbs terminology 4g31 Scope 5g32 References 5g32.1 Normative references . 5g32.2 Informative references 6g33 Definitions and abbreviations . 6g33.1 Definitions 6g33.2 Abbreviations . 7g34 Abstract Test Method (ATM) . 7g35 The ATS development

10、process . 8g35.1 Requirements and test purposes . 8g35.2 ATS structure . 8g35.2.1 Test case grouping 8g35.2.2 Test case identifiers 9g35.3 ATS specification framework . 10g35.3.1 Use of TTCN-3 . 10g35.3.1.1 General 10g35.3.1.2 TTCN-3 naming conventions 10g35.3.1.3 TTCN-3 comment tags 11g35.4 ATS arc

11、hive 14g36 PIXIT conformance 14g37 ATS conformance 14g3Annex A (normative): Abstract Test Suite (ATS) . 15g3A.1 The ATS in TTCN-3 core (text) format . 15g3Annex B (normative): Partial IXIT pro forma 16g3B.0 The right to copy 16g3B.1 Introduction 16g3History 17g3ETSI ETSI TS 102 950-3 V1.6.1 (2017-08

12、)4 Intellectual Property Rights Essential patents IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “

13、Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org/). Pursuant to the ETSI IPR Policy, no investigation,

14、including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Trademarks The present document may incl

15、ude trademarks and/or tradenames which are asserted and/or registered by their owners. ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the prese

16、nt document does not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks. Foreword This Technical Specification (TS) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS). The present document is part3 of a mul

17、ti-part deliverable covering a TTCN-3 conformance test suite, as identified below: Part 1: “Implementation Conformance Statement (ICS)“; Part 2: “Test Suite Structure and Test Purposes (TSS Part 3: “Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (IXIT)“. Modal verbs termi

18、nology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in

19、 ETSI deliverables except when used in direct citation. ETSI ETSI TS 102 950-3 V1.6.1 (2017-08)5 1 Scope The present document specifies the Abstract Test Suite (ATS) for the TTCN-3 conformance test suite, as defined in ETSI ES 201 873-1 1 in compliance with the relevant guidance given in the pro for

20、ma for TTCN-3 reference test suite ETSI TS 102 950-2 8. The objective of the present document is to provide a basis for conformance tests for TTCN-3 tools giving a high probability of standard conformance with respect to TTCN-3 tools from different vendors. In the present document only the core lang

21、uage features, specified in ETSI ES 201 873-1 1 have been considered but not the tool implementation (see i.1 and i.2), language mapping (see i.3, i.4 and i.5) and language extension (see e.g. i.6, i.7 and i.8) aspects. The test notation used in the ATS attached in a zipped file is in TTCN-3 and it

22、is part of the present document. Annex A provides the Tree and Tabular Combined Notation (TTCN-3) part of the ATS. Annex B provides the Partial Implementation Extra Information for Testing (PIXIT) pro forma of the ATS. 2 References 2.1 Normative references References are either specific (identified

23、by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publ

24、icly available in the expected location might be found at https:/docbox.etsi.org/Reference/. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application

25、of the present document. 1 ETSI ES 201 873-1: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language“. 2 ETSI ES 201 873-10: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 10: TT

26、CN-3 Documentation Comment Specification“. 3 ETSI TS 102 351: “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv6 Testing: Methodology and Framework“. 4 ISO/IEC 9646-1 (1992): “Information Technology - Open Systems Interconnection - Conformance Testing Methodology and

27、 Framework - Part 1: General concepts“. 5 ISO/IEC 9646-4: “Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 4: Test realization“. 6 ISO/IEC 9646-5: “Information technology - Open Systems Interconnection - Conformance testing methodology and

28、 framework - Part 5: Requirements on test laboratories and clients for the conformance assessment process“. 7 ISO/IEC 9646-7 (1994): “Conformance testing methodology and framework - Part 7: Implementation Conformance Statement“. 8 ETSI TS 102 950-2: “Methods for Testing and Specification (MTS); TTCN

29、-3 Conformance Test Suite; Part 2: Test Suite Structure and Test Purposes (TSS The Testing and Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI)“. i.2 ETSI ES 201 873-6: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 6: TTCN-

30、3 Control Interface (TCI)“. i.3 ETSI ES 201 873-7: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 7: Using ASN.1 with TTCN-3“. i.4 ETSI ES 201 873-8: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Pa

31、rt 8: The IDL to TTCN-3 Mapping“. i.5 ETSI ES 201 873-9: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 9: Using XML schema with TTCN-3“. i.6 ETSI ES 202 781: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation vers

32、ion 3; TTCN-3 Language Extensions: Configuration and Deployment Support“. i.7 ETSI ES 202 784: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Advanced Parameterization“. i.8 ETSI ES 202 785: “Methods for Testing and Specific

33、ation (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Behaviour Types“. 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the terms and definitions given in ISO/IEC 9646-1 4, ISO/IEC 9646-7 7, ETSI ES 201 873-1 1 (TTCN-3) an

34、d the following apply: Abstract Test Method (ATM): description of how an IUT is to be tested, given at an appropriate level of abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable abstract test cases to be specified for

35、 this method Abstract Test Suite (ATS): test suite composed of abstract test cases Implementation Conformance Statement (ICS): statement made by the supplier of an implementation claimed to conform to a given specification, stating which capabilities have been implemented ICS pro forma: document, in

36、 the form of a questionnaire, which when completed for an implementation or system becomes an ICS Implementation eXtra Information for Testing (IXIT): statement made by a supplier or implementor of an IUT which contains or references all of the information related to the IUT and its testing environm

37、ent, which will enable the test laboratory to run an appropriate test suite against the IUT IXIT pro forma: document, in the form of a questionnaire, which when completed for the IUT becomes the IXIT ETSI ETSI TS 102 950-3 V1.6.1 (2017-08)7 Implementation Under Test (IUT): implementation of one or m

38、ore OSI protocols in an adjacent user/provider relationship, being part of a real open system which is to be studied by testing Means Of Testing (MOT): combination of equipment and procedures that can perform the derivation, selection, parameterization and execution of test cases, in conformance wit

39、h a reference standardized ATS and can produce a conformance log 3.2 Abbreviations For the purposes of the present document, the following abbreviations apply: ATM Abstract Test Method ATS Abstract Test Suite BNF Backus Naur Form ETS Executable Test Suite ICS Implementation Conformance Statement IUT

40、 Implementation under Test IXIT Implementation eXtra Information for Testing MOT Means Of Testing SUT System Under Test TC Test Case TCI TTCN-3 Control Interface TP Test Purpose TRI TTCN-3 Runtime Interface TS Test System TSS Test Suite Structure TSS they are based on the names provided in clause A.

41、2 of ETSI TS 102 950-2 8, but use the naming conventions defined for the ATS (see clause 5.3.1.2). The test case identifier naming scheme differentiates between positive and negative tests as well as syntactical and semantics tests. Syntactical tests are tests that refer to annex A of ETSI ES 201 87

42、3-1 1. They include pure syntactical tests and tests regarding the static semantics to the degree of detail that annex A provides. Semantic tests are tests that refer to the checking of properties regarding the static and dynamic semantics of TTCN-3 according to the specific clauses of ETSI ES 201 8

43、73-1 1. Positive tests are tests that shall work with a standards compliant TTCN-3 tool. Negative tests are tests that shall not work with a standards compliant TTCN-3 tool. The test cases shall conform to the following correctness rules: Negative syntactic tests shall be correct with respect to the

44、 TTCN-3 BNF and the static semantics of TTCN-3, but violate only one specific TTCN-3 BNF rule or static semantic rule specified in annex A of ETSI ES 201 873-1 1. They shall not produce an ETS. Positive syntactic tests shall be correct with respect to the TTCN-3 BNF and the static semantics of TTCN-

45、3. They may produce an ETS and if it contains a control-part or a test case, it should be executed. Negative semantic tests shall be correct with respect to the TTCN-3 BNF and the static semantics of TTCN-3, but violate the semantics of one specific text clause of ETSI ES 201 873-1 1. They may produ

46、ce an ETS. If an ETS is produced and if it contains a control-part or a test case, it should be executed. Positive semantic tests shall be correct with respect to the TTCN-3 BNF, the static semantics of TTCN-3, and the respective text clauses of ETSI ES 201 873-1 1. They shall produce an ETS. If an

47、ETS is produced and if it contains a control-part or a test case, it should be executed. ETSI ETSI TS 102 950-3 V1.6.1 (2017-08)9 The test case identifiers and their group index do not imply the correct execution order of a TTCN-3 tool conformance test. Grouping and subgrouping in the ATS is realize

48、d with the help of the ATS directory structure. Table 1: Example ATS structure of positive tests Group Subgroup Group Index Basic language elements Identifiers and keywords Syn_0501_Identifier Identifiers and keywords Scope rules Sem_0501_Identifier Syn_0502_Scopes Scope rules Ordering of language e

49、lements Sem_0502_Scopes Syn_0503_Ordering Ordering of language elements Parameterization Sem_0503_Ordering Syn_0504_Parameterization Parameterization Cyclic Definitions Sem_0504_Parameterization Syn_0505_Cyclic Cyclic Definitions Sem_0505_Cyclic Sem_0505_Cyclic Table 2: Example ATS structure of negative tests Group Subgroup Group Index Basic language elements Identifiers and keywords NegSyn_0501_Identifier Identifiers and keywords Scope rules NegSem_0501_Identifier NegSyn_0502_Scopes Scope r