ETSI TS 102 950-3-2016 Methods for Testing and Specification (MTS) TTCN-3 Conformance Test Suite Part 3 Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (.pdf

1、 ETSI TS 102 950-3 V1.5.1 (2016-02) Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Part 3: Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (IXIT) floppy3TECHNICAL SPECIFICATION ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)2 Reference RTS/MTS-102950-3

2、ed151T3Conf Keywords ATS, conformance, testing, TTCN ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important n

3、otice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written author

4、ization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should b

5、e aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following service

6、s: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall

7、 not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2016. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for

8、the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)3 Contents Intellectual Prope

9、rty Rights 4g3Foreword . 4g3Modal verbs terminology 4g31 Scope 5g32 References 5g32.1 Normative references . 5g32.2 Informative references 6g33 Definitions and abbreviations . 6g33.1 Definitions 6g33.2 Abbreviations . 7g34 Abstract Test Method (ATM) . 7g35 The ATS development process . 8g35.1 Requir

10、ements and test purposes . 8g35.2 ATS structure . 8g35.2.1 Test case grouping 8g35.2.2 Test case identifiers 9g35.3 ATS specification framework . 10g35.3.1 Use of TTCN-3 . 10g35.3.1.1 General 10g35.3.1.2 TTCN-3 naming conventions 10g35.3.1.3 TTCN-3 comment tags 11g35.4 ATS archive 14g36 PIXIT confor

11、mance 14g37 ATS conformance 14g3Annex A (normative): Abstract Test Suite (ATS) . 15g3A.1 The ATS in TTCN-3 core (text) format . 15g3Annex B (normative): Partial IXIT pro forma 16g3B.1 Introduction 16g3History 17g3ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)4 Intellectual Property Rights IPRs essential or

12、 potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Esse

13、ntial, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee c

14、an be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specification (TS) has been produced by ETSI Technical Committee Methods for Testing

15、 and Specification (MTS). The present document is part 3 of a multi-part deliverable. Full details of the entire series can be found in part 1 i.9. Modal verbs terminology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“

16、are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in ETSI deliverables except when used in direct citation. ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)5 1 Scope The present document specifies

17、 the Abstract Test Suite (ATS) for the TTCN-3 conformance test suite, as defined in ETSI ES 201 873-1 1. The objective of the present document is to provide a basis for conformance tests for TTCN-3 tools giving a high probability of standard conformance with respect to TTCN-3 tools from different ve

18、ndors. In the present document only the core language features, specified in ETSI ES 201 873-1 1 have been considered but not the tool implementation (see i.1 and i.2), language mapping (see i.3, i.4 and i.5) and language extension (see e.g. i.6, i.7and i.8) aspects. The test notation used in the AT

19、S attached in a zipped file is in TTCN-3 and it is part of the present document. Annex A provides the Tree and Tabular Combined Notation (TTCN-3) part of the ATS. Annex B provides the Partial Implementation Extra Information for Testing (PIXIT) Pro forma of the ATS. 2 References 2.1 Normative refere

20、nces References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including any amendments) applies. Refe

21、renced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced

22、 documents are necessary for the application of the present document. 1 ETSI ES 201 873-1 (V4.7.1): “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language“. 2 ETSI ES 201 873-10 (V4.5.1): “Methods for Testing and Specification (MTS

23、); The Testing and Test Control Notation version 3; Part 10: TTCN-3 Documentation Comment Specification“. 3 ETSI TS 102 351: “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv6 Testing: Methodology and Framework“. 4 ISO/IEC 9646-1 (1992): “Information Technology - Ope

24、n Systems Interconnection - Conformance Testing Methodology and Framework - Part 1: General concepts“. 5 ISO/IEC 9646-4: “Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 4: Test realization“. 6 ISO/IEC 9646-5: “Information technology - Ope

25、n Systems Interconnection - Conformance testing methodology and framework - Part 5: Requirements on test laboratories and clients for the conformance assessment process“. 7 ISO/IEC 9646-7 (1994): “Conformance testing methodology and framework - Part 7: Implementation Conformance Statement“. 8 ETSI T

26、S 102 950-2: “Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Part 2: Test Suite Structure and Test Purposes (TSS The Testing and Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI)“. i.2 ETSI ES 201 873-6: “Methods for Testing and Specification (MTS);

27、 The Testing and Test Control Notation version 3; Part 6: TTCN-3 Control Interface (TCI)“. i.3 ETSI ES 201 873-7: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 7: Using ASN.1 with TTCN-3“. i.4 ETSI ES 201 873-8: “Methods for Testing and Specifica

28、tion (MTS); The Testing and Test Control Notation version 3; Part 8: The IDL to TTCN-3 Mapping“. i.5 ETSI ES 201 873-9: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 9: Using XML schema with TTCN-3“. i.6 ETSI ES 202 781: “Methods for Testing and

29、Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Configuration and Deployment Support“. i.7 ETSI ES 202 784: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Advanced Parameteri

30、zation“. i.8 ETSI ES 202 785: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Behaviour Types“. i.9 ETSI TS 102 950-1: “Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Part 1: Implementation Confor

31、mance Statement (ICS)“. 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the terms and definitions given in ISO/IEC 9646-1 4, ISO/IEC 9646-7 7, ETSI ES 201 873-1 1 (TTCN-3) and the following apply: Abstract Test Method (ATM): description of how an IUT is to b

32、e tested, given at an appropriate level of abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable abstract test cases to be specified for this method Abstract Test Suite (ATS): test suite composed of abstract test cases I

33、CS pro forma: document, in the form of a questionnaire, which when completed for an implementation or system becomes an ICS Implementation Conformance Statement (ICS): statement made by the supplier of an implementation claimed to conform to a given specification, stating which capabilities have bee

34、n implemented Implementation eXtra Information for Testing (IXIT): statement made by a supplier or implementor of an IUT which contains or references all of the information related to the IUT and its testing environment, which will enable the test laboratory to run an appropriate test suite against

35、the IUT Implementation Under Test (IUT): implementation of one or more OSI protocols in an adjacent user/provider relationship, being part of a real open system which is to be studied by testing ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)7 IXIT pro forma: document, in the form of a questionnaire, which

36、when completed for the IUT becomes the IXIT Means Of Testing (MOT): combination of equipment and procedures that can perform the derivation, selection, parameterization and execution of test cases, in conformance with a reference standardized ATS and can produce a conformance log 3.2 Abbreviations F

37、or the purposes of the present document, the following abbreviations apply: ASCII American Standard Code for Information Interchange ATM Abstract Test Method ATS Abstract Test Suite BNF Backus Naur Form ETS Executable Test Suite ICS Implementation Conformance Statement IUT Implementation under Test

38、IXIT Implementation eXtra Information for Testing MOT Means Of Testing PIXIT Protocol Implementation eXtra Information for Testing TC Test Case TCI TTCN-3 Control Interface TP Test Purpose TRI TTCN-3 Runtime Interface TS Test System TSS Test Suite Structure TSS they are based on the names provided i

39、n clause A.2 of ETSI TS 102 950-2 8, but use the naming conventions defined for the ATS (see clause 5.3.2.2). The test case identifier naming scheme differentiates between positive and negative tests as well as syntactical and semantics tests. Syntactical tests are tests that refer to annex A of ETS

40、I ES 201 873-1 1. They include pure syntactical tests and tests regarding the static semantics to the degree of detail that annex A provides. Semantic tests are tests that refer to the checking of properties regarding the static and dynamic semantics of TTCN-3 according to the specific clauses of ET

41、SI ES 201 873-1 1. Positive tests are tests that shall work with a standards compliant TTCN-3 tool. Negative tests are tests that shall not work with a standards compliant TTCN-3 tool. The test cases shall conform to the following correctness rules: Negative syntactic tests shall be correct with res

42、pect to the TTCN-3 BNF and the static semantics of TTCN-3, but violate only one specific TTCN-3 BNF rule or static semantic rule specified in annex A of ETSI ES 201 873-1 1. They shall not produce an ETS. Positive syntactic tests shall be correct with respect to the TTCN-3 BNF and the static semanti

43、cs of TTCN-3. They may produce an ETS and if it contains a control-part or a test case, it should be executed. Negative semantic tests shall be correct with respect to the TTCN-3 BNF and the static semantics of TTCN-3, but violate the semantics of one specific text clause of ETSI ES 201 873-1 1. The

44、y may produce an ETS. If an ETS is produced and if it contains a control-part or a test case, it should be executed. Positive semantic tests shall be correct with respect to the TTCN-3 BNF, the static semantics of TTCN-3, and the respective text clauses of ETSI ES 201 873-1 1. They shall produce an

45、ETS. If an ETS is produced and if it contains a control-part or a test case, it should be executed. The test case identifiers and their group index do not imply the correct execution order of a TTCN-3 tool conformance test. Grouping and subgrouping in the ATS is realized with the help of the ATS dir

46、ectory structure. ETSI ETSI TS 102 950-3 V1.5.1 (2016-02)9 Table 1: Example ATS structure of positive tests Group Subgroup Group Index Basic language elements Identifiers and keywords Syn_0501_Identifier Identifiers and keywords Scope rules Sem_0501_Identifier Syn_0502_Scopes Scope rules Ordering of

47、 language elements Sem_0502_Scopes Syn_0503_Ordering Ordering of language elements Parameterization Sem_0503_Ordering Syn_0504_Parameterization Parameterization Cyclic Definitions Sem_0504_Parameterization Syn_0505_Cyclic Cyclic Definitions Sem_0505_Cyclic Sem_0505_Cyclic Table 2: Example ATS struct

48、ure of negative tests Group Subgroup Group Index Basic language elements Identifiers and keywords NegSyn_0501_Identifier Identifiers and keywords Scope rules NegSem_0501_Identifier NegSyn_0502_Scopes Scope rules Ordering of language elements NegSem_0502_Scopes NegSyn_0503_Ordering Ordering of langua

49、ge elements Parameterization NegSem_0503_Ordering NegSyn_0504_Parameterization Parameterization Cyclic Definitions NegSem_0504_Parameterization NegSyn_0505_Cyclic Cyclic Definitions NegSem_0505_Cyclic NegSem_0505_Cyclic 5.2.2 Test case identifiers The test case names are built up according to the following scheme: “_“_“ where: a) double quotes (“) are used to enclose literal strings; b) containing positive and negative syntactic and semantic test, refers to ETSI ES 201 873-1 1