ETSI TS 102 995-2010 Methods for Testing and Specification (MTS) The Testing and Test Control Notation version 3 Proforma for TTCN-3 reference test suite (V1 1 1 Includes Diskette).pdf

1、 ETSI TS 102 995 V1.1.1 (2010-11)Technical Specification Methods for Testing and Specification (MTS);The Testing and Test Control Notation version 3;Proforma for TTCN-3 reference test suitefloppy3ETSI ETSI TS 102 995 V1.1.1 (2010-11)2Reference DTS/MTS-00115ED111 T3RefATS Keywords methodology, MTS, t

2、esting, TTCN, IXIT, ICS, TSS Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investiga

3、tion, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specificat

4、ion (TS) has been produced by ETSI Technical Committee Methods for Testing and Specification (MTS). The present document is the framework for a TTCN-3 Reference Test Suite. ETSI ETSI TS 102 995 V1.1.1 (2010-11)61 Scope The present document proposes a proforma for creating ICS, Test Purposes, and TTC

5、N-3 test cases to assess compliance of TTCN-3 tools to TTCN-3 core language including also an estimate for integrating tool vendor tests into the framework. The present document has been developed to provide the essential structure, naming, style, etc. information to develop conformance test specifi

6、cations for the TTCN-3 language. In the present document only the core language features, specified in ES 201 873-1 1 have been considered but not the tool implementation (see i.1 and i.2), language mapping (see i.3, i.4 and i.5) and language extension (see e.g. i.6, i.7and i.8) aspects. The test no

7、tation used in the ATS attached in a zipped file is in TTCN-3 and it is part of the present document. The following test specification - and design considerations can be found in the body of the present document: the overall test suite structure; the testing architecture; the test methods and port d

8、efinitions; the test configurations; the design principles, assumptions, and used interfaces to the TTCN3 tester (System Simulator); TTCN styles and conventions; the language ICS (Implementation Conformance Statement); the partial Implementation eXtra Information for Testing (IXIT) proforma; the Tes

9、t Suite Structure and Test Purposes (TSS the modules containing the TTCN-3 ATS. Annex A provides the language ICS (Implementation Conformance Statement). Annex B provides the Test Suite Structure and Test Purposes (TSS The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language“. E

10、TSI ETSI TS 102 995 V1.1.1 (2010-11)72 ETSI ES 201 873-10: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 10: TTCN-3 Documentation Comment Specification“. 3 ETSI TS 102 351: “Methods for Testing and Specification (MTS); Internet Protocol Testing (

11、IPT); IPv6 Testing: Methodology and Framework“. 4 ISO/IEC 9646-1 (1992): “Information Technology - Open Systems Interconnection - Conformance Testing Methodology and Framework - Part 1: General concepts“. 5 ISO/IEC 9646-7 (1994): “Conformance testing methodology and framework - Part 7: Implementatio

12、n Conformance Statement“. 2.2 Informative references The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ETSI ES 201 873-5: “Methods for Testing and Specification (MTS); The Testing an

13、d Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI)“. i.2 ETSI ES 201 873-6: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 6: TTCN-3 Control Interface (TCI)“. i.3 ETSI ES 201 873-7: “Methods for Testing and Specification (MT

14、S); The Testing and Test Control Notation version 3; Part 7: Using ASN.1 with TTCN-3“. i.4 ETSI ES 201 873-8: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 8: The IDL to TTCN-3 Mapping“. i.5 ETSI ES 201 873-9: “Methods for Testing and Specificati

15、on (MTS); The Testing and Test Control Notation version 3; Part 9: Using XML schema with TTCN-3“. i.6 ETSI ES 202 781: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Configuration and Deployment Support“. i.7 ETSI ES 202 784

16、: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Advanced Parameterization“. i.8 ETSI ES 202 785: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Language Extensions: Beh

17、aviour Types“. 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the terms and definitions given in ISO/IEC 9646-1 4, ISO/IEC 9646-7 5, ES 201 873-1 1 (TTCN-3) and the following apply: Abstract Test Case (ATC): complete and independent specification of the act

18、ions required to achieve a specific test purpose, defined at the level of abstraction of a particular Abstract Test Method, starting in a stable testing state and ending in a stable testing state Abstract Test Method (ATM): description of how an IUT is to be tested, given at an appropriate level of

19、abstraction to make the description independent of any particular realization of a Means of Testing, but with enough detail to enable abstract test cases to be specified for this method Abstract Test Suite (ATS): test suite composed of abstract test cases ICS proforma: document, in the form of a que

20、stionnaire, which when completed for an implementation or system becomes the ICS ETSI ETSI TS 102 995 V1.1.1 (2010-11)8Implementation Conformance Statement (ICS): statement made by the supplier of an implementation claimed to conform to a given specification, stating which capabilities have been imp

21、lemented Implementation eXtra Information for Testing (IXIT): statement made by a supplier or implementor of an IUT which contains or references all of the information related to the IUT and its testing environment, which will enable the test laboratory to run an appropriate test suite against the I

22、UT Implementation Under Test (IUT): implementation of one or more OSI protocols in an adjacent user/provider relationship, being part of a real open system which is to be studied by testing IXIT proforma: document, in the form of a questionnaire, which when completed for the IUT becomes the IXIT Mea

23、ns Of Testing (MOT): combination of equipment and procedures that can perform the derivation, selection, parameterization and execution of test cases, in conformance with a reference standardized ATS and can produce a conformance log Point of Control and Observation (PCO): point within a testing env

24、ironment where the occurrence of test events is to be controlled and observed, as defined in an Abstract Test Method Pre-Test Condition: setting or state in the IUT which cannot be achieved by providing stimulus from the test environment 3.2 Abbreviations For the purposes of the present document, th

25、e following abbreviations apply: ASP Abstract Service Primitive NOTE: Exchanged between entities inside the TS or between the user of the ATS (operator) and the TS. ATC Abstract Test Case ATM Abstract Test Method ATS Abstract Test Suite EBNF Extended Backus Naur Form ETS Executable Test Suite ICS Im

26、plementation Conformance Statement IUT Implementation under Test IXIT Implementation eXtra Information for Testing MOT Means Of Testing SUT System Under Test TC Test Case TCI TTCN-3 Control Interface TP Test Purpose TRI TTCN-3 Runtime Interface TS Test System TSS Test Suite Structure TSS they are ba

27、sed on the names provided in Annex B, but use the naming conventions defined for the ATS (see Clause 5.3.2.2). The test case identifier naming scheme differentiates between positive and negative tests as well as syntactical and semantics tests. Syntactical tests are tests that refer to Annex A of ES

28、 201 873-1 1. They include pure syntactical tests and tests regarding the static semantics to the degree of detail that Annex A provides. Semantic tests are tests that refer to the checking of properties regarding the static and dynamic semantics of TTCN-3 according to the specific clauses of ES 201

29、 873-1 1. Positive tests are tests that shall work with a standards compliant TTCN-3 tool. Negative tests are tests that shall not work with a standards compliant TTCN-3 tool. The test cases shall conform to the following correctness rules: Negative syntactic tests shall be correct with respect to t

30、he TTCN-3 EBNF and the static semantics of TTCN-3, but violate only one specific TTCN-3 EBNF rule or static semantic rule specified in Annex A of ES 201 873-1 1. They shall not produce an ETS. Positive syntactic tests shall be correct with respect to the TTCN-3 EBNF and the static semantics of TTCN-

31、3. They may produce an ETS and if it contains a control-part or a test case, it should be executed. Negative semantic tests shall be correct with respect to the TTCN-3 EBNF and the static semantics of TTCN-3, but violate the semantics of one specific text clause of ES 201 873-1 1. They may produce a

32、n ETS. If an ETS is produced and if it contains a control-part or a test case, it should be executed. Positive semantic tests shall be correct with respect to the TTCN-3 EBNF, the static semantics of TTCN-3, and the respective text clauses of ES 201 873-1 1. They shall produce an ETS. If an ETS is p

33、roduced and if it contains a control-part or a test case, it should be executed. The test case identifiers and their group index do not imply the correct execution order of a TTCN-3 tool conformance test. Dependencies among test cases are expressed in the test purposes and by the structure of the AT

34、S. Table 1: Example ATS structure of positive tests Group Subgroup Group Index Basic language elements Identifiers and keywords Syn_0501_Identifier Identifiers and keywords Scope rules Sem_0501_Identifier Syn_0502_Scopes Scope rules Ordering of language elements Sem_0502_Scopes Syn_0503_Ordering Ord

35、ering of language elements Parameterization Sem_0503_Ordering Syn_0504_Parameterization Parameterization Cyclic Definitions Sem_0504_Parameterization Syn_0505_Cyclic Cyclic Definitions Sem_0505_Cyclic Sem_0505_Cyclic ETSI ETSI TS 102 995 V1.1.1 (2010-11)11Table 2: Example ATS structure of negative t

36、ests Group Subgroup Group Index Basic language elements Identifiers and keywords NegSyn_0501_Identifier Identifiers and keywords Scope rules NegSem_0501_Identifier NegSyn_0502_Scopes Scope rules Ordering of language elements NegSem_0502_Scopes NegSyn_0503_Ordering Ordering of language elements Param

37、eterization NegSem_0503_Ordering NegSyn_0504_Parameterization Parameterization Cyclic Definitions NegSem_0504_Parameterization NegSyn_0505_Cyclic Cyclic Definitions NegSem_0505_Cyclic NegSem_0505_Cyclic 5.2.2 Test case identifiers The test case names are built up according to the following scheme: “

38、_“_“ where: a) double quotes (“) are used to enclose literal strings; b) containing positive and negative syntactic and semantic test, refers to ES 201 873-1 1 clause numbers and names; c) is a running 3-digit decimal number, starting in each subgroup path with “001“. EXAMPLE: TC_Syn_0501_Identifier

39、_001 i) The example refers to a positive syntactical identifier and keyword test case. ii) It is the first test case of this group/subgroup. NOTE 1: This naming scheme corresponds to the TP identifiers and test case names as defined in Annex B. NOTE 2: The TP identifier of TC_Syn_0501_Identifier_001

40、 is TP_Syn_0501_Identifier_001. 5.3 ATS specification framework 5.3.1 ATS library Table 3 shows the organization of the ATS as library of modules. In general, every test case shall be placed in a separate module. There are exceptions where it is essential to put more than one test case into a module

41、, like tests related to scope or test cases for importing and visibility. ETSI ETSI TS 102 995 V1.1.1 (2010-11)12Table 3: Library of modules (proforma table) Module Class Module Id Description Syn_0501_Identifier_001 Main TTCN-3 module with control-part. NegSyn_0501_Identifier_001 Main TTCN-3 module

42、 with control-part. 5.3.2 Use of TTCN-3 5.3.2.1 General TTCN-3, as defined in ES 201 873-1 1, is used as the ATS specification language. A number of requirements have been identified for the development and production of the TTCN-3 specification for the ATS: 1) Top-down design. 2) A uniquely defined

43、 testing architecture and test method. 3) Uniform TTCN-3 style and naming conventions. 4) Human-readability. 5) The TTCN-3 specification shall be feasible, implementable, compilable, and maintainable. 6) Test cases shall be designed in a way to be easily adaptable, upwards compatible with the evolut

44、ion of the base protocol and protocol interworking of future releases. 7) The test declarations, data structures, and data values shall be largely reusable. 8) Modularity and modular working method. 9) Minimizing the requirements of intelligence on the emulators of the lower testers. 10) Giving enou

45、gh design freedom to the test equipment manufacturers. Fulfilling these requirements should ensure the investment of the TTCN-3 implementation vendors and users of the ATS having stable testing means for a relatively long period. 5.3.2.2 TTCN-3 naming conventions Like in other software projects usin

46、g a programming language, the use of naming conventions supports or increases: a) the readability; b) the detection of semantic errors; c) the shared work of several developers; d) the maintainability. ETSI ETSI TS 102 995 V1.1.1 (2010-11)13The naming conventions applied to Reference Test suite ATS

47、are based on the following underlying principles: when constructing meaningful identifiers, the general guidelines specified for naming in Clause 9 of TS 102 351 3 should be followed; the names of TTCN-3 objects being associated with standardized data types (e.g. in the base protocols) should reflec

48、t the names of these data types as close as possible (of course not conflicting with syntactical requirements or other conventions being explicitly stated); the subfield names of TTCN-3 objects being associated with standardized data type should also be similar to corresponding element names in the

49、base standards (be recognizable in the local context); in most other cases, identifiers should be prefixed with a short alphabetic string (specified in Table 4) indicating the type of TTCN-3 element it represents; prefixes should be separated from the body of the identifier with an underscore (“_“); only test case names, module names, data type names, and module parameters should begin with an upper-case letter. All other names (i.e. the part of the identifier following the prefix)