1、REPORT ETR 346 December 1996 Source: ETSI TC-RES Reference: DTR/RES-06013-1 ICs: 33.020 Key words: Testing, TTCN, abstract test suite, validation Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Air Interface (AI) layer 2 and layer 3 protocol validation; Part 1: Validation of
2、 test suites for Voice plus Data (V+D) ETSI European Telecornmunications Standards Institute ETSI Secretariat Postal address: F-O6921 Sophia Antipolis CEDEX - FRANCE Office address: 650 Route des Lucioles - Sophia Antipolis - Valbonne - FRANCE X.400: c=fr, a=atlas, p=etsi, s=secretanat - Internet: s
3、ecretariatetsi.fr Tel.: +33 4 92 94 42 O0 - Fax: +33 4 93 65 47 16 Copyright Notification: No part may be reproduced except as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in ali media. Q European Telecommunications Standards institute 1996. Ai
4、l rights resewed. Page 2 STDmESI ETR 3qb-ENGL L77b M 3400855 03bZb20 205 = ETR 346: December 1996 Whilst every care has been taken in the preparation and publication of this document, errors in content, typographical or otherwise, may occur. If you have comments concerning its accuracy, please write
5、 to “ETSI Editing and Committee Support Dept.“ at the address shown on the title page. STD-ETSI ETR 34b-ENGL 177b 3400855 OlbZbZ2 141 E Page 3 ETR 346: December 1996 Contents Foreword . 5 1 2 3 4 5 6 7 Scope 7 References 7 Definitions and abbreviations 7 3.1 Definitions 7 3.2 Abbreviations . 8 Intro
6、duction 8 General 8 5.1 5.3 Test suite validation principles . 8 5.2 Validation properties 10 5.3.2 Approach 2 12 5.3.3 Approach 3 12 Different validation approaches . 11 5.3.1 Approach 1 11 Validation performed . 13 6.1 Validation process 13 6.2 Validation results 13 Summary . 13 History 15 STD-ETS
7、I ETR 34b-ENGL L77b E 3q00855 OLb2b22 O88 = Page 4 ETR 346: December 1996 Blank page STD-ETSI ETR 3Lib-ENGL L77b E 340085S OZb2b23 TL4 m Page 5 ETR 346: December 1996 Foreword This ETSI Technical Report (ETR) has been produced by the Radio Equipment and Systems (RES) Technical Committee of the Europ
8、ean Telecommunications Standards Institute (ETSI). ETRs are informative documents resulting from ETSI studies which are not appropriate for European Telecommunication Standard (ETS) or Interim European Telecommunication Standard (LETS) status. An ETR may be used to publish material which is either o
9、f an informative nature, relating to the use or the application of ETSs or I-ETSs, or which is immature and not yet suitable for formal adoption as an ETS or an LETS. STD*ETSI ETR 3ib-ENGL 177b W 3400855 OZb2b24 950 I Page 6 ETR 346: December 1996 Blank page STD*ETSI ETR 34b-ENGL 1SSb B 3Li00055 OLb
10、Zb25 897 Page 7 ETR 346: December 1996 1 Scope This ETSI Technical Report (ETR) defines the methods, procedures, and validation purposes used for the formal validation of the Tree and Tabular Combined Notation (TTCN) conformance test suites for TETRA Voice and Data (V+D) Air Interface (AI) and docum
11、ents the results of the validation. 2 References For the purposes of this ETR, the following references apply: 31 141 71 181 3 3.1 ETS 300 394-2-1 : “Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Conformance testing specification, Part 2: Protocol testing specification for
12、 Voice plus Data (V+D); Part 2-1: Test suites structure and test purposes“. ETS 300 394-2-2: “Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Conformance testing specification, Part 2: Protocol testing specification for Voice plus Data (V+D); Part 2-2: Abstract Test Suite (A
13、TS) for Network (NWK) layer“. ETS 300 394-2-3: “Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Conformance testing specification, Part 2: Protocol testing specification for Voice plus Data (V+D); Part 2-3: Abstract Test Suite (ATS) for Logical Link Control (LLC)“. ETS 300 3
14、94-2-4: “Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Conformance testing specification, Part 2: Protocol testing specification for Voice plus Data (V+D); Part 2-4: Abstract Test Suite (ATS) for Medium Access Control (MAC)“. ETS 300 392-2: “Radio Equipment and Systems (RE
15、S); Trans-European Trunked Radio (TETRA); Voice plus Data (V+D); Part 2: Air Interface (Al)“. ETR 293-1 : “Radio Equipment and Systems (RES); Trans-European Trunked Radio (TETRA); Air Interface (AI) layer 2 and 3 protocol validation; Part 1 : Validation of SDL models for Voice plus Data (V+D)“. ISO/
16、IEC 9646-3 (1 991): “Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 3: The tree and tabular combined notation“. (See also CCIlT Recommendation X.292 (1992). ITU-T Recommendation 2.105 (1995): “SDL combined with ASN.1“. Definitions and abb
17、reviations Definitions For the purposes of this ETR, the following definitions apply: external validation: Validation of TTCN test suite properties except for those related to the TTCN language definition. internal validation: Checking the correctness of a TTCN test suite according to the rules of T
18、TCN as defined in IS0 9646, part 3 7. STD-ETSI ETR 3Lib-ENGL 177b R 3LiOO855 OLb2b2b 723 D Page 8 ETR 346: December 1996 3.2 Abbreviations For the purposes of this ETR, the following abbreviations apply: ASN.l ATS CMCE ExTS LLC MAC MLE MM NWK PDU SCLNP SDL TTCN Abstract Syntax Notation number One Ab
19、stract Test Suite Circuit Mode Control Entity Executable Test Suite Logical Link Control Medium Access Control Mobile Link Entity Mobility Management Network Protocol Data Unit Specific Connectionless Network Protocol Specification and Description Language Tree and Tabular Combined Notation 4 Introd
20、uction This ETR documents the validation of the TETRA V+D protocol conformance test suites for Network (NWK) layer, ETS 300 394-2-2 2, Logical Link Control (LLC), ETS 300 394-2-3 3, and the Medium Access Control (MAC), ETS 300 394-2-4 4. For the test purposes and test suites structure refer to ETS 3
21、00394-2-1 i and for the protocol specification to ETS 300 392-2 5. The purpose of the test suite validation is to ensure the quality of the Abstract Test Suites (ATS) which forms the basis for test suite implementations. This test suite validation has been focused on the use of formal validation met
22、hods that are supported by the tools available at ETSI. The ETR also provides a brief description of other aspects of test suite validation, which were considered as potential validation methods, but were not applied in the validation documented in this ETR. 5 General 5.1 Test suite Validation princ
23、ipies The Abstract Test Suites (ATSs) for the protocols in TETRA are specifications in the process of defining the conformance criterion for TETRA products. An ATS specification serves both as an operational definition of the test purposes and as a basis for development of Executable Test Suites (Ex
24、TSs). The checking of these different properties of an ATS is the purpose of test suite validation. In figure 1 validation properties for an ATS are shown. The term “Protocol requirements“ used in figure i refers to all requirements defined in either of the protocol description, protocol model or pr
25、otocol test purpose documents. Each of the highlighted validation properties are further explained in subclause 5.2. - STD-ETSI ETR 34h-ENGL L77b m 3400855 OLbZb27 bbT M Page 9 ETR 346: December 1996 to derive Validation properties Protocol requirements F-., I validate : Correctness against / Covera
26、ge Test purposes ._i _._._ TTCN ATS -. 1 validation TTCN definition TTCNATS b : Implementability to derive Executability Figure 1 : Test suite validation properties In addition to the validation properties shown in figure 1, a TTCN test suite should also be checked against the rules of TTCN, defined
27、 in IS0 9646, part 3 7. This type of validation is denoted internal validation, while the validation of all other properties is denoted external validation. Methods for TTCN test suite validation can be distinguished on whether they are based on formal models and formally defined relations between s
28、uch models or on informal methods. Informal validation methods often are based on inspection techniques, which can be automated and supported by tools only to a limited extent. Validation methods based on formal models are a prerequisite for tool supported and automatic check of validation propertie
29、s. Formal methods, in principle, allow exhaustive checking of the validation properties. For the TETRA protocol test suites both informal and formal validation methods in principle could be applied as indicated in figure 2. , Informal specifications : Formal specifications (Protocol validation) Form
30、al validation specifications Figure 2: Formal and informal validation possibilities for TETRA protocol ATSs Page 10 STD.ETSI ETR 14b-ENGL 177t 6 3ii008S5 ULb2b28 5b ETR 346: December 1996 Protocol validation is indicated in figure 2 but is not part of test suite validation, rather it can be seen as
31、a prerequisite to ensure the value of the test suite validation. Using protocol validation it is possible to check that a formal model consistently specifies the requirements of the textual protocol specifications. In the case of TETRA, a validated model of the V+D protocols exists. The model is spe
32、cified using formal Specification and Description Language (SDL). For further information on the protocol validation refer to ETR 293-1 6. In figure 2 it is also indicated that the validation of the ATS with respect to the ExTS can only be performed using informal validation methods. In principle th
33、is can be done using formal validation as the ETSI tools for TTCN test case specification includes a TTCN compiler. However, as the implementation of an ATS is outside the scope of standardization, only informal validation methods are supposed to be applied for validation properties like implementat
34、ion and execution. 5.2 Validation properties As the validation properties are open for different interpretations they shall be further refined to be supported by systematic methods. The internal TTCN test suite validation property generally covers validation of an ATS with respect to the rules of th
35、e TTCN definition in IS0 9646, part 3 7. The internal validation property comprises of: - checking the syntactical correctness of a TTCN test suite according to the TTCN syntax rules; - checking that the test suite comply with the static semantics rules for TTCN, e.g. that all test suite and test ca
36、se variables used have been declared and that Protocol Data Units (PDUs) and constraints are of complying types; - checking the dynamic behaviour of the test cases by simulation against e.g. an executable validation model or a prototype implementation. The syntax and static semantics analysis of an
37、ATS is fully supported by the TTCN tool available at ETSI. The simulation of TTCN test cases is also supported by the TTCN tool, but the actual check of the simulation results should be done by inspection. Another way to ensure the correctness of the dynamic behaviour of the test cases is to use a t
38、otally opposite approach to simulation. Using this method, the dynamic behaviour of the test cases are semi-automatically derived from a formal protocol model that is assumed to correctly reflect the protocol requirements, .e. has been validated. This method is also possible within the tools availab
39、le at ETSI. All other validation properties belong to the category of external validation properties. Two main external validation properties can be identified: correctness property and coverage property. Basically, the correctness validation checks that all test cases define a test for a requiremen
40、t specified in the protocol specification. The correctness property can be checked using the ETSI tools which allow a TTCN test case to be simulated against an SDL-protocol model. Using the simulation, it is a prerequisite that all the requirements that are tested in the test cases are also present
41、in the SDL-model used in the simulation. The coverage property validates that the ATS includes test cases for the selected requirements of the protocol description. For a complete conformance test suite this means to check that test cases are specified for all requirements in the protocol descriptio
42、n. In principle this validation can be performed using an SDL protocol model and an ATS alone. For all conformance test suites the test purpose document is needed, as even complete conformance test suites cover only a subset of the protocol requirements. So coverage validation is performed taking th
43、e test purposes into account. Even if the validation of the test cases against an SDL protocol model can be supported by the ETSI tools, only informal methods can be used to check that test cases are specified for all essential requirements. STD-ETSI ETR 34b-ENGL 1771 3400855 DLb2b29 432 Page 11 ETR
44、 346: December1996 The methods are also applied in a certain order to ensure correct validation results. A feasible order of application is illustrated in figure 3. i I 1. Syntax, semantics and I dynamic behaviour Internal validation i 2. Correctness, coverage I I I , 1 , External validation i I 3.
45、Implementability, I I executability I I I I Figure 3: Order of different validation methods The presented order helps to trace any errors to their origin. Until the internal validation has successfully been performed, it might not be possible at all to perform formal correctness or coverage validati
46、on or any other kind of external validation. 5.3 Different validation approaches Different approaches can be defined to check properties that may covered by formal test suite validation methods. With respect to the available tools for the validation, the selection of the approach applied for the TET
47、RA V+D protocol test suites was made among the following alternatives: 5.3.1 Approach 1 The validation is performed in the following steps: 1) For each test purpose a corresponding TTCN test case is created using the ETSI SDL-tool to derive the TTCN test case from the SDL specification. 2) Minor and
48、 systematic modifications are made to the derived TTCN declarations and constraints in order to make them comply with the ASN.l rules. 3) The TTCN test cases are simulated against the same SDL-model from which they were initially derived from. The major disadvantage of this approach is that the func
49、tional context of the test cases is the same as the one of the SDL model, presuming that the tools work correctly. Hence no functional errors can be found in the test cases performing this validation. The approach will only identify errors in the tools used, as inconsistencies discovered during simulation will be related either to errors in the TTCN test case derivation tool or the TTCN-SDL simulation tool. It will, however, be possible to check if the modifications made to TTCN data types and constraints in the TTCN test suites are consistent
