1、 ETSI TS 118 115 V2.0.0 (2016-09) oneM2M; Testing Framework (oneM2M TS-0015 version 2.0.0 Release 2) TECHNICAL SPECIFICATION ETSI ETSI TS 118 115 V2.0.0 (2016-09)2(oneM2M TS-0015 version 2.0.0 Release 2)Reference DTS/oneM2M-000015 Keywords IoT, M2M, methodology, testing ETSI 650 Route des Lucioles F
2、-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice The present document can be downloaded from: http:/www.etsi.org/standards-se
3、arch The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents betwe
4、en such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Informat
5、ion on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Not
6、ification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyrigh
7、t and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2016. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI regis
8、tered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 118 115 V2.0.0 (2016-09)3(oneM2M TS-0015 version 2.0.0 Release 2)Contents Intellectual Property Rights 5g3Foreword . 5g31 Scope
9、 6g32 References 6g32.1 Normative references . 6g32.2 Informative references 6g33 Definitions and abbreviations . 6g33.1 Definitions 6g33.2 Abbreviations . 8g34 Conventions 8g35 Introduction to the oneM2M testing methodology . 8g36 Conformance testing. 10g36.1 Introduction 10g36.2 Test architecture
10、. 11g36.2.1 Selection of Implementation Under Test 11g36.2.1.1 Definition 11g36.2.1.2 oneM2M Service Layer Communication 12g36.2.2 Identification of the Reference Points . 12g36.3 Development of Conformance Test Specifications 12g36.3.1 Implementation Conformance Statement (ICS) 12g36.3.2 Test Suite
11、 Structure Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org/). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, ha
12、s been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specification (TS) has been produced by ET
13、SI Partnership Project oneM2M (oneM2M). ETSI ETSI TS 118 115 V2.0.0 (2016-09)6(oneM2M TS-0015 version 2.0.0 Release 2)1 Scope The present document specifies a testing framework defining a methodology for development of conformance and interoperability test strategies, test systems and the resulting
14、test specifications for oneM2M standards. 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest
15、 version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at https:/docbox.etsi.org/Reference/. NOTE: While any hyperlinks included in this clause were valid at the time of publica
16、tion, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present document. 1 ETSI TS 118 001: “oneM2M; Functional Architecture (oneM2M TS-0001)“. 2 ETSI TS 118 004: “Service layer Core Protocol (oneM2M TS-0004)“. 2.2 Informativ
17、e references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) appl
18、ies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject
19、 area. i.1 oneM2M Drafting Rules. NOTE: Available at http:/www.onem2m.org/images/files/oneM2M-Drafting-Rules.pdf. i.2 ISO/IEC 9646 (all parts): “Information technology - Open Systems Interconnection - Conformance testing methodology and framework“. i.3 ETSI EG 202 237: “Methods for Testing and Speci
20、fication (MTS); Internet Protocol Testing (IPT); Generic approach to interoperability testing“. i.4 ETSI ES 201 873-1: “Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language“. 3 Definitions and abbreviations 3.1 Definitions For the
21、 purposes of the present document, the following terms and definitions apply: conformance: compliance with requirements specified in applicable standards ISO/IEC 9646 i.2 ETSI ETSI TS 118 115 V2.0.0 (2016-09)7(oneM2M TS-0015 version 2.0.0 Release 2)conformance testing: process for testing that an im
22、plementation is compliant with a protocol standard, which is realized by test systems simulating the protocol with test scripts executed against the implementation under test Device Under Test (DUT): combination of software and/or hardware items which implement the functionality of standards and int
23、eract with other DUTs via one or more reference points ICS proforma: document, in the form of a questionnaire, which when completed for an implementation or system becomes an ICS Implementation Conformance Statement (ICS): statement made by the supplier of an implementation or system claimed to conf
24、orm to a given specification, stating which capabilities have been implemented Implementation eXtra Information for Testing (IXIT): checklist which contains or references all of the information (in addition to that given in the ICS) related to the IUT and its testing environment, which will enable t
25、he test laboratory to run an appropriate test suite against the IUT Implementation Under Test (IUT): implementation of one or more Open Systems Interconnection (OSI) protocols in an adjacent user/provider relationship, being the part of a real open system which is to be studied by testing (ISO/IEC 9
26、646-1 i.2) Inopportune Behaviour (BO): test group that handles invalid exchanges of messages, which are properly structured and correctly encoded interoperability: ability of two systems to interoperate using the same communication protocol interoperability testing: activity of proving that end-to-e
27、nd functionality between (at least) two devices is as required by the base standard(s) on which those devices are based InterWorking Function (IWF): translation of one protocol into another one so that two systems using two different communication protocols are able to interoperate Invalid Behaviour
28、 (BI): test group that handles valid exchanges of messages, which are either not properly structured or incorrectly encoded IXIT proforma: document, in the form of a questionnaire, which when completed for an implementation or system, becomes an IXIT Qualified Equipment (QE): grouping of one or more
29、 devices that has been shown and certified, by rigorous and well-defined testing, to interoperate with other equipment NOTE 1: Once an DUT has been successfully tested against a QE, it may be considered to be a QE, itself. NOTE 2: Once a QE is modified, it loses its status as QE and becomes again an
30、 DUT. test case: specification of the actions required to achieve a specific test purpose, starting in a stable testing state, ending in a stable testing state and defined in either natural language for manual operation or in a machine-readable language (such as TTCN-3) for automatic execution testi
31、ng framework: document providing guidance and examples necessary for the development and implementation of a test specification test purpose: description of a well-defined objective of testing, focussing on a single requirement or a set of related requirements Valid Behaviour (BV): test group that h
32、andles valid exchanges of messages, which are properly structured and correctly ETSI ETSI TS 118 115 V2.0.0 (2016-09)8(oneM2M TS-0015 version 2.0.0 Release 2)3.2 Abbreviations For the purposes of the present document, the terms and definitions given in ETSI TS 118 001 1 and the following apply: API
33、Application Programming Interface APT Abstract Protocol Tester ATS Abstract Test Suite BI Invalid BehaviourBO Inopportune Behaviour BV Valid Behaviour CoAP Constrained Application Protocol EUT Equipment Under Test FQDN Fully Qualified Domain Name HTTP HyperText Transfer Protocol IFS Interoperable Fe
34、atures Statement IOP Interoperability IUT Implementation Under Test IWF InterWorking Function JSON JavaScript Object Notation MMI Man-Machine InterfaceMQTT Message Queue Telemetry Transport PICS Protocol Implementation Conformance Statement QE Qualified Equipment SUT System Under Test TC Test Case T
35、CP Transmission Control Protocol TD Test Description TP Test Purpose TSS Test Suite Structure TTCN-3 Testing and Test Control Notation version 3 UDP User Datagram Protocol URI Uniform Resource Identifier XML eXtensible Markup Language4 Conventions The key words “Shall“, “Shall not“, “May“, “Need not
36、“, “Should“, “Should not“ in the present document are to be interpreted as described in the oneM2M Drafting Rules i.1. 5 Introduction to the oneM2M testing methodology The present document provides: Identification of the implementations under test (IUT) for conformance testing and the device under t
37、est (DUTs) for interoperability, i.e. answering the question “what is to be tested“. Definition of the applicable test procedures, i.e. answering the question “how is it to be tested“. Definition of the procedure for development of test specifications and deliverables (for instance: TSS - Test Suite
38、 Structure (TSS); - Test Purposes: square4 Conformance; square4 Interoperability. a methodology linking the individual elements of a test specification together: - style guidelines and examples; - naming conventions; - a structured notation for TP; - guidelines on the development of TTCN-3 Test Case
39、s (TCs); - guidelines on the use of tabulated English Test Descriptions (TDs). Conformance Test Specifications oneM2M Test Methodology IUTs PICS Identification of IUTs and DUTs Development of test specifications Abstract Test Method TSS the high degree of control of the test system over the sequence
40、 and contents of the protocol messages allows to test both valid and invalid behaviour. Figure 6.1-1: Conformance testing Conformance test specifications should be produced following the methodology described in ISO/IEC 9646-1 i.2. In summary, this methodology begins with the collation and categoriz
41、ation of the features and options to be tested into a tabular form which is normally referred to as the “ Implementation Conformance Statement“ (ICS). All implemented capabilities supported by the Implementation Under Test (IUT) are listed by the implementer in the ICS, so that the tester knows whic
42、h options have to be tested. This ensures that complete coverage is obtained. The next step is to collect the requirements from the specification that is tested. For each requirement, one or more tests should be identified and classified into a number of groups which will provide a structure to the
43、overall test suite (TSS). A brief Test Purpose (TP) should then be written for each identified test and this should make it clear what is to be tested but not how this should be done. Although not described or mandated in ISO/IEC 9646-1 i.2, in many situations (particularly where the TPs are complex
44、) it may be desirable to develop a Test Description (TD) for each TP. The TD describes in plain language (often tabulated) the actions required to reach a verdict on whether an implementation passes or fails the test. Finally, a detailed Test Case (TC) is written for each TP. In the interests of tes
45、t automation, TCs are usually combined into an Abstract Test Suite (ATS) using a specific testing language such as TTCN-3. The TCs in the ATS are then “Verified“ against a number of IUTs for correct operation according to some agreed procedures, before being released for use by the industry. An Impl
46、ementation eXtra Information for Test (IXIT) proforma associated to the ATS, should be produced in supplement of the ICS document and Test Cases to help to execute Protocol conformance testing using oneM2M dedicated test equipment. In summary, the oneM2M Conformance Testing methodology consists of:
47、Selection of Implementations Under Test (IUT). Identification of reference points. Development of test specifications, which includes: - Development of “Implementation Conformance Statements“ (ICS), if not already provided as part of the base standard. - Development of “Test Suite Structure and Test
48、 Purposes“ (TSS identification of the implementation; identification of the ; global statement of conformance Part of an example ICS table can be found in Annex A.1. 6.3.2 Test Suite Structure - invalid behaviour test; - timer test; - etc. Usually the identification of the different functional group
49、s of procedures leads to the definition of the top levels of the TSS. Then further levels at the bottom of the TSS is used to group test purposes belonging to the same type of test. Table 6.3.2.2-1 shows an example of a two level TSS used in the TSS square4 the expected behaviour; square4 the final conditions. The prose describing the test behaviour (including initial and final conditions) should follow some rules, as for instance the use of reserved keywords and syntax. Table 6.3.2.3.1-1: TP proforma template TP Id Test objectiv
