1、 ETSI TS 103 268-1 V1.1.1 (2017-04) SmartM2M; Smart Appliances Ontology and Communication Framework Testing; Part 1: Testing methodology TECHNICAL SPECIFICATION ETSI ETSI TS 103 268-1 V1.1.1 (2017-04)2 Reference DTS/SmartM2M-103 268-1 SAP_tst Keywords data, data sharing, IoT, M2M, Model, multi servi
2、ce testing, oneM2M, ontology, testing ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice The prese
3、nt document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI
4、. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that th
5、e document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: ht
6、tps:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not
7、be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2017. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the b
8、enefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 103 268-1 V1.1.1 (2017-04)3 Contents Intellectual Property R
9、ights 5g3Foreword . 5g3Modal verbs terminology 5g31 Scope 6g32 References 6g32.1 Normative references . 6g32.2 Informative references 6g33 Definitions and abbreviations . 7g33.1 Definitions 7g33.2 Abbreviations . 8g34 Introduction to the SAP testing methodology 8g35 Conformance testing. 10g35.1 Intr
10、oduction 10g35.2 Implementation Under Test 11g35.2.1 Introduction. 11g35.2.2 Service Layer Communication . 12g35.2.2.1 oneM2M implementation 12g35.2.2.2 SAP implementation . 12g35.3 Identification of the Reference points 12g35.4 Development of Conformance Test Specifications 12g35.4.1 Protocol Imple
11、mentation Conformance Statements (PICS) 12g35.4.2 Test Suite Structure Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org/). Pursuant to the E
12、TSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Forewo
13、rd This Technical Specification (TS) has been produced by ETSI Technical Committee Smart Machine-to-Machine communications (SmartM2M). The present document is part 1 of a multi-part deliverable covering Conformance test specifications for Smart Appliances Ontology and Communication Framework Testing
14、 as identified below: Part 1: “Testing methodology“; Part 2: “Protocol Implementation Conformance Statement (PICS) pro forma“; Part 3: “Test Suite Structure and Test Purposes (TSS Part 4: “Abstract Test Suite (ATS) and Protocol Implementation eXtra Information for Testing (PIXIT)“. Modal verbs termi
15、nology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in
16、 ETSI deliverables except when used in direct citation. ETSI ETSI TS 103 268-1 V1.1.1 (2017-04)6 1 Scope The scope of the present document is to support Smart Appliance common ontology and communication framework testing needs. It specifies a global methodology for testing for Smart Appliances, base
17、d oneM2M specifications. It analyses the overall testing needs and identifies and defines the additional documentation required. The testing framework proposed in the present document provides methodology for development of conformance and interoperability test strategies, test systems and the resul
18、ting test specifications for SAP. 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version
19、 of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at https:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETS
20、I cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present document. 1 ETSI TS 103 264: “SmartM2M; Smart Appliances; Reference Ontology and oneM2M Mapping“. 2 ETSI TS 103 267: “SmartM2M; Smart Appliances; Communication Framework“.
21、 3 ETSI TS 118 101: “oneM2M; Functional Architecture (oneM2M TS-0001)“. 4 ETSI TS 118 104: “oneM2M; Service Layer Core Protocol Specification (oneM2M TS-0004)“. 5 ETSI TS 118 112: “oneM2M; Base Ontology (oneM2M TS-0012)“. 6 ETSI TS 118 108: “oneM2M; CoAP Protocol Binding (oneM2M TS-0008)“. 7 ETSI TS
22、 118 109: “oneM2M; HTTP Protocol Binding (oneM2M TS-0009)“. 8 ETSI TS 118 110: “oneM2M; MQTT Protocol Binding (oneM2M TS-0010)“. 9 ETSI TS 118 120: “oneM2M; WebSocket Protocol Binding (oneM2M TS-0020)“. 2.2 Informative references References are either specific (identified by date of publication and/
23、or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of p
24、ublication, ETSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ETSI EG 202 798: “Intelligent Transport Systems (ITS); Testing; Framewor
25、k for conformance and interoperability testing“. i.2 ISO/IEC 9646 (all parts): “Information technology - Open Systems Interconnection - Conformance testing methodology and framework“. ETSI ETSI TS 103 268-1 V1.1.1 (2017-04)7 i.3 ETSI ES 201 873 (all parts): “Methods for Testing and Specification (MT
26、S); The Testing and Test Control Notation version 3“. i.4 ETSI EG 202 237: “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); Generic approach to interoperability testing“. i.5 ETSI EG 201 058: “Methods for Testing and Specification (MTS); Implementation Conformance State
27、ment (ICS) pro forma style guide“. i.6 ETSI ETR 266: “Methods for Testing and Specification (MTS); Test Purpose style guide“. i.7 ETSI ETS 300 406: “Methods for Testing and Specification (MTS); Protocol and profile conformance testing specifications; Standardization methodology“. i.8 ISO 10746 (all
28、parts): “Information technology - Open Distributed Processing - Reference model“. i.9 ETSI TS 103 268-2:“SmartM2M; Smart Appliances Ontology and Communication Framework Testing; Part 2: Protocol Implementation Conformance Statement (PICS) proforma“. 3 Definitions and abbreviations 3.1 Definitions Fo
29、r the purposes of the present document, the terms and definitions given in ISO 10746 i.8, ETSI TS 103 264 1, ETSI TS 103 267 2, ETSI TS 118 101 3, ETSI TS 118 104 4 and the following apply: conformance testing: process for testing that an implementation is compliant with a protocol standard, which i
30、s realized by test systems simulating the protocol with test scripts executed against the implementation under test interoperability testing: activity of proving that end-to-end functionality between (at least) two devices is as required by the base standard(s) on which those devices are based testi
31、ng framework: document providing guidance and examples necessary for the development and implementation of a test specification conformance: compliance with requirements specified in applicable standards ISO/IEC 9646 i.2 Device Under Test (DUT): combination of software and/or hardware items which im
32、plement the functionality of standards and interact with other DUTs via one or more reference points Implementation Under Test (IUT): implementation of one or more Open Systems Interconnection (OSI) protocols in an adjacent user/provider relationship, being the part of a real open system which is to
33、 be studied by testing (ISO/IEC 9646-1 i.2) interoperability: ability of two systems to interoperate using the same communication protocol interoperability test suite: collection of test cases designed to prove the ability of two (or more) systems to interoperate InterWorking Function (IWF): transla
34、tion of one protocol into another one so that two systems using two different communication protocols are able to interoperate Qualified Equipment (QE): grouping of one or more devices that has been shown and certified, by rigorous and well-defined testing, to interoperate with other equipment NOTE
35、1: Once a DUT has been successfully tested against a QE, it may be considered to be a QE, itself. NOTE 2: Once a QE is modified, it loses its status as QE and becomes again a DUT. test case: specification of the actions required to achieve a specific test purpose, starting in a stable testing state,
36、 ending in a stable testing state and defined in either natural language for manual operation or in a machine-readable language (such as TTCN-3) for automatic execution ETSI ETSI TS 103 268-1 V1.1.1 (2017-04)8 test purpose: description of a well-defined objective of testing, focussing on a single in
37、teroperability requirement or a set of related interoperability requirements 3.2 Abbreviations For the purposes of the present document, the abbreviations given in ISO 10746 i.8, ETSI TS 103 268-2 i.9, ETSI TS 118 101 3, ETSI TS 118 104 4 and the following apply: API Application Programming Interfac
38、e ATS Abstract Test Suite DUT Device Under Test EUT Equipment Under Test IFS Interoperable Features Statement IUT Implementation Under Test IWF InterWorking Function PICS Protocol Implementation Conformance Statement PIXIT Protocol Implementation eXtra Information for Testing QE Qualified Equipment
39、RP Reference Point SAP Smart Appliance SUT System Under Test TP Test Purpose TSS Test Suite Structure TST Testing 4 Introduction to the SAP testing methodology The present document provides: Identification of the implementations under test (IUT) for conformance testing and the device under test (DUT
40、s) for interoperability, i.e. answering the question “what is to be tested“. Definition of the applicable test procedures, i.e. answering the question “how is to be tested“. Definition of the procedure for development of test specifications and deliverables (for instance: TSS the high degree of cont
41、rol of the test system over the sequence and contents of the protocol messages allows to test both valid and invalid behaviour. Figure 5.1-1: Conformance testing Conformance test specifications should be produced following the methodology described in ISO/IEC 9646-1 i.2. In summary, this methodology
42、 begins with the collation and categorization of the requirements to be tested into a tabular form which is normally referred to as the “Protocol Implementation Conformance Statement“ (PICS). Each PICS relates to a specific protocol standard. In those cases where the requirements are distributed acr
43、oss a large number of documents there may be very little benefit in producing an individual PICS for each document. Consequently, the requirements should be collected together and categorized in a single document, referred as the Requirements Catalogue. The present document could be structured as an
44、 overall PICS covering the requirements of all the relevant specifications. For each requirement in the catalogue, one or more tests should be identified and classified into a number of groups which will provide a structure to the overall test suite (TSS). A brief Test Purpose (TP) should then be wr
45、itten for each identified test and this should make it clear what is to be tested but not how this should be done. Although not described or mandated in ISO/IEC 9646-1 i.2, in many situations (particularly where the TPs are complex) it may be desirable to develop a Test Description (TD) for each TP.
46、 The TD describes in plain language (often tabulated) the actions required to reach a verdict on whether an implementation passes or fails the test. Finally, a detailed Test Case (TC) is written for each TP. In the interests of test automation, TCs are usually combined into an Abstract Test Suite (A
47、TS) using a specific testing language such as TTCN-3. In summary, the SAP M2M Conformance Testing methodology consists of: Selection of Implementations Under Test (IUT). Identification of reference points. Development of test specifications, which includes: - Development of “Implementation Conforman
48、ce Statements“ (ICS), if not already provided as part of the base standard. - Development of “Test Suite Structure and Test Purposes“ (TSS - invalid behaviour test; - timer test; - etc. Usually the identification of the different functional groups of procedures leads to the definition of the top lev
49、els of the TSS. Then further levels at the bottom of the TSS is used to group test purposes belonging to the same type of test. Table 5.4.2.2-1 shows SAP Test Suite Structure (TSS) including its subgroups defined for conformance testing. Table 5.4.2.2-1: TSS for SAP TST Root Group Sub-group category SAP SAREF XXX Valid behaviour Valid behaviourInvalid Syntax or Behaviour Tests Inopportune Behaviour The test suite is structured as a tree with the root defined as SAP. The tree is of rank 3 with the first
