1、 ETSI GS MEC-IEG 006 V1.1.1 (2017-01) Mobile Edge Computing; Market Acceleration; MEC Metrics Best Practice and Guidelines Disclaimer The present document has been produced and approved by the Mobile Edge Computing (MEC) ETSI Industry Specification Group (ISG) and represents the views of those membe
2、rs who participated in this ISG. It does not necessarily represent the views of the entire ETSI membership. GROUP SPECIFICATION ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 2 Reference DGS/MEC-IEG006Metrics Keywords MEC, KPI ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33
3、4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available i
4、n electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevai
5、ling document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI
6、documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized i
7、n any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reprodu
8、ction in all media. European Telecommunications Standards Institute 2017. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3
9、GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 3 Contents Intellectual Property Rights 5g3Foreword . 5g3Modal verbs terminology 5g3Introduction 5g31 Scope 6g32 References 6g32.1 Normative refere
10、nces . 6g32.2 Informative references 6g33 Definitions and abbreviations . 7g33.1 Definitions 7g33.2 Abbreviations . 8g34 Metrics 9g34.1 General . 9g34.2 Latency . 10g34.2.1 General 10g34.2.2 Round-Trip Time 10g34.2.3 One-Way Delay (OWD) . 11g34.2.4 Set-up Time 11g34.2.5 Service Processing Time . 11g
11、34.2.6 Context-update time 12g34.3 Energy efficiency . 12g34.4 Network throughput . 13g34.5 System resource footprint . 14g34.5.1 General 14g34.5.2 Computational load . 14g34.5.3 Non user data volume exchange . 14g34.6 Quality 14g34.6.1 General 14g34.6.2 Objective and service-independent metrics abo
12、ut quality. 14g34.6.3 Objective and service-dependent metrics about quality 15g34.6.4 Subjective and service-dependent metrics about quality 15g34.6.5 Objective metrics about user comfort . 15g35 Measurement methodology 16g35.1 General . 16g35.2 Evaluation of latency 16g35.2.0 Introduction. 16g35.2.
13、1 Measurement methodology . 17g35.2.1.1 Peak workload test 17g35.2.1.2 Uniform workload tests . 17g35.2.1.3 Stress tests. 17g35.2.2 Latency measurement setup 1: passive measurements at the terminal . 17g35.2.3 Latency measurement setup 2: passive measurements by probes . 18g35.2.4 Latency measuremen
14、t setup 3: active measurements 19g35.3 Evaluation of energy efficiency 19g35.3.1 Introduction. 19g35.3.2 Measurement methodology . 20g35.3.2.1 General 20g35.3.2.2 Energy efficiency measurement setup 1 (network side) 20g35.3.2.2.1 General considerations 20g35.3.2.2.2 Baseline: measurement without the
15、 MEC Server 21g35.3.2.2.3 Frontline: measurement with the MEC Server 21g35.3.2.2.4 Computation of EE gains . 21g35.3.2.3 Energy efficiency measurement setup 2 (terminal side) . 22g3ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 4 5.3.2.3.1 General considerations 22g35.3.2.3.2 Baseline: measurement withou
16、t the MEC Server 22g35.3.2.3.3 Frontline: measurement with the MEC Server 22g35.3.2.3.4 Computation of EE gains . 23g35.4 Evaluation of network throughput 23g35.4.1 General 23g35.4.2 Network throughput measurement setups . 23g35.5 Evaluation of resource footprint . 24g35.5.1 General 24g35.5.2 Comput
17、ational load measurement setup 1: isolated execution environment 24g3Annex A (informative): Network Throughput Example . 25g3Annex B (informative): Examples of metric value ranges 26g3B.1 5G latency requirements . 26g3B.2 5G energy efficiency 26g3Annex C (informative): POC#3 RAVEN - example of laten
18、cy metric assessment . 27g3History 29g3ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 5 Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI m
19、embers and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.e
20、tsi.org/). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to
21、the present document. Foreword This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Mobile Edge Computing (MEC). Modal verbs terminology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “canno
22、t“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in ETSI deliverables except when used in direct citation. Introduction Mobile Edge Computing is a new technology that provides an IT s
23、ervice environment and cloud-computing capabilities at the edge of the mobile network, in close proximity to mobile subscribers. In order to make MEC a success and encourage network operators to deploy Mobile Edge (ME) systems as well as to make MEC attractive to application developers and service p
24、roviders, it is necessary to demonstrate the benefits of this technology for fulfilling various requirements. In order to make MEC an attractive proposition for service providers and applications developers to host their applications on a ME Host instead of in a centralized cloud, it is important to
25、 demonstrate a quantifiable performance increase. The present document describes a number of performance metrics which can be used to demonstrate the benefits of deploying services and applications on a ME Host compared to a centralized cloud or server. Examples of how these metrics can be measured
26、are also described. Examples of such metrics KPIs are reducing latency, increasing end-to-end energy efficiency and increasing network throughput. ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 6 1 Scope The present document describes various metrics which can potentially be improved through deploying a
27、service on a MEC platform. Example use cases are used to demonstrate where improvements to a number of key performance indicators can be identified in order to highlight the benefits of deploying MEC for various services and applications. Furthermore, the present document describes best practices fo
28、r measuring such performance metrics and these techniques are further exemplified with use cases. Metrics described in the present document can be taken from service requirements defined by various organizations (e.g. 5G service requirements defined by Next Generation Mobile Networks (NGMN) or 3rdGe
29、neration Partnership Project (3GPP). An informative annex is used to document such desired and/or achieved ranges of performance which could be referenced from the main body of the present document. 2 References 2.1 Normative references References are either specific (identified by date of publicati
30、on and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in th
31、e expected location might be found at https:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present docume
32、nt. 1 ETSI ES 202 706 (V1.4.1): “Environmental Engineering (EE); Measurement method for power consumption and energy efficiency of wireless access network equipment“. 2 ETSI ES 203 228 (V1.1.1): “Environmental Engineering (EE); Assessment of mobile network energy efficiency“. 3 ETSI GS MEC 002: “Mob
33、ile Edge Computing (MEC); Technical Requirements“. 4 ETSI GS MEC 001: “Mobile Edge Computing (MEC); Terminology“. 5 ETSI ES 202 336-12: “Environmental Engineering (EE); Monitoring and control interface for infrastructure equipment (power, cooling and building environment systems used in telecommunic
34、ation networks); Part 12: ICT equipment power, energy and environmental parameters monitoring information model“. 2.2 Informative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the c
35、ited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced docu
36、ments are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 IETF RFC 4656: “One way active measurement protocol“. i.2 IETF RFC 5357: “A two-way active measurement protocol“. ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 7 i.3
37、 IETF IP Performance Metrics Working Group: IPPM status pages. NOTE: Available at https:/tools.ietf.org/wg/ippm/. i.4 IETF IP Performance Metrics Working Group: Charter. NOTE: Available at https:/tools.ietf.org/wg/ippm/charters. i.5 NGMN Alliance 5G White Paper version 1.0 (17 February 2015): “NGMN
38、5G White Paper“. NOTE: Available at https:/www.ngmn.org/uploads/media/NGMN_5G_White_Paper_V1_0.pdf. i.6 J. S. Milton, J. Arnold, “Introduction to Probability and Statistics“, McGraw-Hill Education, 4th Edition. i.7 P. Serrano, M. Zink, J. Kurose, “Assessing the fidelity of COTS 802.11 sniffers“, IEE
39、E INFOCOM 2009, Rio de Janeiro, Brazil, April 2009. i.8 P. Serrano, A. Garcia-Saavedra, G. Bianchi, A. Banchs, A. Azcorra, “Per-frame Energy Consumption in 802.11 Devices and its Implication on Modeling and Design,“ IEEE/ACM Transactions on Networking, vol.23, no.4, pp.1243-1256, Aug. 2015. i.9 N Va
40、llina-Rodriguez, J Crowcroft, “Energy Management Techniques in Modern Mobile Handsets,“ IEEE Communications Surveys 2) assessment of MEC deployments: comparison between different ME host positions within the network. In both cases, the goal is not to compare different vendors or solution providers,
41、but to assess the improvement of MEC introduction with respect to a traditional system (without MEC), e.g. in order to understand the different deployment options against the different use cases (e.g. by minimizing costs, maximizing benefits or flexibility). For this reason, MEC metrics can be class
42、ified into two main groups: functional and non-functional metrics. For both categories (defined here below), metrics can be referred to different MEC use cases, as listed in IETF RFC 4656 i.1, and the actual assessment of these metrics can depend on the particular service and/or application utilizat
43、ion: 1) Functional metrics are related to MEC performances impacting on user perception (often called also KPIs, key performances indicators): - Examples of functional service performance KPIs include: latency (both end-to-end, and one-way), energy efficiency, throughput, goodput, loss rate (number
44、of dropped packets), jitter, number of out-of-order delivery packets, QoS, and MOS. Each of the functional metrics should be defined on per service basis. Note that the latency in localization (time to fix the position) is different from latency in content delivery. 2) Non-functional metrics are rel
45、ated to the performance of the service in terms of deployment and management: - Examples of non-functional metrics include: service lifecycle (instantiation, service deployment, service provisioning, service update (e.g. service scalability and elasticity), service disposal), service availability an
46、d fault tolerance (aka reliability), service processing/computational load, global ME host load, number of API request (more generally number of events) processed/second on ME host, delay to process API request (north and south), number of failed API request. The sum of service instantiation, servic
47、e deployment, and service provisioning provide service boot-time. In both cases, one could measure all the statistics over the above metrics. In fact, all metrics are in principle time-variable, and could be measured in a defined time interval and described by a profile over time or summarized throu
48、gh: the maximum value; mean and minimum value; standard deviation; the value of a given percentile; etc. All MEC metrics assessments can be done by considering the overall system, or portions of that, according to the purpose of the measurement itself. An example below (figure 1) shows a mobile netw
49、ork system with ME host, and the different entities potentially involved in the assessment. ETSI ETSI GS MEC-IEG 006 V1.1.1 (2017-01) 10 Figure 1: Measuring MEC metrics 4.2 Latency 4.2.1 General The concept of latency is wide and encompasses manifolds metrics: in communications, latency refers to a time-interval whose measurement quantifies the delay elapsed between any event and a consequent target effect. Even more, still in the communication domain, latency is useful to measure phenomena both in the control plane (e.g. set-up tim
