1、 ETSI TR 126 909 V15.0.0 (2018-07) Universal Mobile Telecommunications System (UMTS); LTE; Study on improved streaming Quality of Experience (QoE) reporting in 3GPP services and networks (3GPP TR 26.909 version 15.0.0 Release 15) TECHNICAL REPORT ETSI ETSI TR 126 909 V15.0.0 (2018-07)13GPP TR 26.909
2、 version 15.0.0 Release 15Reference RTR/TSGS-0426909vf00 Keywords LTE,UMTS ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06)
3、 N 7803/88 Important notice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without th
4、e prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the pre
5、sent document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment
6、 to one of the following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The
7、content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. ETSI 2018. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are trademarks of ETSI registered for the benefit
8、of its Members. 3GPPTM and LTETMare trademarks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. oneM2M logo is protected for the benefit of its Members. GSMand the GSM logo are trademarks registered and owned by the GSM Association. ETSI ETSI TR 126 909 V15.
9、0.0 (2018-07)23GPP TR 26.909 version 15.0.0 Release 15Intellectual Property Rights Essential patents IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members an
10、d non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org/)
11、. Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the prese
12、nt document. Trademarks The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners. ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no right to use or reproduce any trademark and/o
13、r tradename. Mention of those trademarks in the present document does not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks. Foreword This Technical Report (TR) has been produced by ETSI 3rd Generation Partnership Project (3GPP). The present do
14、cument may refer to technical specifications or reports using their 3GPP identities, UMTS identities or GSM identities. These should be interpreted as being references to the corresponding ETSI deliverables. The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under http:/web
15、app.etsi.org/key/queryform.asp. Modal verbs terminology In the present document “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “m
16、ust not“ are NOT allowed in ETSI deliverables except when used in direct citation. ETSI ETSI TR 126 909 V15.0.0 (2018-07)33GPP TR 26.909 version 15.0.0 Release 15Contents Intellectual Property Rights 2g3Foreword . 2g3Modal verbs terminology 2g3Foreword . 4g31 Scope 5g32 References 5g33 Abbreviations
17、 . 5g34 A/V Quality Monitoring Support for 3GPP PSS 6g34.1 WebTV Quality Monitoring . 6g34.1.1 Use case 6g34.1.2 Potential Recommended requirements 7g34.2 ITU-T P.NATS Quality Assessment Model for HTTP Adaptive Streaming . 8g34.3 Gap analysis of PSS QoE metrics for support of ITU-T P.NATS 11g34.3.1
18、Supported Mode 11g34.3.2 Supported Input parameter 11g34.3.3 Further Parameters for Supporting Video Quality Monitoring . 15g34.3.3.1 Additional Input Parameters 15g34.3.3.2 Dependency of Video Quality Test Results on Visual Size 15g34.3.3.3 Relation between Video Quality MOS and QP Parameter 16g34.
19、4 Calculation of A/V MOS estimation 17g34.4.0 General 17g34.4.1 Network Optimization 17g34.4.2 MOS Models . 17g34.4.3 MOS Windowing 17g34.4.4 Conclusion 18g34.5 Options for applying higher P.NATS Modes . 18g35 QoE metrics support for managed streaming service . 20g35.1 Use case: Managed Streaming Se
20、rvice QoE Improvement 20g35.1.1 Introduction. 20g35.1.2 Use case #1 . 20g35.1.3 Use case #2 . 21g35.2 Managed streaming service deployment model consideration . 21g35.2.1 Introduction. 21g35.2.2 Managed streaming service deployment model 22g35.3 Recommended requirements 23g35.4 GAP Analysis and Eval
21、uation 23g35.4.1 Introduction. 23g35.4.2 Analysis of Activation/Deactivation of QoE reporting . 23g35.5 Assumptions . 23g35.6 Solution 24g35.6.1 Possible Candidate Options 24g35.6.2 QoE metrics collection within a certain geographic area 26g35.6.3 QoE metrics collection of a streaming service of a s
22、pecific 3rd party 26g35.7 QoE Handler API Considerations 26g35.8 Privacy issue on QoE metric collection 28g35.8.1 Analysis 28g35.8.2 Solutions . 28g36 Conclusion 28g3Annex A: Change history 30g3History 31g3ETSI ETSI TR 126 909 V15.0.0 (2018-07)43GPP TR 26.909 version 15.0.0 Release 15Foreword This T
23、echnical Report has been produced by the 3rdGeneration Partnership Project (3GPP). The contents of the present document are subject to continuing work within the TSG and may change following formal TSG approval. Should the TSG modify the contents of the present document, it will be re-released by th
24、e TSG with an identifying change of release date and an increase in version number as follows: Version x.y.z where: x the first digit: 1 presented to TSG for information; 2 presented to TSG for approval; 3 or greater indicates TSG approved document under change control. y the second digit is increme
25、nted for all changes of substance, i.e. technical enhancements, corrections, updates, etc. z the third digit is incremented when editorial only changes have been incorporated in the document. ETSI ETSI TR 126 909 V15.0.0 (2018-07)53GPP TR 26.909 version 15.0.0 Release 151 Scope The present document
26、describes the enhancement of Quality of Experience (QoE) for operator managed streaming service, 3rdparty managed streaming service and Over-The-Top (OTT) streaming service. 2 References The following documents contain provisions which, through reference in this text, constitute provisions of the pr
27、esent document. - References are either specific (identified by date of publication, edition number, version number, etc.) or non-specific. - For a specific reference, subsequent revisions do not apply. - For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP
28、document (including a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same Release as the present document. 1 3GPP TR 21.905: “Vocabulary for 3GPP Specifications“. 2 Yiting Liao, et al. “Achieving high QoE across the compute continuum: How comp
29、ression, content, and devices interact.“ 7th International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, Arizona, USA. 2013. 3 ITU-T SG12 “T13-SG12-150505-TD-GEN-0671!R1!MSW-E-P.NATS Terms of Reference (ToR)“. 4 3GPP TS 26.247: “Transparent end-to-end Packet-
30、switched Streaming Service (PSS); Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)“. 5 3GPP TS 26.233: “Transparent end-to-end packet switched streaming service (PSS); General description“. 6 3GPP TR 26.938: “Packet-switched Streaming Service (PSS); Improved support for dynam
31、ic adaptive streaming over HTTP in 3GPP“. 7 3GPP TS 26.244: “Transparent end-to-end packet switched streaming service (PSS); 3GPP file format (3GP)“. 8 Recommendation ITU-R BT.500: “Methodology for the subjective assessment of the quality of television pictures“. 9 DASH-IF Position Paper: “Proposed
32、QoE Media Metrics standardization for segmented media playback“: http:/dashif.org/wp-content/uploads/2016/10/ProposedMediaMetricsforSegmentedMediaDelivery-r12.pdf 10 ISO/IEC 23001-10: “Information technology - MPEG systems technologies - Part 10: Carriage of timed metadata metrics of media in ISO ba
33、se media file format“. 11 3GPP TS 32.422: “Telecommunication management; Subscriber and equipment trace; Trace control and configuration management“. 3 Abbreviations For the purposes of the present document, the abbreviations given in 3GPP TR 21.905 1 and the following apply. An abbreviation defined
34、 in the present document takes precedence over the definition of the same abbreviation, if any, in 3GPP TR 21.905 1. A/V MOS Audio/Video Mean Opinion Score IQoE Improved Quality of Experience DASH Dynamic Adaptive Streaming over HTTP ETSI ETSI TR 126 909 V15.0.0 (2018-07)63GPP TR 26.909 version 15.0
35、.0 Release 15KPI Key Performance Indicators KQI Key Quality Indicator MDT Minimization of Drive Tests MNO Mobile Network Operator MOS Mean Opinion Score MOS-AVQO Audio-Visual-Quality-Objective MPD Media Presentation Description OTT Over The Top PSS Packet Streaming Server TCE Trace Collection Entity
36、 VSSQM Video Streaming Service Quality Monitoring 4 A/V Quality Monitoring Support for 3GPP PSS 4.1 WebTV Quality Monitoring 4.1.1 Use case Over-The-Top video streaming is increasingly dominating the traffic in the networks. An increasing number of services employing a variety of streaming formats a
37、re appearing. In addition, movie services are increasingly causing traffic in mobile networks. It is crucial for mobile network operators to manage the video traffic in their networks and services in an optimal manner. A major objective is to ensure that the customers remain satisfied. Hence monitor
38、ing the users QoE with an appropriate quality indicator is of fundamental importance. The derivation of quality indicators for the streaming quality may be supported by the service to a lesser or greater extend. Further dedicated agreements between mobile operator and service provider would help to
39、obtain better quality indicators. The level of accuracy on the quality indicator would depend on the level of agreement between the mobile network operator and the service provider. The main Key Performance Indicators (KPIs) for characterizing insufficient video streaming performance as perceived by
40、 the user are: - initial stalling of the playout; - periods of stalling and freezing of the video while playing; - interruption of the audio while playing; - low coding quality appearing as blurring, macroblocking or mosquito artefacts; - varying coding quality while playing. Beside these KPIs the m
41、onitoring system will also provide a Key Quality Indicator (KQI) characterizing the users quality experience. Quality is fundamentally related to the subjective assessment of the considered aspect. The KQI will be related to corresponding subjective quality assessments. The quality often is rated as
42、 an opinion score on a 5-point scale ranging from “bad“ (1), “poor“ (2), “fair“ (3), “good“ (4) to “excellent“ (5). The average of these scores calculated from a group of subjects is the Mean Opinion Score (MOS). For operational tasks instrumental measurement tools are required. Hence, the subjectiv
43、e test results are used to develop instrumental methods that replicates the MOS scores obtained in subjective tests. Ideally, the derived quality model will be established as internationally agreed standard specification, for increasing the confidence that the measured quality results are reliable a
44、nd comparable. The estimated MOS values will be labelled as result stemming from an objective model. For example, the estimated AV-streaming MOS could be named MOS-AVQO (Audio-Visual-Quality-Objective). The use of a KQI, estimating the MOS of a video streaming session has the major advantage of comb
45、ining the quality impact of occurring stalling and coding effects in a single number. By this means the service can be monitored efficiently. ETSI ETSI TR 126 909 V15.0.0 (2018-07)73GPP TR 26.909 version 15.0.0 Release 15In addition to the KPIs and KQI, further the related data such as time, locatio
46、n, service and network will be collected for each considered streaming session. The use case “WebTV Quality Monitoring“ demands for means that allow collecting accurate KPIs and the audio-visual KQI for supporting the quality management of the networks and customer services. The collected KPI and KQ
47、I data supports a wide range of utilizations such as the short-term detection of problems and the long term quality monitoring. With additional other monitoring data sources, the complete delivery chain, from server, via networks, to the users terminal can be observed. Data analytics methods can be
48、applied to reveal inherent quality dependencies. The detection of critical combinations of KPIs with the help of a KQI focuses on the users quality experience and is therefore very efficient. In conclusion, the MOS-AVQO is foreseen as mean to increase the usefulness of the quality monitoring systems
49、 significantly. 4.1.2 Potential Recommended requirements The proposed potential requirements for implementing the WebTV Quality Monitoring use case are based on the assumption that the users end device is collecting and aggregating the QoE-related data. The collected QoE-related data then is further processed and forwarded to a central QoE monitoring server in a next step: - The monitoring software of the end device can provide a generic applicable subsystem for Video Streaming Service Quality Monitoring (VSSQM). - The VSSQM subsystem provides
