1、 ETSI TS 103 222-4 V1.1.1 (2015-12) Speech and multimedia Transmission Quality (STQ); Reference benchmarking, background traffic profiles and KPIs; Part 4: Reference benchmarking for IPTV, Web TV and RCS-e Video Share TECHNICAL SPECIFICATION ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)2 Reference DTS/STQ
2、-219-4 Keywords KPI, QoS ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice The present document c
3、an be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of
4、any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document ma
5、y be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: https:/portal.etsi.org/Pe
6、ople/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the
7、 written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2015. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
8、3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)3 Contents Intellectual Property Rights 4g3Foreword . 4g3
9、Modal verbs terminology 4g3Introduction 4g31 Scope 5g32 References 5g32.1 Normative references . 5g32.2 Informative references 6g33 Definitions and abbreviations . 6g33.1 Definitions 6g33.2 Abbreviations . 6g34 Management Summary. 7g35 Reference Benchmarking . 7g35.1 Introduction 7g35.2 IPTV Quality
10、 9g35.2.1 Audiovisual quality . 9g35.2.2 Functional QoS parameters of the video channel . 9g35.2.3 Network performance parameters in the video channel 10g35.3 IPTV benchmark measurements . 10g35.3.1 IPTV STB Startup Response Time . 10g35.3.2 IPTV STB Startup Time . 11g35.3.3 IPTV STB Startup Failure
11、 Ratio . 11g35.3.4 IPTV Zapping Response Time 11g35.3.5 IPTV Zapping Time 11g35.3.6 IPTV Zapping Failure Ratio . 12g35.3.7 IPTV Packet Loss . 12g35.3.8 IPTV Bitrate Video/Other/Total . 13g35.4 WebTV . 13g35.4.1 Introduction. 13g35.4.2 Progressive Download 13g35.4.2.1 Introduction . 13g35.4.2.2 WebTV
12、 Quality . 14g35.4.2.3 WebTV Initial Loading Time 14g35.4.2.4 WebTV Number of Stalling Events 14g35.4.2.5 WebTV Average Duration of Stalling Events . 14g35.4.2.6 WebTV QoS parameters . 14g35.4.3 Adaptive Streaming 16g35.4.3.1 Introduction . 16g35.4.3.2 WebTV Quality . 16g35.4.3.3 WebTVQoS parameters
13、 16g35.5 RCS-e Video Share 16g35.5.1 Introduction. 16g35.5.2 RCS-e Video Share Setup Time . 16g35.5.3 RCS-e Video Share Setup Failure Ratio . 17g35.5.4 RCS-e Bitrate Video/Total 17g35.5.5 RCS-e Video Share I-/P-/B-Slices 17g35.5.6 RCS-e Video Share Packet Loss . 18g35.5.7 RCS-e Video Quality Score 1
14、8g35.5.8 RCS-e Video Share Drop Ratio 18g35.5.9 RCS-e Video Share Failure Ratio . 18g3Annex A (informative): Bibliography . 19g3History 20g3ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)4 Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to
15、ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is avai
16、lable from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/ipr.etsi.org). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETS No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000
17、314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specification (TS) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality (STQ). The present document is part 4 of a multi-par
18、t deliverable covering the Reference benchmarking, background traffic profiles and KPIs as identified below: Part 1: “Reference benchmarking, background traffic profiles and KPIs for VoIP and FoIP in fixed networks“; Part 2: “Reference benchmarking and KPIs for High speed internet“; Part 3: “Referen
19、ce benchmarking, background traffic profiles and KPIs for UMTS and VoLTE“; Part 4: “Reference benchmarking for IPTV, Web TV and RCS-e Video Share“. Modal verbs terminology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“
20、are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in ETSI deliverables except when used in direct citation. Introduction The present document describes the reference benchmarking, backgro
21、und traffic profiles and key performance Indicators for IPTV, WebTV and Video RCS-e. ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)5 1 Scope The offer of new NGN services requires new: KPIs; QoS measurement; and benchmarking methods; which are needed to ensure the quality of new services. To ensure the com
22、parability of test results, reference benchmarking methods and background traffic load profiles are needed. The present document describes key performance indicators and benchmarking methods for the spectrum of potential applications. All access technologies offered by the operator under test are co
23、nsidered. 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference docume
24、nt (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their l
25、ong term validity. The following referenced documents are necessary for the application of the present document. 1 Recommendation ITU-T E.800 (2008): “Definitions of terms related to quality of service“. 2 Recommendation ITU-T Y.1541 (2011): “Network performance objectives for IP-based services“. 3
26、IETF RFC 3357: “One-way Loss Pattern Sample Metrics“. 4 Recommendation ITU-T P.1201 (2012): “Parametric non-intrusive assessment of audiovisual media streaming quality“. 5 Recommendation ITU-T P.1201.2 (2012): “Parametric non-intrusive assessment of audiovisual media streaming quality - Higher resol
27、ution application area“. 6 Recommendation ITU-T P.1202 (2012): “Parametric non-intrusive bitstream assessment of video media streaming quality“. 7 Recommendation ITU-T P.1202.2 (2013): “Parametric non-intrusive bitstream assessment of video media streaming quality - Higher resolution application are
28、a“. 8 Recommendation ITU-T P.1201 Amendment 2 (2013): “New Appendix III - Use of P.1201 for non-adaptive, progressive download type media streaming“. ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)6 2.2 Informative references References are either specific (identified by date of publication and/or edition n
29、umber or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, E
30、TSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ETSI TR 101 578 (V1.1.1): “Speech and multimedia Transmission Quality (STQ); QoS aspe
31、cts of TCP-based video services like YouTube“. 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the following terms and definitions apply: benchmark: evaluation of performance value/s of a parameter or set of parameters for the purpose of establishing value/s
32、 as the norm against which future performance achievements may be compared or assessed NOTE: The definition is taken from Recommendation ITU-T E.800 1. 3.2 Abbreviations For the purposes of the present document, the following abbreviations apply: AAC Advanced Audio Coding CS Circuit Switched GSMA GS
33、M Association HDTV High-Definition Television HEVC High Efficiency Video Coding HTTP HyperText Transfer Protocol IP Internet Protocol IPDV IP Delay Variation IPLR IP Packet Loss Ratio IPTD IP Packet Transfer Delay IPTV IP Television ITU-T Telecommunication Standardization Sector of the International
34、 Telecommunications Union MOS Mean Opinion Score MPEG Moving Picture Experts Group NGN New Generation Network PCAP Packet CAPtures QoE Quality of Experience QoS Quality of Service RCS Rich Communication Services RFC Requests for Comments SDTV Standard-Definition Television SIP Session Initiation Pro
35、tocol STB Set-Top Box STQ Speech and multimedia Transmission Quality TCP Transmission Control Protocol TS Technical Specification TV Television UMTS Universal Mobile Telecommunications System ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)7 UNI User Network Interface UNI-to-UNI User Network Interface to Use
36、r Network Interface WAP Wireless Access-Point WSP Wireless Session Protocol 4 Management Summary The assessment of IPTV QoS and QoE from an end-user perspective is performed through MOS values for the video and audio signals in the transmitted streams while using the set-top box and the associated r
37、emote control. The video quality is estimated based on several parameters extracted from the transmitted bitstream. The measurement of the so-called “zapping time“ provides information about the time it takes to switch between TV channels. 5 Reference Benchmarking 5.1 Introduction To determine the q
38、uality of IPTV, WebTV and RCS-e the following measurement values are determined. Table 1: Overview of quality benchmarks for IPTV measurements IPTV 1. IPTV Quality (P.1201, P.1202) 2. IPTV STB Startup Response Time 3. IPTV STB Startup Time 4. IPTV STB Startup Failure Ratio 5. IPTV Zapping Response T
39、ime 6. IPTV Zapping Time 7. IPTV Packet Loss 8. IPTV Bitrate Video/Other/Total ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)8 Table 2: Overview of quality benchmarks for WebTV measurements WebTV 1. Progressive Download 2. WebTV Quality 3. WebTV Initial Loading Time 4. WebTV Average Duration of Stalling Ev
40、ents 5. WebTV QoS parameters Player IP Service Access Failure Ratio Player IP Service Access Time Player Download Cut-off Ratio Player Download Time Player Session Failure Ratio Player Session Time Video IP Service Access Failure Ratio Video IP Service Access Time Video Reproduction Start Failure Ra
41、tio Video Reproduction Start Delay Video Play Start Failure Ratio Video Play Start Time IP Service Access Failure Ratio IP Service Access Time Video Session Cut-off Ratio Video Session Time Impairment Free Video Session Ratio Expected Size Downloaded Size Compression Ratio Transfer Cut-off Ratio Tra
42、nsfer Time Mean User Data Rate Playout Cut-off Ratio Playout Cut-off Time Expected Duration Playout Duration Freeze Occurrences Count Accumulated Video Freezing Duration Video Skip Occurrences Count Accumulated Video Skips Duration Video Maximum Freezing Duration Video Freezing Impairment Ratio Fail
43、ure Ratio Video Freezing Time Proportion End-to-End Session Failure Ratio Table 3: Overview of quality benchmarks for RCS-e measurements RCS-e Video 1. RCS-e Video Share Setup Time 2. RCS-e Video Quality Score 3. RCS-e Video Share Setup Failure Ratio 4. RCS-e Bitrate Video/Total 5. RCS-e I-/P-/B-Sli
44、ces 6. RCS-e Packet Loss 7. RCS-e Video Share Failure Ratio 8. RCS-e Video Share Drop Ratio ETSI ETSI TS 103 222-4 V1.1.1 (2015-12)9 5.2 IPTV Quality 5.2.1 Audiovisual quality The IPTV audiovisual quality is estimated by bitstream/parametric models, which have no access to a reference signal (see no
45、te) and are therefore called non-intrusive. An audiovisual, packet-header-based quality model is described in Recommendation ITU-T P.1201.2 5 and two differently complex bitstream-based video quality models in Recommendation ITU-T P.1202.2 7. These are suited for the high resolution application area
46、s (i.e. HDTV, including 720p25, 1080i50, 1080p25, 1080p50, 1080p60) as well as SDTV, that is, the video resolutions that are relevant for IPTV. NOTE: A full-reference approach, in which a reference video signal is compared to the signal at the receiver side, is neither practically feasible due to th
47、e amount of data processing needed, nor guaranteeing a fair estimation of the perceived quality, as the reference content could be specifically prepared so as to deliver appealing (D)MOS results. The models receive as input the media stream as packet captures (PCAP) and may also use out-of-band info
48、rmation (such as video resolution) which may not be part of the packet information. P.1201 only operates on the packet header information to provide audiovisual quality estimation, whereas P.1202 may also decode part of the video bitstream for a more precise assessment. Therefore, P.1202 is computat
49、ionally more complex, and also requires a non-encrypted video payload. The test sequences are 10 seconds long. The selected video sequences should cover different types of content. In particular, it is proposed to use at least three different types of content ranging from mostly stable scenes to fast moving scenes. Naturally, a type of the content chosen for a test should be carefully described in a test report. For the analysis, also running live IPTV sessions can be used, co
