1、 ETSI TR 102 714 V1.1.1 (2011-08)Technical Report Speech and multimedia Transmission Quality (STQ);Multimedia quality measurement;End-to-end quality measurement frameworkETSI ETSI TR 102 714 V1.1.1 (2011-08) 2Reference DTR/STQ-00113 Keywords multimedia, QoS ETSI 650 Route des Lucioles F-06921 Sophia
2、 Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice Individual copies of the present document can be downloaded from: http:/www.etsi.org The
3、present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF). In case of dispute, the reference shall be the printing on ETSI prin
4、ters of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/portal.etsi.org/t
5、b/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced except as authorized by written permission. The copyright and the foregoing rest
6、riction extend to reproduction in all media. European Telecommunications Standards Institute 2011. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of
7、 its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TR 102 714 V1.1.1 (2011-08) 3Contents Intellectual Property Rights 4g3Foreword . 4g3Introduction 4g31 Scope 5g32 References 5g32.1 Normative references .
8、5g32.2 Informative references 5g33 Abbreviations . 7g34 Measurement methods and tools 8g34.1 Subjective vs. objective assessments 8g34.2 Measurement approach . 9g34.3 Classification of objective media quality assessment tools 9g34.4 MOS principles and values . 10g34.5 Audio quality algorithm . 11g34
9、.6 Video quality algorithms 11g34.6.1 ITU-T Recommendation J.144 . 11g34.6.2 ITU-T Recommendations J.246 and J.247 12g34.6.3 ITU-T Recommendation J.341 . 13g34.6.4 Error calculation . 13g34.7 Media Delivery Index (MDI) . 13g34.8 Non-intrusive evaluation models of multimedia quality 14g34.9 Audio-Vid
10、eo Interaction . 14g34.9.1 Lip synchronization 15g34.9.2 Quality interaction 15g34.10 Transmission degradation . 15g34.10.1 Packet loss 15g34.10.2 Delay and delay variation . 15g34.10.3 QoS mechanisms 16g34.10.3.1 QoS mechanisms at link, network or transport layer 16g34.10.3.2 QoS mechanisms at appl
11、ication layer . 17g34.11 IP protocols 18g35 Quality Measurement Framework 18g35.1 Pre-transmission Measurement 19g35.2 Network Measurement . 20g35.3 Post-transmission Measurement . 20g3Annex A: Bibliography 22g3History 23g3ETSI ETSI TR 102 714 V1.1.1 (2011-08) 4Intellectual Property Rights IPRs esse
12、ntial or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentia
13、lly Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/ipr.etsi.org). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guar
14、antee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimed
15、ia Transmission Quality (STQ). Introduction The present document describes a framework for end-to-end objective quality measurement of multimedia services. The present document outlines a framework that specifies measurement options (e.g. parametric measures, perceptual models) at different points i
16、n the transmission chain. The framework defines measurement options for performing pre-transmission quality assurance, in-service network quality measurement, and for client-device quality monitoring. ETSI ETSI TR 102 714 V1.1.1 (2011-08) 51 Scope The present document provides a framework for end-to
17、-end quality measurement of multimedia services. The framework defines measurement options for performing pre-transmission quality assurance, in-service network quality measurement, and for client-device quality monitoring at different points in the transmission chain. The document is defined for se
18、rvices delivered on an IP network and is concerned solely with non-interactive services (e.g. IPTV, video streaming). The present document is intended to provide industry and metric developers with a framework for implementing the appropriate quality metric at each point in the transmission chain. R
19、ecommendations are provided on the type and purpose of metric(s) for use at the head-end, network and end device (e.g. STB). Wherever possible, existing standardised metrics and measurement methods being appropriate for use in measurement tools in the transmission chain belonging to the defined fram
20、ework are identified. 2 References References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including
21、 any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term vali
22、dity. 2.1 Normative references The following referenced documents are necessary for the application of the present document. Not applicable. 2.2 Informative references The following referenced documents are not necessary for the application of the present document but they assist the user with regar
23、d to a particular subject area. i.1 ITU-T Recommendation P.800 (1996): “Methods for subjective determination of transmission quality“. i.2 ITU-R Recommendation BT.500-10 (2000): “Methodology for the subjective assessment of the quality of television pictures“. i.3 ITU-R Recommendation BT.1788 (2006)
24、: “SAMVIQ - subjective assessment methodology for video quality“. i.4 ITU-R Recommendation BS.1116-1 (1997): “Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems“. i.5 ITU-R Recommendation BS.1284 (1997): “Methods for the subjective assess
25、ment of sound quality“. i.6 ITU-R Recommendation BS.1534 (2001): “Method for the subjective assessment of intermediate quality level of coding systems“. i.7 ITU-R Recommendation BS.775-2 (2006): “Multichannel stereophonic sound system with and without accompanying picture“. ETSI ETSI TR 102 714 V1.1
26、.1 (2011-08) 6i.8 ITU-R Recommendation BS.1286 (2007): “Methods for the subjective assessment of audio systems with accompanying picture“. i.9 ITU-T Recommendation P.910 (1999): “Subjective video quality assessment methods for multimedia applications“. i.10 ITU-T Recommendation P.911 (1998): “Subjec
27、tive audiovisual quality assessment methods for multimedia applications“. i.11 ITU-T Recommendation P.920 (1996): “Interactive test methods for audiovisual communications“. i.12 ITU-T Recommendation G.1070: “Opinion model for video-telephony applications“. i.13 IETF RFC 4445: “A proposed Media Deliv
28、ery Index (MDI)“. i.14 ITU-T Recommendation P.800.1: “Mean Opinion Score (MOS) terminology“. i.15 ITU-T Recommendation J.144: “Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference“. i.16 ETSI TS 183 063: “Telecommunications and In
29、ternet converged Services and Protocols for Advanced Networking (TISPAN); IMS-based IPTV stage 3 specification“. i.17 IETF RFC 2330: “Framework for IP Performance Metrics“. i.18 ITU-T Recommendation J.246: “Perceptual visual quality measurement techniques for multimedia services over digital cable t
30、elevision networks in the presence of a reduced bandwidth reference“. i.19 ITU-T Recommendation J.247: “Objective perceptual multimedia video quality measurement in the presence of a full reference“. i.20 ITU-T Recommendation J.341: “Objective perceptual multimedia video quality measurement of HDTV
31、for digital cable television in the presence of a full reference“. i.21 ITU-T Recommendation G.1081: “Performance monitoring points for IPTV“. i.22 ITU-R Recommendation BT.1359-1: “Relative timing of sound and vision for broadcasting“. i.23 NTIA Technical Memorandum TM-10-472 (September 2010): “Rela
32、ting Audio and Video quality using CIF Video“. NOTE: Available at http:/www.its.bldrdoc.gov/pub/ntia-rpt/10-472/. i.24 ITU-R Recommendation BS.1387: “Method for Objective Measurements of Perceived Audio Quality“. i.25 ISO/IEC 13818: “Information technology - Generic coding of moving pictures and ass
33、ociated audio information (MPEG 2, 9 parts)“. i.26 ITU-T Recommendation H.264: “Advanced video coding for generic audiovisual services“. i.27 SMPTE 421M: “Television - VC-1 Compressed Video Bitstream Format and Decoding Process, 2006“. i.28 ISO/IEC 14496: “Information technology - Coding of audio-vi
34、sual objects“ (MPEG 4; currently in 11 parts). i.29 ETSI ES 202 667: “Speech and multimedia Transmission Quality (STQ); Audiovisual QoS for communication over IP networks“. i.30 ETSI TR 102 720: “Speech and multimedia Transmission Quality (STQ); Delay variation on unshared access lines“. i.31 IEEE 8
35、02.3: “Local Area Networks“. i.32 IEEE 802.1Q: “Virtual Bridged Local Area Networks“. ETSI ETSI TR 102 714 V1.1.1 (2011-08) 7i.33 IEEE 802.1p: “Traffic Class Expediting and Dynamic Multicast Filtering“. i.34 IETF RFC 3471: “Generalized Multi-Protocol Label Switching (GMPLS) Signalling Functional Des
36、cription“. i.35 IETF RFC 2474: “Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers“. i.36 IETF RFC 2460: “Internet Protocol, Version 6 (IPv6) Specification“. i.37 IETF RFC 3595: “Textual Conventions for IPv6 Flow Label“. i.38 IETF RFC 3697: “IPv6 Flow Label Speci
37、fication“. i.39 IETF RFC 792: “Internet Control Message Protocol“. i.40 IETF RFC 4443: “Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification“. i.41 IETF RFC 4969: “Stream Control Transmission Protocol“. i.42 ITU-T Recommendation G.107: “The E-model: a c
38、omputational model for use in transmission planning“. i.43 IETF RFC 4585: “Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)“. i.44 IETF RFC 4588: “RTP Retransmission Payload Format“. i.45 ITU-T Recommendation G.722: “7 kHz audio-coding within 64 kbit/s“.
39、 i.46 ITU-T Recommendation G.729: “Coding of speech at 8 kbit/s using conjugate-structure algebraic-code-excited linear prediction (CS-ACELP)“. 3 Abbreviations For the purposes of the present document, the following abbreviations apply: ACR Absolute Category Rating AQ Audio Quality AVQ Audio-Visual
40、Quality CIF Common Intermediate Format CQ Conversation Quality DF Delay FactorDPI Deep Packet Inspection DTT Digital Terrestrial Television FEC Forward Error Correction FR Full Reference HDTV High Definition TeleVision IP Internet Protocol IPTV IP TeleVisionLQ Listening Quality MDI Media Delivery In
41、dex MLR Media Loss Rate MOS Mean Opinion Scores MOSe estimated MOS MOSo objective MOS MOSp predicted MOS MOSs subjective MOS MOVs Model Output Variables MSE Mean Square Error NR No Reference P2P Peer-to-PeerETSI ETSI TR 102 714 V1.1.1 (2011-08) 8PEAQ Perceptual Evaluation of Audio Quality PSNR Peak
42、Signal to Noise Ratio QCIF Quarter Common Intermediate Format QOE Quality of Experience QoS Quality of Service RR Reduced ReferenceSCTP Stream Control Transmission Protocol SDTV Simple Definition TeleVision STB Set Top Box TCP Transmission Control Protocol UDP User Datagram Protocol VGA Video Graphi
43、cs Array VLAN Virtual Local Area Network VoD Video on Demand VQEG Video Quality Experts Group WLAN Wireless Local Area Network xDSL x Digital Subscriber Line XOR eXclusive OR (logical functionality) 4 Measurement methods and tools 4.1 Subjective vs. objective assessments The measurement of end-to-en
44、d quality of multimedia services, as perceived by users, can be achieve through two types of measure: the instrumental measure (“objective“ methods) are based on the use of measurement tools that catch and analyze the signal in or at the output of the network or use network parameters, to predict a
45、quality score. The validation of these tools is based on the perceptive measure (“subjective“ methods). The general principle of the subjective methods consists in presenting audio and/or video sequences processed by the system under study, to testers who are asked to give their opinion on the quali
46、ty of the presented sequences. The differences between methodologies mainly lie on the way of presenting sequences (sequence is presented alone, with the reference i.e. non distorted signal, or with other sequences), on the type of scale (scales with five, seven, ten categories, or continuous) and o
47、n the labels describing the scale. The most used scale is the quality scale given in table 1. Table 1: Levels of end-to end perceived quality Quality Score Excellent 5 Good 4 Fair 3 Poor 2 Bad 1 This scale is used either as a simple category scale either combined to a continuous scale from 0 to 100,
48、 the five categories Excellent, Good, Fair, Poor, Bad then being used as anchor point. The average of the obtained scores gives a score generally called MOS for “Mean Opinion Score“. Most of the recommended subjective methodologies address one single modality (ITU-T Recommendation P.800 i.1 and P8xx
49、 series for speech quality, ITU-R Recommendations BT.500-10 i.2 and BT.1788 i.3 for video quality, ITU-R Recommendations BS.1284 i.5, BS.1534 i.6 and BS.1116-1 i.4 for audio quality). Some recommendations make suggestions for evaluating one modality in an audiovisual context or in the presence of an accompanying signal in the other modality (ITU-R Recommendation BS.775-2 i.7) for multichannel audio with accompanying picture, ITU-R Recommendation BS.1286 i.8 for the t
