ITU-T P 800 1-2016 Mean opinion score (MOS) terminology (Study Group 12)《平均意见得分术语(研究组12)》.pdf

1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T P.800.1 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (02/2016) SERIES P: TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Methods for objective and subjective assessment of speech quality Mean opinion score (MOS

2、) terminology Recommendation ITU-T P.800.1 ITU-T P-SERIES RECOMMENDATIONS TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Voice terminal characteristics Series P.30 P.300 Reference sys

3、tems Series P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of speech quality Series P.80 P.800 Audiovisual quality in multimedia services Se

4、ries P.900 Transmission performance and QoS aspects of IP end-points Series P.1000 Communications involving vehicles Series P.1100 Models and tools for quality assessment of streamed media Series P.1200 Telemeeting assessment Series P.1300 Statistical analysis, evaluation and reporting guidelines of

5、 quality measurements Series P.1400 Methods for objective and subjective assessment of quality of services other than voice services Series P.1500 For further details, please refer to the list of ITU-T Recommendations. Rec. ITU-T P.800.1 (02/2016) i Recommendation ITU-T P.800.1 Mean opinion score (M

6、OS) terminology Summary Recommendation ITU-T P.800.1 provides a terminology which shall be used in conjunction with audio, video and audiovisual quality expressions in terms of mean opinion score (MOS). This terminology is motivated by the intention to avoid misunderstanding as to whether specific v

7、alues of MOS for audio are related to listening quality, talking quality or conversational quality, whether they originate from subjective tests, from objective models or from network planning models, as well as extending the concept to video and audiovisual quality. Furthermore, this Recommendation

8、 provides identifiers regarding the audio bandwidth, the type of interface (electrical or acoustical) and the video resolution. Further guidance on the interpretation of MOS is given in Recommendation ITU-T P.800.2. History Edition Recommendation Approval Study Group Unique ID* 1.0 ITU-T P.800.1 200

9、3-03-16 12 11.1002/1000/6257 2.0 ITU-T P.800.1 2006-07-14 12 11.1002/1000/8860 3.0 ITU-T P.800.1 2016-02-29 12 11.1002/1000/12749 * To access the Recommendation, type the URL http:/handle.itu.int/ in the address field of your web browser, followed by the Recommendations unique ID. For example, http:

10、/handle.itu.int/11.1002/1000/11830-en. ii Rec. ITU-T P.800.1 (02/2016) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Se

11、ctor (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets eve

12、ry four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purvie

13、w, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is volunt

14、ary. However, the Recommendation may contain certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the

15、 negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws attention to the possibility that the practice or implementation of this Recommendation may involve t

16、he use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommend

17、ation, ITU had not received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at htt

18、p:/www.itu.int/ITU-T/ipr/. ITU 2016 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T P.800.1 (02/2016) iii Table of Contents Page 1 Scope . 1 2 References . 1 3 Definitions 3 4 Abbreviations and acron

19、yms 3 5 Conventions 3 6 Recommended MOS terminology for audio . 3 6.1 MOS related to listening-only situations 4 6.2 MOS related to conversational situations . 5 6.3 MOS related to talking situations . 5 6.4 Relationship between some audio MOS qualifiers 6 7 Recommended MOS terminology for Video 6 7

20、.1 MOS-VQS 7 7.2 MOS-VQO . 7 7.3 MOS-VQE 7 7.4 Relationship between some video MOS qualifiers 7 8 Recommended MOS terminology for audiovisual . 8 8.1 MOS-AVQS . 9 8.2 MOS-AVQO . 9 8.3 MOS-AVQE . 9 8.4 Relationship between some audiovisual MOS qualifiers . 9 Rec. ITU-T P.800.1 (02/2016) 1 Recommendat

21、ion ITU-T P.800.1 Mean opinion score (MOS) terminology 1 Scope This Recommendation provides a terminology which shall be used in conjunction with audio, video and audio-visual quality expressions in terms of mean opinion score (MOS). 2 References The following ITU-T Recommendations and other referen

22、ces contain provisions which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to invest

23、igate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the stat

24、us of a Recommendation. ITU-T G.107 Recommendation ITU-T G.107 (2015), The E-model, a computational model for use in transmission planning. ITU-T G.113 Ap.I Recommendation ITU-T G.113 (2007), Appendix I, Provisional planning values for the equipment impairment factor, Ie, and packet-loss robustness

25、factor, Bpl. ITU-T G.1070 Recommendation ITU-T G.1070 (2012), Opinion model for video-telephony applications ITU-T G.1071 Recommendation ITU-T G.1071 (2015), Opinion model for network planning of video and audio streaming applications. ITU-T J.140 Recommendation ITU-T J.140 (1998), Subjective pictur

26、e quality assessment for digital cable television systems. ITU-T J.144 Recommendation ITU-T J.144 (2004), Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. ITU-T J.247 Recommendation ITU-T J.247 (2008), Objective perceptual mu

27、ltimedia video quality measurement in the presence of a full reference ITU-T J.341 Recommendation ITU-T J.341 (2016), Objective perceptual multimedia video quality measurement of HDTV for digital cable television in the presence of a full reference ITU-T P.10 Recommendation ITU-T P.10/G.100 (2006),

28、Vocabulary for performance and quality of service. ITU-T P.562 Recommendation ITU-T P.562 (2004), Analysis and interpretation of INMD voice-service measurements. ITU-T P.563 Recommendation ITU-T P.563 (2004), Single-ended method for objective speech quality assessment in narrow-band telephony applic

29、ations. ITU-T P.800 Recommendation ITU-T P.800 (1996), Methods for subjective determination of transmission quality. ITU-T P.800.2 Recommendation ITU-T P.800.2 (2013), Mean opinion score interpretation and reporting 2 Rec. ITU-T P.800.1 (02/2016) ITU-T P.830 Recommendation ITU-T P.830 (1996), Subjec

30、tive performance assessment of telephone-band and wideband digital codecs. ITU-T P.831 Recommendation ITU-T P.831 (1998), Subjective performance evaluation of network cancellers. ITU-T P.832 Recommendation ITU-T P.832 (2000), Subjective performance evaluation of hands-free terminals. ITU-T P.833 Rec

31、ommendation ITU-T P.833 (2001), Methodology for the derivation of equipment impairment factors from subjective listening-only tests. ITU-T P.834 Recommendation ITU-T P.834 (2015), Methodology for the derivation of equipment impairment factors from instrumental models. ITU-T P.835 Recommendation ITU-

32、T P.835 (2003), Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm. ITU-T P.840 Recommendation ITU-T P.840 (2003), Subjective listening test method for evaluating circuit multiplication equipment. ITU-T P.862 Recommendation ITU-T P.862 (2

33、001), Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs. ITU T P.862.1 Recommendation ITU-T P.862.1 (2003), Mapping function for transforming P.862 raw result scores to MOS-LQO. ITU-T P.862

34、.2 Recommendation ITU-T P.862.2 (2007), Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs. ITU-T P.863 Recommendation ITU-T P.863 (2014), Perceptual objective listening quality assessment ITU-T P.910 Recommendation ITU-T P.910 (2008), Subj

35、ective video quality assessment methods for multimedia applications. ITU-T P.911 Recommendation ITU-T P.911 (1998), Subjective audiovisual quality assessment methods for multimedia applications. ITU-T P.912 Recommendation ITU-T P.912 (2016), Subjective video quality assessment methods for recognitio

36、n tasks. ITU-T P.913 Recommendation ITU-T P.913 (2016), Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment. ITU-T P.920 Recommendation ITU-T P.920 (2000), Interactive test methods for

37、audiovisual communications. ITU-T P.1201 Recommendation ITU-T P.1201 (2012), Parametric non-intrusive assessment of audiovisual media streaming quality. ITU-T P.1202 Recommendation ITU-T P.1202 (2012), Parametric non-intrusive bitstream assessment of video media streaming quality. ITU-T P.1301 Recom

38、mendation ITU-T P.1301 (2012), Subjective quality evaluation of audio and audiovisual multiparty telemeetings. ITU-R BT.500 Recommendation ITU-R BT.500 (2012), Methodology for the subjective assessment of the quality of television pictures. Rec. ITU-T P.800.1 (02/2016) 3 ITU-R BT.709-6 Recommendatio

39、n ITU-R BT.709-6 (2015), Parameter values for the HDTV standards for production and international programme exchange. ITU-R BT.2020-2 Recommendation ITU-R BT.2020 (2015), Parameter values for ultra-high definition television systems for production and international programme exchange. 3 Definitions

40、None. 4 Abbreviations and acronyms This Recommendation uses the following abbreviations and acronyms: ACR Absolute Category Rating HD High Definition IRS Intermediate Reference System MOS Mean Opinion Score SD Standard Definition UHD Ultra High Definition 5 Conventions In this Recommendation the fol

41、lowing conventions are used: MOS-AVQE is used to refer to MOS estimated audiovisual quality; MOS-AVQO is used to refer to MOS objective audiovisual quality; MOS-AVQS is used to refer to MOS subjective audiovisual quality; MOS-CQE is used to refer to MOS estimated conversational quality; MOS-CQO is u

42、sed to refer to MOS objective conversational quality; MOS-CQS is used to refer to MOS subjective conversational quality; MOS-LQE is used to refer to MOS estimated listening quality; MOS-LQO is used to refer to MOS objective listening quality; MOS-LQS is used to refer to MOS subjective listening qual

43、ity; MOS-TQE is used to refer to MOS estimated talking quality; MOS-TQO is used to refer to MOS objective talking quality; MOS-TQS is used to refer to MOS subjective talking quality; MOS-VQE is used to refer to MOS estimated video quality; MOS-VQO is used to refer to MOS objective video quality; MOS

44、-VQS is used to refer to MOS subjective video quality. 6 Recommended MOS terminology for audio The mean opinion score (MOS) is defined in ITU-T P.10 in the following way: 4 Rec. ITU-T P.800.1 (02/2016) The mean of opinion scores, i.e., of the values on a predefined scale that subjects assign to thei

45、r opinion of the performance of the telephone transmission system used either for conversation or for listening to spoken material. Apart from subjective opinion, the abbreviation MOS is also used for scores that originate from objective models or network planning models. The following identifiers,

46、listed in Table 1, are recommended to be used together with the abbreviation MOS in order to distinguish the area of application, where N refers to narrow-band, W refers to wideband, LQ refers to listening quality, CQ refers to conversational quality, TQ refers to talking quality, S refers to subjec

47、tive, O refers to objective, and E refers to estimated. For the purposes of this Recommendation, the term Audio for this clause primarily relates to speech quality. Table 1 Audio identifiers Listening-only Conversational Talking Subjective MOS-LQSy MOS-CQSy MOS-TQSy Objective MOS-LQOy MOS-CQOy MOS-T

48、QOy Estimated MOS-LQEy MOS-CQEy MOS-TQEy NOTE The letter “y“ at the end of above acronyms is a placeholder for the descriptor of the respective audio bandwidth, see the following provisional instructions. N for MOS scores obtained for narrow-band (300-3400 Hz) speech relative to a narrow band high q

49、uality reference. This is applicable for instance to narrow-band only subjective tests or to ITU-T P.862.1 or ITU-T P.863 (narrow-band operational mode) scores. W for MOS scores obtained for wideband (50-7000 Hz) speech relative to a wideband high quality reference. This is applicable for instance to wideband only subjective tests or to ITU-T P.862.2 scores. M for MOS scores obtained for narrow-band or wideband speech relative to a wideband high quality reference in a mixed ban