1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T P.863 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (09/2014) SERIES P: TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Methods for objective and subjective assessment of speech quality Perceptual objective list
2、ening quality assessment Recommendation ITU-T P.863 ITU-T P-SERIES RECOMMENDATIONS TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Voice terminal characteristics Series P.30 P.300 Refe
3、rence systems Series P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of speech quality Series P.80 P.800 Audiovisual quality in multimedia se
4、rvices Series P.900 Transmission performance and QoS aspects of IP end-points Series P.1000 Communications involving vehicles Series P.1100 Models and tools for quality assessment of streamed media Series P.1200 Telemeeting assessment Series P.1300 Statistical analysis, evaluation and reporting guid
5、elines of quality measurements Series P.1400 Methods for objective and subjective assessment of quality of services other than voice services Series P.1500 For further details, please refer to the list of ITU-T Recommendations. Rec. ITU-T P.863 (09/2014) i Recommendation ITU-T P.863 Perceptual objec
6、tive listening quality assessment Summary Recommendation ITU-T P.863 describes an objective method for predicting overall listening speech quality from narrowband (NB) (300 to 3 400 Hz) to super-wideband (SWB) (50 to 14 000 Hz) telecommunication scenarios as perceived by the user in an ITU-T P.800 o
7、r ITU-T P.830 absolute category rating (ACR) listening-only test. Recommendation ITU-T P.863 supports two operational modes, one for narrowband and one for super-wideband. This Recommendation presents a high-level description of the method, advice on how to use it, and some results from a benchmark
8、carried out in the period 2006-2010. All essential parts of the model are described in detail, and are provided in separate pdf-files (see Annex B). These files form an integral part of this Recommendation and shall take precedence in case of conflicts between the high-level descriptions included in
9、 the main body of this Recommendation and the corresponding detailed description parts. A conformance testing procedure is also specified in Annex A to allow a user to validate that an alternative implementation of the model is correct. This Recommendation includes an electronic attachment containin
10、g detailed descriptions in pdf format (see Annex B) and conformance testing data (see Annex A). The 2014 revision of ITU-T P.863 introduces bug fixes and resolves reported issues from ITU-T P.863 field deployments. On average the scores produced by this revised version of P.863 are very close to the
11、 values obtained from the previous version (V1.1). Due to the improvements and bug fixes in the revised version, there may however be significant differences for some individual measurements, especially for cases where the previous version failed. It should be observed that there is also a 2014 revi
12、sion of the companion Recommendation P.863.1. History Edition Recommendation Approval Study Group Unique ID* 1.0 ITU-T P.863 2011-01-13 12 11.1002/1000/11009 1.1 ITU-T P.863 (2011) Amd. 1 2011-11-09 12 11.1002/1000/11463 2.0 ITU-T P.863 2014-09-11 12 11.1002/1000/12174 Keywords Listening quality, ob
13、jective quality, perceptual model, voice quality. _ * To access the Recommendation, type the URL http:/handle.itu.int/ in the address field of your web browser, followed by the Recommendations unique ID. For example, http:/handle.itu.int/11.1002/1000/11830-en. ii Rec. ITU-T P.863 (09/2014) FOREWORD
14、The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying tech
15、nical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which,
16、 in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IE
17、C. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to en
18、sure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words do
19、es not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concernin
20、g the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, w
21、hich may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2015 All rights reserved. No part of this publication
22、 may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T P.863 (09/2014) iii Table of Contents Page 1 Scope . 1 2 References . 5 3 Definitions 6 3.1 Terms defined elsewhere 6 4 Abbreviations and acronyms 6 5 Conventions 7 6 Overview of the ITU-T P.863 algo
23、rithm 7 7 Comparison between objective and subjective scores 9 8 Speech material . 10 8.1 Recommendations on source speech material 10 8.2 Insertion of source speech material into the system under test 12 8.3 Recommendations on processed and degraded speech material 12 8.4 Special requirements for a
24、coustical captured speech material . 13 8.5 Acoustical insertion/capture for loudspeaker phones . 13 8.6 Technical requirements on signals to be processed by ITU-T P.863 . 14 8.7 Predicted scores by the model 14 9 Description of the ITU-T P.863 algorithm . 14 9.1 Overview 14 9.2 Temporal alignment 1
25、5 9.3 Joining sections with constant delay 29 9.4 Sample rate ratio detection . 29 9.5 Resampling . 30 9.6 Level, frequency response and time alignment pre-processing 30 9.7 Perceptual model 31 10 Conclusions. 43 Annex A Conformance data and conformance tests . 45 A.1 List of files provided for conf
26、ormance validation 45 A.2 Conformance tests 45 A.3 Conversion of sampling rates . 47 A.4 Digital attachments . 48 Annex B Detailed Descriptions of the ITU-T P.863 algorithm in pdf-format 49 Appendix I Reporting of the performance results for the ITU-T P.863 algorithm based on the rmse* metric 50 I.1
27、 Purpose of this appendix 50 I.2 Overview 50 I.3 Performance results for the ITU-T P.863 algorithm 51 I.4 Calculation of rmse* . 55 iv Rec. ITU-T P.863 (09/2014) Page I.5 Scatter plots 58 Appendix II Description of the “full-scale“ subjective tests in a super-wideband context conducted for the ITU-T
28、 P.863 algorithm training and validation . 62 II.1 Database structure and subjects requirement . 62 II.2 Anchor conditions 62 II.3 Design rules of test conditions for full-scale mandatory tests 63 II.4 Reference and degraded speech material . 63 II.5 Transmission and capturing capture of speech mate
29、rial superimposed interlaced with background noises . 64 II.6 Transmission and capturing capture of speech material under time warping conditions . 65 II.7 Subjective test set up for assessing super-wideband speech quality 65 II.8 Limitations in subjective test results 65 Appendix III Prediction of
30、acoustically recorded narrowband speech . 67 III.1 Background . 67 III.2 Requirements for acoustically recorded speech data to be assessed by ITU-T P.863 . 67 III.3 Pre-processing of speech and use of ITU-T P.863 . 67 III.4 Interpretation of results . 68 III.5 Example results 68 Bibliography. 71 Ele
31、ctronic attachment: Detailed descriptions in pdf-format and conformance testing data. Rec. ITU-T P.863 (09/2014) v Introduction Recommendation ITU-T P.863 defines a single algorithm for assessing the speech quality of current and near future telephony systems that utilize a broad variety of coding,
32、transport and enhancement technologies. The measurement algorithm is a full reference model which operates by performing a comparison between a known reference signal and a captured degraded signal. This is consistent with the algorithms described in Recommendations ITU-T P.861 and ITU-T P.862. Reco
33、mmendation ITU-T P.861, published in 1996, was primarily focused on identifying the quality impact of codecs. Subsequent to its release, work on a successor was started to create an algorithm suitable for assessing the additional impact of network impairments. The work resulted in the publishing of
34、Recommendation ITU-T P.862 in 2001. Recommendation ITU-T P.863 (which during its development was known as P.OLQA) incorporates current industry requirements and in particular allows the assessment of super-wideband speech as well as networks and codecs that introduce time warping. Rec. ITU-T P.863 (
35、09/2014) 1 Recommendation ITU-T P.863 Perceptual objective listening quality assessment 1 Scope This Recommendation1 defines a single algorithm for assessing the speech quality of current and near future telephony systems that utilize a broad variety of coding, transport and speech enhancement techn
36、ologies. Based on the benchmark results presented within the studies of ITU-T, an overview of the test factors, coding technologies and applications to which this Recommendation applies is given in Tables 1 to 4. Table 1 presents factors and applications included in the requirement specification and
37、 which were used in the selection phase of the ITU-T P.863 algorithm. It should be noted that the performance of the ITU-T P.863 algorithm under each individual condition in Table 1 is not reflected in this table. Additional and detailed analysis will be undertaken in the characterization phase of t
38、he ITU-T P.863 algorithm. Table 2 presents a list of conditions for which this Recommendation is not intended to be used. Table 3 presents test variables for which further investigation is needed, or for which ITU-T P.863 is subject to claims of providing inaccurate predictions when used in conjunct
39、ion with these. Finally, Table 4 lists factors, technologies and applications for which the ITU-T P.863 algorithm has not currently been validated. Note that the ITU-T P.863 algorithm cannot be used to replace subjective testing. It should also be noted that the ITU-T P.863 algorithm does not provid
40、e a comprehensive evaluation of transmission quality. It only measures the effects of one-way speech distortion and noise on speech quality. The effects of delay, sidetone, echo, and other impairments related to two-way interaction (e.g., centre clipper) are not reflected in the ITU-T P.863 scores.
41、Therefore, it is possible to have high ITU-T P.863 scores, yet poor overall conversational quality. A characterization phase will follow the approval of this Recommendation. The purpose of the characterization phase is to prove the applicability of the ITU-T P.863 algorithm in real applications and
42、may include new test conditions, new test scenarios and alternate test methodologies. Table 1 Factors and applications included in the requirement specification and used in the selection phase of the ITU-T P.863 algorithm Test factors Speech input levels to a codec Transmission channel errors Packet
43、 loss and packet loss concealment Bit rates if a codec has more than one bit-rate mode Transcodings Acoustic noise in sending environment Effect of varying delay in listening-only tests Short-term time warping of audio signal Long-term time warping of audio signal Listening levels between 53 and 78
44、dB(A) SPL in super-wideband mode _ 1 This Recommendation includes an electronic attachment containing detailed descriptions in pdf format (see Annex B) and conformance testing data (see Annex A). 2 Rec. ITU-T P.863 (09/2014) Table 1 Factors and applications included in the requirement specification
45、and used in the selection phase of the ITU-T P.863 algorithm Test factors Packet loss and packet loss concealment with PCM type codecs Temporal and amplitude clipping of speech Linear distortions, including bandwidth limitations and spectral shaping (non-flat frequency responses) Frequency response
46、Coding technologies ITU-T G.711, ITU-T G.711 PLC, ITU-T G.711.1 ITU-T G.718, ITU-T G.719, ITU-T G.722, ITU-T G.722.1, ITU-T G.723.1, ITU-T G.726, ITU-T G.728, ITU-T G.729 GSM-FR, GSM-HR, GSM EFR AMR-NB, AMR-WB (ITU-T G.722.2), AMR-WB+ PDC-FR, PDC-HR EVRC (ANSI/TIA-127-A), EVRC-B (TIA-718-B) Skype (S
47、ILK V3, iLBC, iSAC and ITU-T G.729) Speex, QCELP (TIA-EIA-IS-733), iLBC, CVSD (64 kbit/s, “Bluetooth“) MP3, AAC, AAC-LD Applications Codec evaluation Terminal testing, influence of the acoustical path and the transducer in sending and receiving direction. (NOTE Acoustical path in receiving direction
48、 only for super-wideband mode.) Bandwidth extensions Live network testing using digital or analogue connection to the network Testing of emulated and prototype networks UMTS, CDMA, GSM, TETRA, WB-DECT ,VoIP, POTS, PSTN, Video Telephony, Bluetooth Voice Activity Detection (VAD), Automatic Gain Contro
49、l (AGC) Voice Enhancement Devices (VED), Noise Reduction (NR) Discontinuous Transmission (DTX), Comfort Noise Insertion NOTE Individual conditions will be analysed during the characterization phase of ITU-T P.863 and details will be made available. Rec. ITU-T P.863 (09/2014) 3 Table 2 ITU-T P.863 is not intended to be used with these variables Test factors Effect of delay in conversational tests Talker echo Sidetone Acoustic noise in receiving environment Coding technologies Applications Non-intrus
