1、 International Telecommunication Union ITU-T P.564TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (11/2007) SERIES P: TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Objective measuring apparatus Conformance testing for voice over IP transmission quality assessment model
2、s ITU-T Recommendation P.564 ITU-T P-SERIES RECOMMENDATIONS TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Subscribers lines and sets Series P.30 P.300 Transmission
3、 standards Series P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of quality Series P.80 P.800Audiovisual quality in multimedia services Seri
4、es P.900 Transmission performance and QoS aspects of IP end-points Series P.1000 For further details, please refer to the list of ITU-T Recommendations. ITU-T Rec. P.564 (11/2007) i ITU-T Recommendation P.564 Conformance testing for voice over IP transmission quality assessment models Summary ITU-T
5、Recommendation P.564 specifies the minimum criteria for objective speech quality assessment models that predict the impact of observed IP network impairments on the one-way listening quality experienced by the end-user in IP/UDP/RTP-based telephony applications (3.1-kHz narrow-band in the main body,
6、 7 kHz wideband in Annex B). Models compliant with this Recommendation predict mean opinion scores (MOS) on the ACR listening quality scale. It is expected that the primary applications for such models are monitoring of transmission quality for operations and maintenance purposes, and measurements i
7、n support of service level agreements (SLAs) between service providers and their customers. P.564-conformant models may be deployed both in endpoint locations and at mid-network monitoring points. This Recommendation includes an electronic attachment containing the test vectors. Source ITU-T Recomme
8、ndation P.564 was approved on 13 November 2007 by ITU-T Study Group 12 (2005-2008) under the ITU-T Recommendation A.8 procedure. Keywords ACR, conformance, model, MOS, P.862, VoIP. ii ITU-T Rec. P.564 (11/2007) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialize
9、d agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a
10、 view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recomme
11、ndations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conc
12、iseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure e.g. interoperability or applicability) and compliance with the Recommendat
13、ion is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. I
14、NTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights
15、, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are
16、cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2008 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission
17、 of ITU. ITU-T Rec. P.564 (11/2007) iii CONTENTS Page 1 Scope 1 2 References. 2 3 Definitions 3 4 Abbreviations and acronyms 3 5 Conventions 4 6 Application scenarios 4 6.1 General comments 4 6.2 Network problem diagnosis 4 6.3 IP Centrex. 4 6.4 Managed IP service SLA 5 6.5 Call quality monitoring f
18、or reporting . 5 6.6 Active test calls. 5 7 Modes of operation. 5 7.1 Overview 5 7.2 Dynamic operation . 6 7.3 Static operation. 6 7.4 Embedded operation. 6 7.5 Reference endpoints . 7 8 Inputs 7 9 Outputs 8 9.1 Speech quality (mandatory) 8 9.2 Interpretation of quality predictions . 8 9.3 Diagnosti
19、cs (optional) 9 9.4 Compliance statement (mandatory) 10 10 Conformance test methodology 11 10.1 Overview 11 10.2 Test vector generation 14 10.3 Processing the test vectors 17 10.4 Performance analysis 17 11 Conformance accuracy requirements . 19 Annex A File formats and naming conventions 21 A.1 Gen
20、eral . 21 A.2 Test vector files 21 A.3 Test vector quality score files. 21 Annex B Extension of P.564 to wideband voice over IP quality assessment models 22 Appendix I Example statement of compliance . 23 Electronic attachment: Test vectors for narrow-band and wideband applications ITU-T Rec. P.564
21、(11/2007) 1 ITU-T Recommendation P.564 Conformance testing for voice over IP transmission quality assessment models 1 Scope This Recommendation specifies minimum criteria for objective speech quality assessment models that predict the impact of observed IP network impairments on the one-way listenin
22、g quality experienced by the end-user in IP/UDP/RTP-based 3.1-kHz narrow-band telephony applications. An extension to 7 kHz wideband telephony is also provided in Annex B. It is expected that the primary applications for such models are monitoring of transmission quality for operations and maintenan
23、ce purposes, and measurements in support of service level agreements (SLAs) between service providers and their customers. Models compliant with this Recommendation predict mean opinion scores (MOS) on the ACR listening quality scale. Their performance is based and estimated on the MOS scale (as def
24、ined in ITU-T P.8001). The primary quality prediction made by such a model is not based on the payload of the RTP stream being analysed, but assumes a typical, or generic, voice payload. Some additional diagnostic outputs may be based on the payload, if available. A model compliant with this Recomme
25、ndation should always take the voice codec into account. If any input parameter is unavailable and an assumed value is used, this fact should be reported. A model compliant with this Recommendation cannot provide a comprehensive end-to-end evaluation of transmission quality because its scores can on
26、ly reflect the impairments on the IP network being measured that may only be part of the end-to-end connection. The effects of speech level, acoustic background noise, delay, sidetone, echo and other impairments related to the payload are not reflected in the scores computed by such a model. Therefo
27、re, it is possible to have high scores with a model compliant with this Recommendation, yet have a poor quality of the connection overall. The accuracy criteria herein were derived with the intent to avoid the frequent occurrence of “false positive“ or “false negative“ errors. This is an especially
28、important consideration when a model compliant with this Recommendation is used, for example, in assessing compliance with SLAs. The criteria for the model described in this Recommendation are applicable to devices that may reside anywhere within the packet transport network, including edge devices.
29、 As such, each will be able to use only information present at the location deployed. However, this does include information that can be extracted from RTCP-SR, RR and XR. The accuracy criteria are based on a comparison of a models performance with the P.862 perceptual evaluation of speech quality (
30、PESQ) algorithm using the output mapping defined in ITU-T P.862.1 for 3.1-kHz narrow-band telephony, hereafter referred to as P.862.1. Hence, compliance to this Recommendation shall only be claimed for factors, technologies or applications that are within the scope of P.862, or for which the operati
31、on of P.862 has been verified against subjective test data. The speech test material described in clause 10.2.1 is provided in an electronic attachment and forms an integral and normative part of this Recommendation. _ 1These scores can be translated into Ie-eff values using the formulae given in An
32、nex B of ITU-T G.107 (equation B-4) and in Appendix I of ITU-T G.107 (equations I-1, I-2 and I-3). 2 ITU-T Rec. P.564 (11/2007) 2 References The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recommendation
33、. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references list
34、ed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. ITU-T G.107 ITU-T Recommendation G.107 (2005), The E-model, a computational model
35、for use in transmission planning. ITU-T G.191 ITU-T Recommendation G.191 (2005), Software tools for speech and audio coding standardization. ITU-T G.711 ITU-T Recommendation G.711 (1988), Pulse code modulation (PCM) of voice frequencies. ITU-T G.729 ITU-T Recommendation G.729 (2007), Coding of speec
36、h at 8 kbit/s using conjugate-structure algebraic-code-excited linear prediction (CS-ACELP). ITU-T G.799.1 ITU-T Recommendation G.799.1/Y.1451.1 (2004), Functionality and interface specifications for GSTN transport network equipment for interconnecting GSTN and IP networks. ITU-T G.1020 ITU-T Recomm
37、endation G.1020 (2006), Performance parameter definitions for quality of speech and other voiceband applications utilizing IP networks. ITU-T G.1050 ITU-T Recommendation G.1050 (2007), Network model for evaluating multimedia transmission performance over Internet Protocol. ITU-T H.248.30 ITU-T Recom
38、mendation H.248.30 (2007), Gateway control protocol: RTCP extended performance metrics packages. ITU-T H.460.9 ITU-T Recommendation H.460.9 (2002), Support for online QoS-monitoring reporting within H.323 systems. ITU-T P.56 ITU-T Recommendation P.56 (1993), Objective measurement of active speech le
39、vel. ITU-T P.501 ITU-T Recommendation P.501 (2007), Test signals for use in telephonometry. ITU-T P.800 ITU-T Recommendation P.800 (1996), Methods for subjective determination of transmission quality. ITU-T P.800.1 ITU-T Recommendation P.800.1 (2006), Mean Opinion Score (MOS) terminology. ITU-T P.86
40、2 ITU-T Recommendation P.862 (2001), Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs. ITU-T P.862.1 ITU-T Recommendation P.862.1 (2003), Mapping function for transforming P.862 raw result
41、 scores to MOS-LQO. ITU-T P.862.2 ITU-T Recommendation P.862.2 (2007), Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs. ITU-T Rec. P.564 (11/2007) 3 ITU-T Y.1541 ITU-T Recommendation Y.1541 (2006), Network performance objectives for IP-b
42、ased services. IETF RFC 768 IETF RFC 768 (1980), User Datagram Protocol. IETF RFC 791 IETF RFC 791 (1981), Internet Protocol. IETF RFC 3550 IETF RFC 3550 (2003), RTP: A Transport Protocol for Real-Time Applications. IETF RFC 3555 IETF RFC 3555 (2003), MIME Type Registration of RTP Payload Formats. I
43、ETF RFC 3611 IETF RFC 3611 (2003), RTP Control Protocol Extended Reports (RTCP XR). 3 Definitions This Recommendation defines the following term: 3.1 speech sample: A series of one or more speech utterances. 4 Abbreviations and acronyms This Recommendation uses the following abbreviations and acrony
44、ms: ACR Absolute Category Rating IP Internet Protocol LAN Local Area Network MOS Mean Opinion Score MOS-LQONMean Opinion Score, Listening Quality Objective measurement in Narrow-band audio context OOS Out Of Sequence PBX Private Branch eXchange PCAP Packet CAPture PDV Packet Delay Variation PESQ Per
45、ceptual Evaluation of Speech Quality PLC Packet Loss Concealment RFC Request For Comments ROI Region Of Interest RR Receiver Report RTCP Real-time Transport Control Protocol RTP Real-time Transport Protocol SLA Service Level Agreement SR Sender Report UDP User Datagram Protocol VAF Voice Activity Fa
46、ctor VoIP Voice over Internet Protocol XR eXtended Report 4 ITU-T Rec. P.564 (11/2007) 5 Conventions The conventions used in mathematical equations in this Recommendation are defined at the appropriate points in the text. 6 Application scenarios 6.1 General comments When using a P.564 device as a mi
47、dpoint monitor, it is assumed that it will have visibility of all packets in the data stream of interest. Note, however, that due to the nature of IP networks, each packet is routed individually with no guarantee of any path consistency. In situations where not all packets are visible, scores may no
48、t represent the actual user experience. 6.2 Network problem diagnosis By continuously monitoring network performance, midpoint probes, based on a model compliant with this Recommendation, can continuously gather and report information on service quality. This information can then be used to classify
49、 and isolate network problems. 6.2.1 Temporal correlation It may be possible to correlate service quality events with other network problems that caused the change in quality. In this application, it is useful to know the approximate time interval during which service quality was degraded (e.g., start time and duration). 6.2.2 Impairment correlation It may also be possible to identify events occurring on multiple call streams that are similar in nature, to help identify common causes of problems within a network
