1、 ETSI TR 102 716-2 V1.1.2 (2010-11)Technical Report Speech and multimedia Transmission Quality (STQ);Guidelines, objectives and results of speech quality analysisin the context of interworking Plugtestsfor multiplay services; Part 2: ResultsETSI ETSI TR 102 716-2 V1.1.2 (2010-11)2Reference RTR/STQ-0
2、0178-2 Keywords analysis, interoperability, quality, voice ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Impor
3、tant notice Individual copies of the present document can be downloaded from: http:/www.etsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the
4、 Portable Document Format (PDF). In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Infor
5、mation on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No pa
6、rt may be reproduced except as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2010. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTM, TIPHONTM, the TIPHON logo and the ETSI logo are Tr
7、ade Marks of ETSI registered for the benefit of its Members. 3GPPTM is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. LTE is a Trade Mark of ETSI currently being registered for the benefit of its Members and of the 3GPP Organizational Partners
8、. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)3Contents Intellectual Property Rights 5g3Foreword . 5g31 Scope 6g32 References 6g32.1 Normative references . 6g32.2 Informative references 6g33 Definitions and abbreviations .
9、7g33.1 Definitions 7g33.2 Abbreviations . 7g34 Context . 7g35 Platform presentation . 8g36 Presentation of test conditions 8g36.1 Indicator description . 8g36.1.1 Post Dialling Delay . 8g36.1.2 Listening speech quality . 9g36.1.3 Listening speech quality stability 9g36.1.4 End to end delay . 9g36.1.
10、5 End to end delay variation 10g36.1.6 Level of active speech signal at reception 10g36.1.7 Noise level at reception . 10g36.1.8 DTMF integrity . 11g36.2 Use of platforms . 11g36.3 Pie diagram presentation 11g3Annex A: Performance of VoIP for IP to PSTN connection condition. Overview of results obta
11、ined in October 2009 . 13g3A.1 Platforms presentation 13g3A.2 Description of the methodology . 13g3A.3 Results presentation 14g3A.4 Conclusion 17g3Annex B: Performance of of VoIP for IP to PSTN connection condition. Overview of results obtained over one year 18g3B.1 Platforms and methodology present
12、ation . 18g3B.2 Post Dialling Delay 18g3B.3 Listening speech quality . 20g3B.4 Listening speech quality stability . 21g3B.5 End to end delay . 23g3B.6 End to end delay variation 24g3B.7 Noise level at reception 26g3B.8 Conclusion 27g3Annex C: Performance of VoIP offers in interconnection conditions
13、. 28g3C.1 Context . 28g3ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)4C.2 Platforms presentation 29g3C.3 Description of the methodology . 29g3C.4 Overview of results obtained in January 2010 . 31g3C.5 Post Dialling Delay 31g3C.6 Listening speech quality . 33g3C.7 End to end delay . 35g3C.8 Conclusion 37g3
14、History 38g3ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)5Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can
15、 be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursua
16、nt to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present docum
17、ent. Foreword This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality (STQ). The present document is part 2 of a multi-part deliverable covering Guidelines, objectives and results of speech quality analysis in the context of interworking Pl
18、ugtests for multiplay services, as identified below: Part 1: “Guidelines and objectives“; Part 2: “Results“. ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)61 Scope The present document shows the results obtained on technological watch platforms on Triple Play offerings. The determinate indicators and the u
19、sed measurement methods are presented in part 1 of this multi-part deliverable i.6. The results shown come from a survey of various service performance, and show the applicability of the method provided in the part 1 of this multi-part deliverable i.6 intended for Plugtests. The main part of the pre
20、sent document defines the generic technological platform, the test conditions and provides general information about the test campaigns. The different annexes detail each measurement campaign and the results obtained during these campaigns. 2 References References are either specific (identified by
21、date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the reference document (including any amendments) applies. Referenced documents which are not found to be publicly
22、 available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity. 2.1 Normative references The following referenced documents are necessary fo
23、r the application of the present document. Not applicable. 2.2 Informative references The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 ETSI EG 202 765-2: “Speech Processing, Transmi
24、ssion and Quality Aspects (STQ); QoS and network performance metrics and measurement methods; Part 2: Transmission Quality Indicator combining Voice Quality Metrics“. i.2 ITU-T Recommendation P.862: “Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality as
25、sessment of narrow-band telephone networks and speech codecs“. i.3 ITU-T Recommendation P.862.1: “Mapping function for transforming P.862 raw result scores to MOS-LQO“. i.4 ITU-T Recommendation P.56: “Objective measurement of active speech level“. i.5 ITU-T Recommendation P.505: “One-view visualizat
26、ion of speech quality measurement results“. i.6 ETSI TR 102 716-1: “Speech and multimediaTransmission Quality (STQ); Guidelines, objectives and results of speech quality analysis in the context of interworking Plugtests for multiplay services Part 1: Guidelines and objectives“. ETSI ETSI TR 102 716-
27、2 V1.1.2 (2010-11)73 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the following terms and definitions apply: platform: premise installed in residential environment where the accesses to different Multi Play offers proposed by ISP on the same country are mad
28、e available NOTE: This platform is generally installed in the centre of a city. 3.2 Abbreviations For the purposes of the present document, the following abbreviations apply: ADSL Asymmetric Digital Subscriber Line CPE Customer Premise Equipment DTMF Dual Tone Multi-Frequency HGW Home GateWay NOTE:
29、Referenced also as Residential Gateway. IP Internet Protocol IPTV IP TeleVisionNOTE: System where a digital television service is delivered using Internet Protocol. ISP Internet Service Provider ITU-T International Telecommunication Union - Telecommunication standardization sector MGCP Media Gateway
30、 Control Protocol MOS Mean Opinion Score MOS-LQON Mean Opinion Store - Listening Quality Objective Narrowband PDD Post Dialling Delay PESQ Perceptual Evaluation of Speech Quality PSTN Public Switched Telephone Network SIP Session Initiation Protocol TDM Time Division Multiplexing VoIP Voice over Int
31、ernet Protocol 4 Context To have an overview of the performances of Triple Play offers deployed in France (and used by customers), several platforms dedicated to technological watch on Triple Play offerings were organized. These platforms consist in installing in the same place all the offers propos
32、ed by different ISP to residential customers. The offer subscriptions are made from the point of view of the user. Particular care is taken to make sure that the ISP cannot be aware of the real use of these offers. This is an important point because in such a way we can objectively determine the qua
33、lity offered to the users. In fact, if the ISP is aware that an offer is made as part of a platform, it is then possible that the operator will adjust (or optimize) the functioning of this offer. The results presented in the present document, concern the performances of VoIP service associated to Tr
34、iple Play offers implemented on these technological platforms installed in France. The results are not obtained during Plugtests but the implemented methodology and the indicators are perfectly in accordance with elements presented in TR 102 716-1 i.6. The interest of these results is to give an ove
35、rview of the performance of deployed telephony services and not an overview of the performance of the prototype on a test platform. ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)8Annex A presents results obtained in October 2009 on two technological platforms installed in two geographic locations for IP to
36、 PSTN call configuration. Annex B presents results obtained during one year, from November 2008 till October 2009. These results refer to two platforms intalled in two cities in France. As for annex A, performance results are for IP to PSTN call configuration. Annex C presents call performances obta
37、ined in IP to IP calls between different operators in comparison with IP to PSTN configuration. These measurements were performed in the same platform in January 2010. 5 Platform presentation The platform is a premise installed in residential environment and in which there are the accesses to differ
38、ent Multi Play offers proposed by ISP on the same country. This platform is in most cases installed in the centre of a city. The platform is characterized by: Ability to implement of different offers concerning different ISPs. Each offer proposes 2 or 3 services: Internet access and VoIP, and when p
39、ossible IPTV. Access to services is obtained by an Home GateWay (HGW). Access to the network can be ADSL or cable type depending on the ISP. A PSTN access line is available for the speech quality analyses (IP to PSTN). As for network access, signalization protocol for voice service depend of ISP. So
40、 signalization protocol is not identical on all the offers and H.323, SIP or MGCP are implemented. 6 Presentation of test conditions The indicators and the implemented method are identical for the platforms. Concerning the tests, there is no difference between the offers and between the platforms. T
41、hat allows the comparison of performance between the offers of the same platform and globally comparison of performance between the two platforms. 6.1 Indicator description The indicators are the following ones. 6.1.1 Post Dialling Delay Definition Post Dialling Delay is the time interval between th
42、e end of dialling by the caller and the reception back by him of the appropriate ringing tone or recorded announcement. This indicator characterizes only the caller part of the call configuration. Assessment method Several measurements are performed sequentially and the mean value of measurement res
43、ults represents the determined value of the indicator. Unit Millisecond. ETSI ETSI TR 102 716-2 V1.1.2 (2010-11)96.1.2 Listening speech quality Definition Represents the intrinsic quality of speech signal after transmission. This indicator takes into account the degradations generated on the signal
44、by the transmission links. Assessment method Voice quality is evaluated by using the ITU-T Recommendation P.862 i.2 with the mapping functions according to ITU-T Recommendation P.862.1 i.3. Several MOS scores are determined in series during the same call. So listening speech quality performance duri
45、ng the call is defined by the mean value of MOS-LQON measurements (in the same transmission way). The voice quality indicator is determined in the two transmission directions by alternating the transmission way at each MOS score determination. For each transmission direction, 10 analyses are perform
46、ed. As the duration of the voice sample for speech analysis is about 20 seconds and a MOS score is determined every 30 seconds, the duration of a test call is about 10 minutes. Unit Rating between 1 (= very bad) and 5 (= excellent) determines on MOS-LQON scale. 6.1.3 Listening speech quality stabili
47、ty Definition This metric represents the stability of the voice quality during a communication of several minutes long. This indicator takes into account the signal degradation due to the transmission links. Assessment method The MOS scores determined for speech quality evaluation are used to calcul
48、ate the indicator characterizing speech quality stability. The methodology to perform this metric is described in EG 202 765-2 i.1. The major steps of stability indicator calculation are: - Determination of difference between successive MOS scores. - Evaluation of an instability level. - Transfer on
49、 a stability scale by using a linear function. This indicator is determined in the two directions of transmission. Unit Statistics on MOS score variation are plotted on a 0 to 100 scale. 6.1.4 End to end delay Definition Represent the global delay from one access to the other one. This indicator takes into account the transmission delay on networks but also processing delay in sending and receiving terminals. Assessment method Measuring the end to end delay is necessary to ensure a synchronization of both transmission ends of the measuremen
