ETSI EG 201 377-1-2009 Speech and multimedia Transmission Quality (STQ) Specification and measurement of speech transmission quality Part 1 Introduction to objective comparison mea.pdf

1、 ETSI EG 201 377-1 V1.3.2 (2009-10)ETSI Guide Speech and multimedia Transmission Quality (STQ);Specification and measurement ofspeech transmission quality;Part 1: Introduction to objective comparison measurementmethods for one-way speech quality across networksETSI ETSI EG 201 377-1 V1.3.2 (2009-10)

2、 2Reference REG/STQ-00105-1 Keywords interworking, quality, speech, testing, transmission, voice ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Pr

3、fecture de Grasse (06) N 7803/88 Important notice Individual copies of the present document can be downloaded from: http:/www.etsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such

4、versions, the reference version is the Portable Document Format (PDF). In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject

5、to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_su

6、pport.asp Copyright Notification No part may be reproduced except as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2009. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTM, TIPHONTM, th

7、e TIPHON logo and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. LTE is a Trade Mark of ETSI currently being registered for the benefit of its Members a

8、nd of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI EG 201 377-1 V1.3.2 (2009-10) 3Contents Intellectual Property Rights 5g3Foreword . 5g31 Scope 6g32 References 8g32.1 Normative references . 8g32.2 Informative reference

9、s 8g33 Definitions and abbreviations . 11g33.1 Definitions 11g33.2 Abbreviations . 11g34 Overview 12g34.1 Objective 13g34.2 Related work in standardization . 13g35 Definition of mouth-to-ear speech quality . 14g35.1 General definition . 14g35.2 Human perception characteristics of speech quality 14g3

10、5.2.1 Physical characteristics and psychological impacts 14g35.2.2 Inter-subject differences . 16g35.2.3 Intra-subject differences . 16g35.2.4 Language-dependent differences 16g35.3 Network-related issues . 16g35.3.1 Reference configuration for mouth-to-ear measurement 16g35.3.2 Standardization of q

11、uality parameters. 17g35.3.3 Modelling of networks - anomalies 17g35.4 Terminal equipment related issues . 18g35.5 Technical basis for measurement . 18g35.5.1 Quantification and measurement of speech quality 18g35.5.2 Required characteristics of speech samples 19g36 Subjective measurement of speech

12、quality. 20g36.1 Subjective measurement methods 20g36.2 Application of statistical methods 20g37 Objective measurement methods 21g37.1 Basics of speech sample based objective measurement methods . 22g37.2 Pre-processing 24g37.2.1 Adjustment unit 24g37.2.2 Modelling and/or measuring transmitter and r

13、eceiver environment . 24g37.3 Psycho-acoustic sound perception 24g37.3.1 Time-frequency mapping 25g37.3.2 Linear prediction coefficients . 25g37.3.3 Cepstrum . 25g37.3.4 Mapping to perceptual (critical band) domain 26g37.3.5 Frequency masking . 26g37.3.6 Time masking . 27g37.3.7 Psycho-acoustic loud

14、ness . 27g37.3.8 Hair cell firing . 28g37.3.9 Specific modelling of annoying components “Relative Approach“ 28g37.4 Comparison of reference and transmitted signal 29g37.4.1 Euclidean distance 29g37.4.2 Generalized distance . 29g37.4.3 Asymmetric differences 30g37.4.4 Distance between probability fun

15、ctions 30g37.4.5 Multi-resolution analysis 30g37.4.6 Compression to single number 30g3ETSI ETSI EG 201 377-1 V1.3.2 (2009-10) 47.4.7 Mapping to MOS scale . 30g37.5 Comparability of Objective Model Results 31g37.5.1 Comparison of Results between Models . 31g37.5.2 Comparison of Results of one Model I

16、mplemented in Different Test Equipment 31g37.5.3 Comparison of Results of one Model in Different Scenarios . 31g37.5.4 Optimization of Systems based on the Results of One Model 31g38 Overview of INMD 32g39 Overview of Single-ended Objective Speech Quality Assessment 32g310 Overview of the E-Model .

17、33g311 Use of building blocks in some known systems . 34g311.1 Comparison-based schemes 34g311.2 E-Model 35g3Annex A: Examples of specific systems 36g3A.1 Perceptual Speech Quality Measure (PSQM) 36g3A.2 Measuring Normalizing Blocks (MNB) . 37g3A.3 PACE 38g3A.4 Telecommunication Objective Speech Qua

18、lity Assessment (TOSQA) . 39g3A.5 Perceptual Analysis/Measurement System (PAMS) 40g3A.6 ITU-T Recommendation P.862: Perceptual Evaluation of Speech Quality . 41g3A.7 EG 202 396-3: Background noise transmission - Objective test methods . 42g3Annex B: Terminal equipment related issues 44g3B.1 Overview

19、 44g3Annex C: Subjective measurement methods . 47g3C.1 Absolute Category Rating (ACR) 47g3C.2 Degradation Category Rating (DCR) . 47g3C.3 Comparison Category Rating (CCR) . 47g3C.4 Interview and survey test 48g3C.5 Conversational tests 48g3C.6 Double talk tests . 49g3C.7 Talking and listening tests

20、49g3C.8 Listening-only test procedure . 49g3Annex D: Application of statistical methods 50g3D.1 Statistical relevance of results 50g3D.2 Estimation of confidence intervals . 51g3D.3 ANOVA . 52g3Annex E: Bibliography 53g3History 54g3ETSI ETSI EG 201 377-1 V1.3.2 (2009-10) 5Intellectual Property Right

21、s IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential,

22、or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been c

23、arried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This ETSI Guide (EG) has been produced by ETSI Technical Committe

24、e Speech and multimedia Transmission Quality (STQ). The present document is part 1 of a multi-part deliverable covering the specification and measurement of speech transmission quality, as identified below: Part 1: “Introduction to objective comparison measurement methods for one-way speech quality

25、across networks“; Part 2: “Mouth-to-ear speech transmission quality including terminals“; Part 3: “Non-intrusive objective measurement methods applicable to networks and links with classes of services“. ETSI ETSI EG 201 377-1 V1.3.2 (2009-10) 61 Scope The present document is part 1 of a series of do

26、cuments on the specification and measurement of mouth-to-ear (also end-to-end) speech transmission quality. Its main objective is to describe objective comparison-based methods and systems for measuring mouth-to-ear speech quality in networks. Apart from this, it gives an overview on other important

27、 aspects of mouth-to-ear speech quality. As the need arises, these other aspects will be covered in more detail in subsequent parts of the present document. Although some of the models described in the present document are superseded the description of the models is kept for information. The present

28、 document gives an overview of the methods available for measuring one-way speech transmission quality. Its purpose is to give information and guidance primarily for operators, users, consumer organizations and regulators who wish to measure or compare the speech transmission quality provided by dif

29、ferent networks. The need for the present document has been increased by: the liberalization of voice services, which has introduced alternative competing providers of voice services; the introduction of new mobile and IP based technologies; which has increased the range of services and cost/quality

30、 options for users. The present document applies to both fixed and mobile networks with or without terminal equipment connected to the network. It applies only for narrowband (i.e. between 300 Hz and 3 400 Hz) communications. In principle, comparison methods can be used for IP-based (internet protoc

31、ol-based) networks, but further work is needed on the calibration of the methods for such networks. The present document describes: methods for measurements of individual impairments or combinations of impairments to be made at acoustic or electrical interfaces; methods for combining measures of dif

32、ferent impairments into a single objective measure; methods for predicting the subjective effect of impairments that would be perceived by users. The methods in the present document assume that subjects with normal hearing have been involved in the test. Therefore, the instrumental methods estimate

33、the perceived speech quality of persons with normal hearing. For each method, the guide contains a general description to highlight the main points, and provides references for more detailed information. The present document does not contain detailed specifications of the individual methods. The pre

34、sent document concentrates on one-way speech quality in networks. It gives no guidance on how to evaluate systems that include equipment such as echo cancellers or in which interactive impairments such as talker echo are significant. The perceived quality in such cases depends not only on the one-wa

35、y performance, but very much on the behaviour of the equipment under duplex conditions; specifically, the influence of double-talk and delay needs to be considered. Although all assessments of overall speech quality are ultimately subjective because they depend on the users opinion, a distinction is

36、 made between: subjective methods, which involve real time user assessment; and objective methods, which use stored information on the users assessment and therefore involve some degree of calibration. Objective methods for the evaluation of speech quality fall into three categories: a) Comparison M

37、ethods: Methods based on the comparison of transmitted speech signal and a known reference. b) Absolute Estimation Methods: Methods based on the absolute estimation of the speech quality (i.e. there is no known reference signal); e.g. INMD (ITU-T Recommendation P.561 i.16). c) Transmission Rating Mo

38、dels: Methods that derive a value for the expected speech quality from knowledge about the network; e.g. ETSI Model (ETR 250 i.1, ITU-T Recommendation G.107 i.14). ETSI ETSI EG 201 377-1 V1.3.2 (2009-10) 7The classification of assessment methods is depicted in figure 1. Practical implementations of

39、test equipment may include combinations of these methods. The focus of the present document is on comparison methods (intrusive methods), which currently yield the most accurate results. The other categories are only covered in short overviews, although they may be preferable for certain application

40、s. individually measured impairments of all impairments E - Model Predict combined effect of Measure combined effect Subjective and Objective Methods Signal - based measurements e.g. speech samples and other test signals Additional information e.g.: radio link budget error rate cell loss rate Subjec

41、tive methods e.g., listening - only and conversational Real - time assessment Stored experience (i.e., past assessments) Additional experience Calibration Objective methods Comparison Absolute Estimation Transmission Rating Models Parameter - based models Users subjective assessment a) b) c) Talker

42、Listener Network Measure- ment Talker Listener Network Measure- ment Computa- tion using knowledge of network Parameters of: “Talker“, Network, “Listener“ Figure 1: Classification of assessment methods showing: a) Comparison methods, b) Absolute estimation methods, c) Transmission rating models NOTE

43、: As an ETSI Guide, the present document provides guidelines for test methods that may be implemented. However, a test method and especially quality models can only be applied in the way and within the scope defined in the reference standard. A “Warning“ indicates when this applies. ETSI ETSI EG 201

44、 377-1 V1.3.2 (2009-10) 82 References References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For a specific reference, subsequent revisions do not apply. Non-specific reference may be made only to a complete document or a part ther

45、eof and only in the following cases: - if it is accepted that it will be possible to use all future changes of the referenced document for the purposes of the referring document; - for informative references. Referenced documents which are not found to be publicly available in the expected location

46、might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee their long term validity. 2.1 Normative references The following referenced documents are indispensable for the application of the present

47、 document. For dated references, only the edition cited applies. For non-specific references, the latest edition of the referenced document (including any amendments) applies. Not applicable. 2.2 Informative references The following referenced documents are not essential to the use of the present do

48、cument but they assist the user with regard to a particular subject area. For non-specific references, the latest version of the referenced document (including any amendments) applies. i.1 ETSI ETR 250: “Transmission and Multiplexing (TM); Speech communication quality from mouth to ear for 3,1 kHz h

49、andset telephony across networks“. i.2 ETSI EG 201 050: “Speech Processing, Transmission and Quality Aspects (STQ); Overall Transmission Plan Aspects for Telephony in a Private Network“. i.3 ETSI TR 102 082: “Speech Processing, Transmission and Quality Aspects (STQ); Guidance on writing specifications and tests for non-linear and time variant telephony terminals“. i.4 ETSI EG 202 396-1: “Speech and multimedia Transmission Quality (STQ); Speech quality performance in the presence of background noise; Part 1: Background noise simula