1、 ETSI TS 103 106 V1.3.1 (2014-04) Speech and multimedia Transmission Quality (STQ); Speech quality performance in the presence of background noise: Background noise transmission for mobile terminals-objective test methods Technical Specification ETSI ETSI TS 103 106 V1.3.1 (2014-04)2Reference RTS/ST
2、Q-221 Keywords noise, quality, speech, testing, transmission ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Imp
3、ortant notice The present document can be downloaded from: http:/www.etsi.org The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization o
4、f ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware t
5、hat the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/
6、portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified
7、without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2014. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of it
8、s Members. 3GPPTM and LTETMare Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI TS 103 106 V1.3.1 (2014-04)3Contents Intellectual Property Rights 5g3Forew
9、ord . 5g31 Scope 6g32 References 6g32.1 Normative references . 7g32.2 Informative references 7g33 Abbreviations . 8g34 Introduction 9g35 Underlying speech databases and preparations 9g36 Modifications to the model described in EG 202 396-3 . 10g36.1 Prefiltering in Narrowband Mode (NB) . 10g36.2 Det
10、ection of the speech parts 10g36.3 Speech level adjustment in wideband . 11g36.4 Replacement of parameter regression for S-MOS 11g36.5 Retraining of parameter regression for N-MOS and G-MOS . 14g37 Comparison of objective and subjective results after the training process . 14g37.1 Results in wideban
11、d mode 15g37.1.1 Results for database “Audience - Test 3“ 15g37.1.2 Results for database “Audience - Test 3L“ (excluded during retraining) . 16g37.1.3 Results for database “Audience - Test 4“ 16g37.1.4 Results for database “Audience - Test 4L“ . 17g37.1.5 Results for database “Nokia - Test 1“ . 18g3
12、7.1.6 Results for database “Nokia - Test 2“ (excluded during retraining) . 18g37.1.7 Results for database “Orange“ 19g37.1.8 Results for database “Qualcomm - Test 3“ . 20g37.1.9 Results for database “Qualcomm - Test 4“ . 20g37.2 Results in narrowband mode 21g37.2.1 Results for database “Audience - T
13、est 1“ 21g37.2.2 Results for database “Audience - Test 1L“ . 22g37.2.3 Results for database “Audience - Test 2“ 22g37.2.4 Results for database “Audience - Test 2L“ . 23g37.2.5 Results for database “Qualcomm- Test 1“ 24g37.2.6 Results for database “Qualcomm- Test 2“ 24g38 Validation results 25g38.1 A
14、udience validation data 25g38.1.1 Description of tests . 25g38.1.2 Description of validation results . 27g38.1.2.1 Experiment 5: Narrowband . 27g38.1.2.2 Experiment 6: Narrowband . 29g38.1.2.3 Experiment 7: Wideband . 31g38.1.2.4 Experiment 8: Wideband . 33g38.2 Orange validation data 35g38.2.1 Desc
15、ription of tests . 35g38.2.2 Description of validation results . 36g38.3 Qualcomm validation data 38g38.3.1 Description of tests . 38g38.3.2 Description of validation results . 40g38.4 Validation data for additional use cases . 44g38.4.1 Tests 1 Essential, or potentially Essential, IPRs notified to
16、ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/ipr.etsi.org). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the exi
17、stence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specification (TS) has been produced by ETSI Technical Committee Speech and multimedia Transmission Quality (
18、STQ). The present document is to be used in conjunction with the ETSI standard series EG/S 202 396 i.2 to i.4: Part 1: “Background noise simulation technique and background noise database“; Part 2: “Background noise transmission - Network simulation - Subjective test database and results“; Part 3: “
19、Background noise transmission - Objective test methods“. The present document is based on the objective test method described in EG 202 396-3 i.4 and contains modifications of the model required in order to provide a good prediction of the uplink speech quality in the presence of background noise of
20、 modern mobile terminals. ETSI ETSI TS 103 106 V1.3.1 (2014-04)61 Scope The present document describes testing methodologies which can be used to objectively evaluate the performance of narrowband and wideband mobile terminals for speech communication in the presence of background noise. Background
21、noise is a problem in mostly all situations and conditions and needs to be taken into account in both, terminals and networks. The present document provides information about the testing methods applicable to objectively evaluate the speech quality of mobile terminals with AMR and AMR-WB codecs in t
22、he presence of background noise. The present document includes: The method which is applicable to objectively determine the different parameters influencing the speech quality in the presence of background noise taking into account: - the speech quality; - the background noise transmission quality;
23、- the overall quality. The description of the adaptation of the test method described in ES 202 396-1 i.2. The model results in comparison with the underlying subjective tests used for the retraining of the objective model. The model validation results: - Additional validation results are provided f
24、or cases which include some conditions outside the scope of ES 202 396-1 i.2. These include music as background noise, and user holding a handset in other than nominal position, as defined in Recommendation ITU-T P.64 i.24. In addition, validation results are provided for Chinese language. The prese
25、nt document is to be used in conjunction with: - ES 202 396-1 i.2 which describes a recording and reproduction setup for realistic simulation of background noise scenarios in lab-type environments for the performance evaluation of terminals and communication systems. - EG 202 396-2 i.3 which describ
26、es the simulation of network impairments and how to simulate realistic transmission network scenarios and which contains the methodology and results of the subjective scoring for the data forming the basis of the present document. - EG 202 396-3 i.4 which describes the basic objective model underlyi
27、ng to the Model described in the present document. - American English speech sentences as enclosed in the present document. 2 References References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited
28、 version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks in
29、cluded in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. ETSI ETSI TS 103 106 V1.3.1 (2014-04)72.1 Normative references The following referenced documents are necessary for the application of the present document. Not applicable. 2.2 Informative re
30、ferences The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 3GPP S4-120542: “Common subjective testing framework for training of P.835 test predictors“. i.2 ETSI ES 202 396-1: “Speech
31、 and multimedia Transmission Quality (STQ); Speech quality performance in the presence of background noise; Part 1: Background noise simulation technique and background noise database“. i.3 ETSI EG 202 396-2: “Speech Processing, Transmission and Quality Aspects (STQ); Speech Quality performance in t
32、he presence of background noise; Part 2: Background Noise Transmission - Network Simulation - Subjective Test Database and Results“. i.4 ETSI EG 202 396-3: “Speech and multimedia Transmission Quality (STQ); Speech Quality performance in the presence of background noise Part 3: Background noise trans
33、mission - Objective test methods“. i.5 ETSI TS 126 073: “Digital cellular telecommunications system (Phase 2+); Universal Mobile Telecommunications System (UMTS); LTE; ANSI C code for the Adaptive Multi Rate (AMR) speech codec (3GPP TS 26.073)“. i.6 Recommendation ITU-T P.835: “Subjective test metho
34、dology for evaluating speech communication systems that include noise suppression algorithm“. i.7 Recommendation ITU-T G.722.2: “Wideband coding of speech at around 16 kbit/s using Adaptive Multi-Rate Wideband (AMR-WB)“. i.8 Recommendation ITU-T P.56: “Objective measurement of active speech level“.
35、i.9 Recommendation ITU-T P.1401: “Methods, metrics and procedures for statistical evaluation, qualifying and comparison of objective quality prediction models“. i.10 Recommendation ITU-T G.160 Appendix II, Amendment 2: “Voice enhancement devices: Revised Appendix II - Objective measures for the char
36、acterization of the basic functioning of noise reduction algorithms“. i.11 Recommendation ITU-T G.191: “Software tools for speech and audio coding standardization“. i.12 Hastie, T.; Tibshirani, R.; Friedman, J.: “The Elements of Statistical Learning: Data Mining, Inference, and Prediction“, New York
37、: Springer-Verlag, 2001. i.13 Recommendation ITU-T P.501: “Test Signals for Use in Telephonometry“. i.14 Recommendation ITU-T P.58: “Head and Torso simulator for telephonometry“. i.15 Recommendation ITU-T P.57: “Artificial ears“. i.16 ETSI TS 126 131: “Universal Mobile Telecommunications System (UMT
38、S); LTE; Terminal acoustic characteristics for telephony; Requirements (3GPP TS 26.131 version 10.2.0 Release 10)“. i.17 Recommendation ITU-T P.800: “Methods for subjective determination of transmission quality“. i.18 ETSI TS 126 132: “Universal Mobile Telecommunications System (UMTS); LTE; Speech a
39、nd video telephony terminal acoustic test specification (3GPP TS 26.132)“. i.19 Void. ETSI ETSI TS 103 106 V1.3.1 (2014-04)8i.20 Recommendation ITU-T TD 477 (GEN/12): “Handbook of subjective test practical procedures“ (temporary document) - Geneva, 18-27 January 2011. i.21 AH-11-029, Better Referenc
40、e System for the P.835 SIG Rating Scale, Q7/12 Rapporteurs meeting, 20-21 June 2011, Geneva, Switzerland. i.22 3GPP, Tdoc S4(12)0621, Ext-ATS Permanent document (EATS-3): “Common subjective testing framework for validation of P.835 test predictors“. i.23 Recommendation ITU-T P.50: “Artificial voices
41、“. i.24 Recommendation ITU-T P.64: “Determination of sensitivity/frequency characteristics of local telephone systems“. 3 Abbreviations For the purposes of the present document, the following abbreviations apply: AMR Adaptive MultiRate AMR-NB Adaptive Multirate Codec - Narrow Band AMR-WB Adaptive Mu
42、lti-Rate Wideband Speech Codec BAK Background Noise Component dB SPL Sound Pressure Level re 20 Pa in dB DRP Drum Reference Point DTX Discontinous Transmission G-MOS Global MOS NOTE: MOS related to the overall sample. HATS Head and Torso Simulator HHHF Hand-Held Hands-FreeIRS Intermediate Reference
43、System NTT Nippon Telegraph and Telephone ITU International Telecommunication Union ITU-T Telecommunication Standardization Sector of ITU MOS Mean Opinion Score MRP Mouth Reference Point MSIN Mobile Station Input Filter NB NarrowBand N-MOS Noise MOSNOTE: MOS related to the noise transmission only. N
44、S Noise Suppression OVRL Overall (speech + noise) Component RCV ReCeiVe RMSE Root Mean Square Error RMSE* epsilon insensitive Root Mean Square Error SIG SIGnal component S-MOS Speech MOS NOTE: MOS related to the speech signal only. SND Sending Direction SNR Signal to Noise Ratio SPL Sound Pressure L
45、evelWB WideBandWCDMA Wideband Code Division Multiple Access ETSI ETSI TS 103 106 V1.3.1 (2014-04)94 Introduction The present document describes the modifications of the EG 202 396-3 i.4 model which were necessary to adapt to the training databases provided by the 3GPP contributors listed in annex A.
46、 The core model itself retains mainly unmodified except the points given in the clauses below. Modifications affect the narrow- and wideband mode in different ways. The adapted objective method described in the present document is intended to be used for all types of modern mobile terminals using di
47、fferent bitrates of AMR i.5 and AMR-WB i.7 coding. 5 Underlying speech databases and preparations The base for each mode of the objective model (wideband/narrowband) as described in EG 202 396-3 i.4 are listening tests conducted according to Recommendation ITU-T P.835 i.6. From the beginning of the
48、development, these listening test databases were designed to be a training set for predicting Recommendation ITU-T P.835 i.6 scores. They included a huge amount of conditions ( 170) and a wide range of speech and noise quality. Besides real terminals also terminal simulations and transmission impair
49、ments were included. However, the data and processing included were based on technologies actual at the time when the standard and its updates were created. The underlying databases for the retraining as described in the present document were created using real state-of-the-art mobile devices and thus the quality ranges yielded may not be normally distributed over all MOS scales. The context between the databases can also differ (e.g. pure handset recordings vs. mixed handset/hands-free dat
