1、 ETSI TS 103 106 V1.5.1 (2018-04) Speech and multimedia Transmission Quality (STQ); Speech quality performance in the presence of background noise: Background noise transmission for mobile terminals-objective test methods TECHNICAL SPECIFICATION ETSI ETSI TS 103 106 V1.5.1 (2018-04)2Reference RTS/ST
2、Q-271 Keywords noise, quality, speech, testing, transmission ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Imp
3、ortant notice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior writte
4、n authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document
5、should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the
6、 following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the
7、 PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. ETSI 2018. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are trademarks of ETSI registered for the benefit of its Members
8、. 3GPPTM and LTETMare trademarks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. oneM2M logo is protected for the benefit of its Members. GSMand the GSM logo are trademarks registered and owned by the GSM Association. ETSI ETSI TS 103 106 V1.5.1 (2018-04)3C
9、ontents Intellectual Property Rights 5g3Foreword . 5g3Modal verbs terminology 5g31 Scope 6g32 References 6g32.1 Normative references . 6g32.2 Informative references 7g33 Abbreviations . 8g34 Introduction 9g35 Underlying speech databases and preparations 9g36 Modifications to the model described in E
10、TSI EG 202 396-3 . 10g36.1 Prefiltering in Narrowband Mode (NB) . 10g36.2 Void 10g36.3 Speech level adjustment in wideband . 11g36.4 Modified neural network for S-MOS . 11g36.5 Retraining of parameter regression for N-MOS and G-MOS . 12g37 Comparison of objective and subjective results after the tra
11、ining process . 13g37.0 General . 13g37.1 Results in wideband mode 13g37.1.0 General 13g37.1.1 Results for database “Audience - Test 3“ 14g37.1.2 Results for database “Audience - Test 3L“ (excluded during retraining) . 14g37.1.3 Results for database “Audience - Test 4“ 15g37.1.4 Results for database
12、 “Audience - Test 4L“ . 16g37.1.5 Results for database “Nokia - Test 1“ . 16g37.1.6 Results for database “Nokia - Test 2“ (excluded during retraining) . 17g37.1.7 Results for database “Orange“ 18g37.1.8 Results for database “Qualcomm - Test 3“ . 18g37.1.9 Results for database “Qualcomm - Test 4“ . 1
13、9g37.2 Results in narrowband mode 19g37.2.0 General 19g37.2.1 Results for database “Audience - Test 1“ 20g37.2.2 Results for database “Audience - Test 1L“ . 20g37.2.3 Results for database “Audience - Test 2“ 21g37.2.4 Results for database “Audience - Test 2L“ . 22g37.2.5 Results for database “Qualco
14、mm- Test 1“ 22g37.2.6 Results for database “Qualcomm- Test 2“ 23g38 Validation results 23g38.0 Preamble . 23g38.1 Audience validation data 24g38.1.1 Description of tests . 24g38.1.2 Description of validation results . 25g38.1.2.0 General explanation 25g38.1.2.1 Experiment 5: Narrowband . 25g38.1.2.2
15、 Experiment 6: Narrowband . 27g38.1.2.3 Experiment 7: Wideband . 29g38.1.2.4 Experiment 8: Wideband . 31g38.2 Orange validation data 33g38.2.1 Description of tests . 33g38.2.2 Description of validation results . 34g38.3 Qualcomm validation data 35g38.3.1 Description of tests . 35g3ETSI ETSI TS 103 1
16、06 V1.5.1 (2018-04)48.3.2 Description of validation results . 38g38.4 Validation data for additional use cases . 42g38.4.1 Tests 1 Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on t
17、he ETSI Web server (https:/ipr.etsi.org/). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may
18、be, or may become, essential to the present document. Trademarks The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners. ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no righ
19、t to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks. Foreword This Technical Specification (TS) has been produced by ETSI Technical
20、 Committee Speech and multimedia Transmission Quality (STQ). The present document is to be used in conjunction with the ETSI ES 202 396-1 i.2 and ETSI EG 202 396-3 i.4: ETSI ES 202 396-1: “Background noise simulation technique and background noise database“; ETSI EG 202 396-3: “Background noise tran
21、smission - Objective test methods“. The present document is based on the objective test method described in ETSI EG 202 396-3 i.4 and contains modifications of the model required in order to provide a good prediction of the uplink speech quality in the presence of background noise of modern mobile t
22、erminals. Modal verbs terminology In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “m
23、ust not“ are NOT allowed in ETSI deliverables except when used in direct citation. ETSI ETSI TS 103 106 V1.5.1 (2018-04)61 Scope The present document describes testing methodologies which can be used to objectively evaluate the performance of narrowband and wideband mobile terminals for speech commu
24、nication in the presence of background noise. Background noise is a problem in mostly all situations and conditions and needs to be taken into account in both, terminals and networks. The present document provides information about the testing methods applicable to objectively evaluate the speech qu
25、ality of mobile terminals with AMR and AMR-WB codecs in the presence of background noise. The present document includes: The method which is applicable to objectively determine the different parameters influencing the speech quality in the presence of background noise taking into account: - the spee
26、ch quality; - the background noise transmission quality; - the overall quality. The description of the adaptation of the test method described in ETSI ES 202 396-1 i.2. The model results in comparison with the underlying subjective tests used for the retraining of the objective model. The model vali
27、dation results: - Additional validation results are provided for cases which include some conditions outside the scope of ETSI ES 202 396-1 i.2. These include music as background noise, and user holding a handset in other than nominal position, as defined in Recommendation ITU-T P.64 i.24. In additi
28、on, validation results are provided for Chinese language. The present document is to be used in conjunction with: - ETSI ES 202 396-1 i.2 which describes a recording and reproduction setup for realistic simulation of background noise scenarios in lab-type environments for the performance evaluation
29、of terminals and communication systems. - ETSI EG 202 396-3 i.4 which describes the basic objective model underlying to the Model described in the present document. - American English speech sentences as enclosed in the present document. 2 References 2.1 Normative references References are either sp
30、ecific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are n
31、ot found to be publicly available in the expected location might be found at https:/docbox.etsi.org/Reference/. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary f
32、or the application of the present document. Not applicable. ETSI ETSI TS 103 106 V1.5.1 (2018-04)72.2 Informative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version app
33、lies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are not ne
34、cessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 3GPP S4-120542: “Common subjective testing framework for training of P.835 test predictors“. i.2 ETSI ES 202 396-1: “Speech and multimedia Transmission Quality (STQ); Speech qua
35、lity performance in the presence of background noise; Part 1: Background noise simulation technique and background noise database“. i.3 Void. i.4 ETSI EG 202 396-3: “Speech and multimedia Transmission Quality (STQ); Speech Quality performance in the presence of background noise Part 3: Background no
36、ise transmission - Objective test methods“. i.5 ETSI TS 126 073: “Digital cellular telecommunications system (Phase 2+) (GSM); Universal Mobile Telecommunications System (UMTS); LTE; ANSI-C code for the Adaptive Multi Rate (AMR) speech codec (3GPP TS 26.073)“. i.6 Recommendation ITU-T P.835: “Subjec
37、tive test methodology for evaluating speech communication systems that include noise suppression algorithm“. i.7 Recommendation ITU-T G.722.2: “Wideband coding of speech at around 16 kbit/s using Adaptive Multi-Rate Wideband (AMR-WB)“. i.8 Recommendation ITU-T P.56: “Objective measurement of active
38、speech level“. i.9 Recommendation ITU-T P.1401: “Methods, metrics and procedures for statistical evaluation, qualifying and comparison of objective quality prediction models“. i.10 Void. i.11 Recommendation ITU-T G.191: “Software tools for speech and audio coding standardization“. i.12 Void. i.13 Re
39、commendation ITU-T P.501: “Test Signals for Use in Telephonometry“. i.14 Recommendation ITU-T P.58: “Head and Torso simulator for telephonometry“. i.15 Recommendation ITU-T P.57: “Artificial ears“. i.16 ETSI TS 126 131: “Universal Mobile Telecommunications System (UMTS); LTE; Terminal acoustic chara
40、cteristics for telephony; Requirements (3GPP TS 26.131)“. i.17 Recommendation ITU-T P.800: “Methods for subjective determination of transmission quality“. i.18 ETSI TS 126 132: “Universal Mobile Telecommunications System (UMTS); LTE; Speech and video telephony terminal acoustic test specification (3
41、GPP TS 26.132)“. i.19 Void. i.20 Recommendation ITU-T TD 477 (GEN/12): “Handbook of subjective test practical procedures“ (temporary document) - Geneva, 18-27 January 2011. i.21 AH-11-029: “Better Reference System for the P.835 SIG Rating Scale“, Q7/12 Rapporteurs meeting, 20-21 June 2011, Geneva, S
42、witzerland. ETSI ETSI TS 103 106 V1.5.1 (2018-04)8i.22 3GPP, Tdoc S4(12)0621, Ext-ATS Permanent document (EATS-3): “Common subjective testing framework for validation of P.835 test predictors“. i.23 Recommendation ITU-T P.50: “Artificial voices“. i.24 Recommendation ITU-T P.64: “Determination of sen
43、sitivity/frequency characteristics of local telephone systems“. 3 Abbreviations For the purposes of the present document, the following abbreviations apply: 78KBP 7,8 kHz band-pass according to Recommendation ITU-T G.191 AMR Adaptive MultiRate AMR-NB Adaptive Multirate Codec - Narrow Band AMR-WB Ada
44、ptive Multi-Rate Wideband Speech Codec BAK Background Noise Component dB SPL Sound Pressure Level re 20 Pa in dB DRP Drum Reference Point DTX Discontinuous Transmission EATS Enhanced Acoustic Test Specification EXP Experiment FB FullbandG-MOS Global MOS NOTE: MOS related to the overall sample. HATS
45、Head And Torso Simulator HHHF Hand-Held Hands-Free IRS Intermediate Reference System ITU International Telecommunication Union ITU-T Telecommunication Standardization Sector of ITU MOS Mean Opinion Score MRP Mouth Reference Point MSIN Mobile Station Input Filter NB Narrowband N-MOS Noise MOSNOTE: MO
46、S related to the noise transmission only. NS Noise Suppression NTT Nippon Telegraph and Telephone OVRL Overall (speech + noise) Component PRO Professional RCV ReCeiVeRMSE Root Mean Square Error RMSE* epsilon insensitive Root Mean Square Error SIG SIGnal component S-MOS Speech MOS NOTE: MOS related t
47、o the speech signal only. SND Sending Direction SNR Signal to Noise Ratio SPL Sound Pressure LevelWB WidebandWCDMA Wideband Code Division Multiple Access ETSI ETSI TS 103 106 V1.5.1 (2018-04)94 Introduction The present document describes the modifications of the ETSI EG 202 396-3 i.4 model which wer
48、e necessary to adapt to the training databases provided by the 3GPP contributors listed in annex A. The core model itself retains mainly unmodified except the points given in the clauses below. Modifications affect the narrow- and wideband mode in different ways. The adapted objective method describ
49、ed in the present document is intended to be used for all types of modern mobile terminals using different bitrates of AMR i.5 and AMR-WB i.7 coding. 5 Underlying speech databases and preparations The base for each mode of the objective model (wideband/narrowband) as described in ETSI EG 202 396-3 i.4 are listening tests conducted according to Recommendation ITU-T P.835 i.6. From the beginning of the development, these listening test databases were designed to be a training set for predicting R
