ITU-T P 835 AMD 1-2007 Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm Amendment 1 New Appendix III C Additional pr(St.pdf

1、 International Telecommunication Union ITU-T P.835TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU Amendment 1(10/2007) SERIES P: TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Methods for objective and subjective assessment of quality Subjective test methodology for eva

2、luating speech communication systems that include noise suppression algorithm Amendment 1: New Appendix III Additional provisions for nonstationary noise suppressors ITU-T Recommendation P.835 (2003) Amendment 1 ITU-T P-SERIES RECOMMENDATIONS TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS,

3、LOCAL LINE NETWORKS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Subscribers lines and sets Series P.30 P.300 Transmission standards Series P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Seri

4、es P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of quality Series P.80 P.800 Audiovisual quality in multimedia services Series P.900 Transmission performance and QoS aspects of IP end-points Series P.1000 For further details, please refer t

5、o the list of ITU-T Recommendations. ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) i ITU-T Recommendation P.835 Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm Amendment 1 New Appendix III Additional provisions for nonstationary noise suppre

6、ssors Source Amendment 1 to ITU-T Recommendation P.835 (2003) was agreed on 11 October 2007 by ITU-T Study Group 12 (2005-2008). ii ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunicat

7、ions. The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunicatio

8、n Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of info

9、rmation technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating age

10、ncy. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure e.g. interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shall“ or some

11、other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or im

12、plementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development proc

13、ess. As of the date of approval of this Recommendation, ITU had not received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore strongl

14、y urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2008 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) 1 ITU-T Recommendation P.835 Subjecti

15、ve test methodology for evaluating speech communication systems that include noise suppression algorithm Amendment 1 New Appendix III Additional provisions for nonstationary noise suppressors (This appendix does not form an integral part of this Recommendation) III.1 General The purpose of this appe

16、ndix is to describe recommended test procedures for non-stationary acoustic noise suppressors. The voice quality test is designed to evaluate the performance of the noise suppressor in nominal/optimal physical positions. The tests are conducted in physical positions corresponding to the usage mode:

17、handset, far-talk, headset, speakerphone, and car handsfree mode. The recording environment uses a 4-speaker plus optional subwoofer configuration based on the b-ETSI EG 202 396-1 project recommendation. Male and female voices are used for the speech sources, and a variety of stationary and non-stat

18、ionary noise sources are used, including single voice, music, babble, street noise and car noise. III.2 Usage mode Handset mode: In handset mode, the device is held on the head-and-torso simulator and oriented and positioned as described in b-ITU-T P.64. Far-talk mode: In far-talk mode, the noise su

19、ppressor is held in front of the head-and-torso simulators face and oriented and positioned as described in b-3GPP TS 26.132. Headset mode: In headset mode, the device is placed on the ear of the head-and-torso simulator and oriented and positioned as described in b-ITU-T P.380. Speakerphone mode: I

20、n speakerphone mode, the noise suppressor is placed on a table in front of the head-and-torso simulator and oriented and positioned as described in b-ITU-T P.340. Car handsfree mode: In car handsfree mode, the device shall be positioned as described in b-ITU-T P.581. III.3 General recording properti

21、es Source recordings of both speech and noise are to be made separately, and then played into the apparatus of the noise suppressor and re-recorded in multiple tracks. Sample rate and bandwidth: For narrow-band noise suppressors, the sample rate of the recordings shall be 8 kHz, and the bandwidth sh

22、all be 300-3400 Hz, according to ITU-T Rec. P.48 and ITU-T P.830. For wideband noise suppressors, the sample rate of the recordings shall be 16 kHz, and the bandwidth shall be 100-7000 Hz, according to ITU-T P.830. Duration: The duration of each recording shall be at least 8 seconds (1 second backgr

23、ound noise, 2 seconds of talking and noise, 2 seconds of noise, 2 seconds of talking and noise, 1 second background noise). 2 ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) III.4 Speech source recordings Head-and-torso simulator with mouth simulator to play speech recordings: A head-and-torso simulator wit

24、h mouth simulator shall be used to play the speech recordings into the noise suppressor, at a nominal/optimal controlled distance and orientation according to the desired usage mode, following b-ITU-T P.51, b-ITU-T P.57, b-ITU-T P.58 and b-ITU-T P.340. Gender representation: The speech files shall i

25、nclude recordings from at least two male talkers and two female talkers. III.5 Noise source recordings Quad speakers plus subwoofer to play noise recordings: At least four loudspeakers plus subwoofer shall be used to play the noise recordings into the noise suppressor, at a distance of two metres fr

26、om the noise suppressor, consistent with b-ETSI EG 202 396-1 specification. Noise source files may be drawn from the b-ETSI EG 202 396-1 corpus. It is also acceptable to use four loudspeakers without separate subwoofer provided that the low frequency content of the spectrum of the audio content is f

27、aithfully reproduced. Noise source virtual motion: The noise recordings shall include noise source virtual motion, that is, noise sources that are played from one speaker, then played from another speaker in quick succession. Noise source multiple simultaneous sources: The noise recordings shall als

28、o include multiple simultaneous noise sources played from multiple speakers. Noise source types: The noise recordings shall include the following noise source types: Pink noise: Stationary noise recordings shall include pink noise files. Babble noise: Babble recordings shall be used that include spo

29、ken voices of at least 4 people, with equal numbers of male and female talkers, talking simultaneously. In at least some of the tests, babble shall be created with each of 4 separate recordings played through its own speaker, to create the acoustic environment of 4 separate noise sources. The distri

30、bution of male and female talkers should be as spatially balanced as possible. Street noise: Noise recordings shall include those made on a busy street. Car noise: Noise recordings shall include those made in a moving automobile. Single voice: Single-voice recordings shall be made with at least one

31、male speaker and at least one female speaker. Music: Music recordings shall be used that include drums. III.6 Signal-to-noise ratios Tests shall be performed using at least the following signal-to-noise ratios: 12 dB SNR 6 dB SNR 0 dB SNR In addition, there shall be a test performed on clean speech

32、(no added noise) to ensure no degradation by the noise suppressor (in the presence of a representative voice codec). While not required, it is also recommended that informal tests be performed at low speech levels to ensure that speech is not clipped. ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) 3 III.7

33、Post-processing by voice codec Files shall be post-processed by a representative voice codec, suitable for the system in which the noise suppressor will be deployed (AMR, EVRC, etc.). If the codec contains a noise suppression algorithm (i.e., EVRC, SMV, EVRC-B), this noise suppression algorithm shal

34、l be disabled during the post-processing. III.8 Initial convergence time The initial convergence time of the device under test shall be discarded so as to ensure that the device has converged. III.9 Test types Overall quality on clean speech: For a clean speech sample, the overall mean-opinion-score

35、 of the ITU-T P.835 procedures shall be measured with and without the noise suppressor. Overall quality improvement in noise: For the noisy speech samples, the overall mean-opinion-score of the ITU-T P.835 procedures shall be measured with and without the noise suppressor. III.10 Example: acceptance

36、 test format Recommendations This clause describes a recommended format for non-stationary noise suppressor acceptance tests. For each test, the ITU-T P.835 methodology is to be used. It is also possible to specify performance on the intermediate measures of voice quality and noise intrusiveness, or

37、 a combination of all the measures. 4 ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) Test Test Name Noise Source SNR Position Acceptance Criterion 12 dB X MOS 6 dB X MOS Overall Quality (Absolute MOS) 0 dB X MOS 12 dB X MOS 6 dB X MOS Overall MOS Improvement Pink 0 dB Nominal X MOS 12 dB X MOS 6 dB X MOS O

38、verall Quality (Absolute MOS) 0 dB X MOS 12 dB X MOS 6 dB X MOS Overall MOS Improvement Single-Voice 0 dB Nominal X MOS 12 dB X MOS 6 dB X MOS Overall Quality (Absolute MOS) 0 dB X MOS 12 dB X MOS 6 dB X MOS Overall MOS Improvement Music 0 dB Nominal X MOS 12 dB X MOS 6 dB X MOS Overall Quality (Abs

39、olute MOS) 0 dB X MOS 12 dB X MOS 6 dB X MOS Overall MOS Improvement Babble 0 dB Nominal X MOS 12 dB X MOS 6 dB X MOS Overall Quality (Absolute MOS) 0 dB X MOS 12 dB X MOS 6 dB X MOS Overall MOS Improvement Street 0 dB Nominal X MOS 12 dB X MOS 6 dB X MOS Overall Quality (Absolute MOS) 0 dB X MOS 12

40、 dB X MOS 6 dB X MOS Overall MOS Improvement Car 0 dB Nominal X MOS Voice Quality Test Overall MOS Improvement None Infinity Nominal X MOS ITU-T Rec. P.835 (2003)/Amd.1 (10/2007) 5 Bibliography b-ITU-T P.51 ITU-T Recommendation P.51 (1996), Artificial mouth. b-ITU-T P.57 ITU-T Recommendation P.57 (2

41、005), Artificial ears. b-ITU-T P.58 ITU-T Recommendation P.58 (1996), Head and torso simulator for telephonometry. b-ITU-T P.64 ITU-T Recommendation P.64 (1999), Determination of sensitivity/frequency characteristics of local telephone systems. b-ITU-T P.340 ITU-T Recommendation P.340 (2000), Transm

42、ission characteristics and voice quality parameters of hands-free terminals. b-ITU-T P.380 ITU-T Recommendation P.380 (2003), Electro-acoustic measurements on headsets. b-ITU-T P.581 ITU-T Recommendation P.581 (2000), Use of head and torso simulator (HATS) for hands-free terminal testing. b-ITU-T P.

43、862 ITU-T Recommendation P.862 (2001), Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs. b-3GPP TS 26.077 3GPP TS 26.077 (2003), Third Generation Partnership Project; Technical Specificati

44、on Group Services and System Aspects; Minimum Performance Requirements for Noise Suppresser; Application to the Adaptive Multi-Rate (AMR) speech encoder. b-3GPP TS 26.132 3GPP TS 26.132 (2007), Third Generation Partnership Project; Technical Specification Group Services and System Aspects; Universal

45、 Mobile Telecommunications System (UMTS); Speech and video telephony terminal acoustic test specification. b-ETSI EG 202 396-1 ETSI EG 202 396-1 (2006), Speech Processing, Transmission and Quality Aspects (STQ); Speech quality performance in the presence of background noise; Part 1: Background noise

46、 simulation technique and background noise database. Printed in Switzerland Geneva, 2008 SERIES OF ITU-T RECOMMENDATIONS Series A Organization of the work of ITU-T Series D General tariff principles Series E Overall network operation, telephone service, service operation and human factors Series F N

47、on-telephone telecommunication services Series G Transmission systems and media, digital systems and networks Series H Audiovisual and multimedia systems Series I Integrated services digital network Series J Cable networks and transmission of television, sound programme and other multimedia signals

48、Series K Protection against interference Series L Construction, installation and protection of cables and other elements of outside plant Series M Telecommunication management, including TMN and network maintenance Series N Maintenance: international sound programme and television transmission circu

49、its Series O Specifications of measuring equipment Series P Telephone transmission quality, telephone installations, local line networks Series Q Switching and signalling Series R Telegraph transmission Series S Telegraph services terminal equipment Series T Terminals for telematic services Series U Telegraph switching Series V Data communication over the telephone network Series X Data networks, open system communications and security Series Y Global information infrastructure, Inte