ITU-T P 833-2001 Methodology for Derivation of Equipment Impairment Factors from Subjective Listening-Only Tests Series P Telephone Transmission Quality Telephone Installations Locbj.pdf

1、INTERNATIONAL TELECOMMUNICATION UNION ITU-T TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU P.833 (02/200 I ) SERIES P: TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Methods for objective and subjective assessment of quality Methodology for derivation of equipment impa

2、irment factors from subjective listen i ng-on ly tests ITU-T Recommendation P.833 (Formerly CCITT Recommendation) ITU-T P-SERIES RECOMMENDATIONS TELEPHONE TRANSMISSION OUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NE“WOW Vocabulary and effects of transmission parameters on customer opinion of transmi

3、ssion quality Subscribers lines and sets Transmission standards Objective measuring apparatus Objective electro-acoustical measurements Measurements related to speech loudness Methods for objective and subjective assessment of quality Audiovisual quality in multimedia services Series P.10 Series P.3

4、0 Series P.40 Series P.50 Series P.60 Series P.70 Series P.80 P.800 Series P.900 P.300 P.500 For further details, please refer to the list of ITU-T Recommendations. ITU-T Recommendation P.833 Methodology for derivation of equipment impairment factors from subjective listening-only tests Summary This

5、 Recommendation gives a methodology for deriving equipment impairment factors (les) for digital signal processing devices, namely low bit-rate codecs with or without transmission errors. The le values are derived from the results of subjective listening-only tests. They are intended to be used as an

6、 input to the E-model (see ITU-T G. 107). Source ITU-T Recommendation P.833 was prepared by ITU-T Study Group 12 (2001-2004) and approved under the WTSA Resolution 1 procedure on 23 February 200 1. Keywords E-model, equipment impairment factor, impairment factor method, low bit-rate codecs. I ITU-T

7、P.833 (02/2001) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications. The IT Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff

8、 questions and issuing Recommendations on them with a view to stanardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommenda

9、tions on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with IS0 and IEC. NOTE In this Recommendati

10、on, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a c

11、laimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU ha

12、d received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementors are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database. o ITU 2002 All rights

13、reserved. No part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from ITU. 11 ITU-T P.833 (02/2001) CONTENTS 1 2 3 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.4 5.5 5.6 6 6.1 6.2 6.3 6

14、.4 6.5 6.6 7 Scope . References . Introduction . Selection of experiment parameters Preparation of test material . Selection of reference conditions 4.2.1 Reference conditions for low bit-rate codecs without transmission errors . 4.2.2 Reference conditions for low bit-rate codecs with transmission e

15、rrors . Test environment Listening panel Test method . Experiment design Choice of test stimuli Presentation method Opinion scales . Instructions to test subjects . Analysis of results . Derivation of equipment impairment factors Necessary amount of data . Scale transformation (Step 1) . Linear inte

16、rpolation of the test results (Step 2) . Additivity check (Step 3) Derivation of le values for transmission error conditions (Step 4) . Interpretation of derived equipment impairment factor values Additivity check (Step 5) Appendix I . Rating scale related to impairment factors . Appendix II . Bibli

17、ography . ITU-T P.833 (02/2001) Page 1 1 2 3 3 4 5 6 7 7 7 8 8 9 9 9 9 10 10 11 11 13 13 14 14 14 16 . 111 ITU-T Recommendation P.833 Methodology for derivation of equipment impairment factors from subjective listening-only tests 1 Scope This Recommendation describes the methodology for deriving equ

18、ipment impairment factors (les) fi-om subjective listening-only tests. It is intended that it primarily be applied to determining les for digital signal processing devices used in the network that have not otherwise been covered by the E-model. The equipment impairment factors derived by this method

19、ology are intended to be used in the E-model (see ITU-T G.107). They will reflect the auditory impairments of the corresponding equipment in a listening-only mode. 2 References The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitu

20、te provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recom

21、mendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published. ITU-T G. 107 (2000), The E-Model, a computational model for use in transmission planning. ITU-T G. 108.1 (2000), Guidance for assessing conversational speech transmission quality

22、 eflects not covered by the E-model. ITU-T G. 1 13 (200 1 ), Transmission impairments due to speech processing. ITU-T G.7 1 1 (1 988), Pulse code modulation (PCM) of voice9equencies. ITU-T G.726 (1 990), 40, 32, 24, 16 kbith adaptive diserential pulse code modulation (ADPCM). ITU-T G.727 (1 990), 5,

23、 4-, 3- and 2-bithample embedded adaptive diflerential pulse code modulation (ADPCM). ITU-T G.728 (1992), Coding of speech at 16 kbit/s using low-delay code excited linear prediction. ITU-T G.729 (1996), Coding of speech at 8 kbits using conjugate-structure algebraic-code-excited linear-prediction (

24、CS-A CELP). ITU-T P.800 (1 996), Methods for subjective determination of transmission quality. ITU-T P.8 1 O (1 996), Modulated noise reference unit (MNRU). ITU-T P.830 (1 996), Subjective pe$ormance assessment of telephone-band and wideband digital codecs. ITU-T P.83 1 (1 998), Subjective pe$ormanc

25、e evaluation of network echo cancellers. ITU-T Handbook, Handbook on Telephonometry, 1992. 1 ITU-T P.833 (02/2001) 3 Introduction The equipment impairment factor method (ITU-T G.113) is now the only one recommended by ITU-T for describing the subjective effects of digital processes other than pure P

26、CM on the integral quality for transmission planning purposes. It is based on the principle that individual transmission impairments can be transformed into “psychological factors“, and that these psychological factors are additive on a “psychological scale“. It is assumed that the transmission rati

27、ng scale underlying the E-model (ITU-T G. 107) has the properties of such a “psychological scale“. In transmission planning of modern telecommunication networks, it is important to consider the combined effects of different types of impairments occurring simultaneously in a connection. This is perfo

28、rmed by computational models like the E-model. Individual sources of degradations, such as a non-optimum overall loudness rating or sidetone path attenuation, talker echo attenuation and delay, quantizing distortion, absolute delay, etc., are transformed into so-called impairment factors. Degradatio

29、ns due to low bit-rate codecs are taken into account by means of an equipment impairment factor. Whereas interrelationships exist between the other impairment factors due to the underlying instrumentally measurable quantities of the connection (e.g. loudness ratings), the equipment impairment factor

30、 is by its very definition independent of all the other impairment factors. It is only dependent on the digital process whose perceptual characteristics it aims to model. The final transmission rating for the entire connection is calculated from all the individual impairment factors, by subtracting

31、them from a basic signal-to-noise ratio. For asynchronous tandems of multiple codecs of the same type, or of multiple codecs of different types, it is assumed that individual equipment impairment factors are additive. The overall equipment impairment factor for the chain of codecs is then calculated

32、 as a simple sum of all the individual les. Experimental test data collected provide evidence that this simple additivity is not satisfied for all the potential combinations of codecs 5. In some cases order effects exist, i.e. the tandem of codec A followed by codec B results in a different degradat

33、ion than codec B followed by codec A. These deviations from the pure additivity property will be examined in more detail. Equipment impairment factors have been introduced as a simplified measure of the perceptual effects of non-waveform codecs on the integral transmission quality from mouth to ear,

34、 for narrow-band (300-3400 Hz) handset telephony. They are in no way an exact description of the effects related to each individual codec or codec tandem, which may be very diverse in their perceptual nature. Instead, they represent the relative degradation in comparison to other impairments occurri

35、ng in a connection. In order to provide guidance on the quantitative amount of impairment introduced by such codecs, a fiamework of equipment impairment factors for the most common non-waveform codecs has been derived by ITU-T, see Appendix I/G. 1 13. Derivation is based on many subjective listening

36、-only tests carried out in different test laboratories, so as to guarantee stable values for all the codecs under investigation so far, as well as their relations to each other. If new equipment impairment factor values for different codecs have to be derived, then the overall consistency with the e

37、stablished framework is of primary importance if results are to be obtained that are valid for network planning. The methodology described in this Recommendation was designed to fulfil this requirement. As long as no instrumental methods can be defined, equipment impairment factors have to be derive

38、d from subjective test data. Such tests are generally carried out in a listening-only mode, which allows several, different connections to be tested within one test session. As a result, equipment impairment factors derived in this way will only reflect the influence on the one-way speech transmissi

39、on quality, and will disregard degradations in the conversational quality. It has not yet been proven that the quantitative degradation in the listening-only and in the conversational modes are similar, but for reasons of simplicity a strong interrelation is assumed. Users of transmission rating mod

40、els, however, should be aware that differences may exist, and that most of the data derived only reflects the listening-only situation. 2 ITU-T P.833 (02/2001) Subjective tests, even if carefully designed and carried out under controlled conditions, cannot provide quality ratings which are 100% repr

41、oducible under the same conditions. The composition and experience of the test panel, choice of test conditions and stimulus material, test Set-up and environment lead to an inherent variability. This variability can also be found in the mean ratings calculated over a large number of individual resp

42、onses. As a consequence, equipment impairment factors derived from one test will vary to a certain extent if compared to other test data. The degree of variability, however, should be limited to what can be tolerated in the transmission planning of networks, the latter including a reasonable safety

43、margin. The comparison and integration of test data obtained under different experimental conditions is one way to reduce variability. The strict application of the experimental design and test method described in the following clauses is an additional one. 4 Selection of experiment parameters In or

44、der to obtain valid and reliable results, some general requirements for the design of listening-only tests should be satisfied. Many of these requirements are described in detail in ITU-T P.800 and ITU-T P.830; therefore, wherever possible, reference to these Recommendations is made. Practical impli

45、cations on how to carry out subjective tests can be found in the ITU-T Handbook on Telephonometry. As a general fact, listening-only tests do not achieve the same standard of realism regarding the actual service conditions experienced by telephone customers as conversation tests do. However, whereve

46、r possible, the most realistic approach should also be chosen in listening-only tests, in order to guarantee that results obtained in the relatively artificial laboratory situation have significance for the actual service. 4.1 Preparation of test material The preparation of source recordings should

47、satisfy- with the exception of the points explicitly raised hereafter- the description given in B.lP.800, and in clause 7P.830, for narrow-band systems. This rule applies to the recording environment, the sending as well as the recording system, and the recording procedure. ITU-TP.800 and ITU-T P.83

48、0 recommend the use of simple, meaningful, short sentences as language material. Although tests carried out with this material will provide reasonable results, test realism will be enhanced by the use of longer, more diverse and meaningful material, such as short, meaningful test passages. Using suc

49、h material, the listeners attention will be focused more on the contents and not uniquely on the form of the speech sample, an effect which corresponds better to actual service conditions. Useful language material can be obtained e.g. from non-technical literature, newspapers, etc. The overall length of the test passages will be similar to the maximum obtained for the isolated sentences, i.e. around 12-1 5 seconds. If transmission error conditions are included in the test, a sufficient exposure to the degraded speech parts should be ensured due to the length of the test material.