1、Rec. ITU-R BS.1285 1RECOMMENDATION ITU-R BS.1285PRE-SELECTION METHODS FOR THE SUBJECTIVE ASSESSMENTOF SMALL IMPAIRMENTS IN AUDIO SYSTEMS(Questions ITU-R 85/10 and 106/10)(1997)Rec. ITU-R BS.1285The ITU Radiocommunication Assembly,consideringa) that Recommendations ITU-R BS.1116, ITU-R BT.500, ITU-R
2、BS.1284, ITU-R BT.710, ITU-R BT.1128 andITU-R BT.1129 have established some methods for assessing the subjective quality of audio and video systems;b) that subjective listening tests permit assessment of the degree of annoyance caused to the listener by anyimpairment of the wanted signal during its
3、transmission between the source and the listener;c) that classical objective methods may not be adequate for assessing advanced audio coding schemes and thatperceptual objective assessment methods are being developed for testing the sound quality of sound systems;d) that the use of standardized meth
4、ods is important for the exchange, compatibility and correct evaluation of thetest data;e) that the introduction of new advanced digital audio systems exploiting psycho-acoustic properties, especiallywith small impairments, requires advancements in subjective assessment methods;f) that Recommendatio
5、n ITU-R BS.1116 has established a method for the evaluation of the sound quality ofsound systems which introduce small impairments;g) that the detailed procedures of Recommendation ITU-R BS.1116 can be time-consuming and comparativelyexpensive to carry out and therefore a pre-selection method for re
6、jecting systems which introduce large impairments isdesirable,recommends1 that the testing, evaluation and reporting procedures given in Annex 1 be used as a means of distinguishingbetween those audio systems which introduce large impairments and other systems.ANNEX 11 GeneralThis Annex is divided i
7、nto the following sections, giving detailed requirements for various aspects of the tests:1 General2 Experimental design3 Selection of the listening panel4 Test method5 Attributes6 Programme material7 Choice of reproduction devices8 Listening conditions9 Statistical analysis10 Presentation of result
8、s11 Contents of test report.2 Rec. ITU-R BS.1285This Recommendation is based on Recommendation ITU-R BS.1116. It differs mainly in the section referring to testmethod.Recommendation ITU-R BS.1116 Methods for the subjective assessment of small impairments in audio systems,including multichannel sound
9、 systems, is intended for the assessment of audio systems which introduce impairments thatcan be hard to detect. For that purpose, it is necessary to control as many of the test parameters as possible and to carryout a sufficient number of assessments to produce statistically reliable estimates of t
10、he impairments. The requirements ofRecommendation ITU-R BS.1116 are consequently stringent. The procedure is costly in time and effort and is unneces-sary if the systems under test produce significant impairments.In order to avoid carrying out such tests unnecessarily, a pre-selection methodology th
11、at can reliably reject systemsintroducing large impairments is desirable. This reduction in the extent of the test method may reduce its sensitivity, butthe method must still be capable of distinguishing between systems which introduce large impairments and those thatdo not.2 Experimental designIn d
12、esigning the tests, the considerations of Recommendation ITU-R BS.1116, 2, should be taken into account.3 Selection of the listening panelFor the selection of the listening panel, the considerations of Recommendation ITU-R BS.1116, 3, should be taken intoaccount.However, to maximize the sensitivity
13、of the test and minimize the number of trials required to obtain reliable results, theexpert listeners should be thoroughly familiar with the types of impairment expected. That is, they should preferably bespecialists in the assessment of the types of coding systems under test.4 Test methodFor the d
14、etails of the test method, the general considerations of Recommendation ITU-R BS.1116, 4, should be takeninto account. As the test procedure is the most time-consuming aspect of Recommendation ITU-R BS.1116, it is byreduction of the test methodology that most savings in time and effort can be made.
15、At present, the following methodscan be recommended.4.1 Assessment using headphonesIf it can be shown that the systems under test can be properly assessed using headphone listening, then the assessmentsmay be carried out by more than one subject at the same time. This excludes systems which inherent
16、ly require the use ofloudspeakers, any systems or tests which involve the acoustics of the listening environment and systems involving anyspatial effects which cannot be properly reproduced by headphones.Using subjects in groups, individual switching is not practical and it is not possible to random
17、ize the order ofpresentation of stimuli for each test subject within a group. Between groups, different orders of presentation should beused. The principle of double-blind testing should be maintained.The data analysis should be carried out according to Recommendation ITU-R BS.1116, 9, as far as is
18、appropriate. Theanalysis should take into account the sub-division of the test subjects into groups.4.2 Assessment using small numbers of expert listenersThe time taken by the test procedure can be reduced greatly by reducing the number of listeners. This is only practicablewith expert listeners, wh
19、o have previously demonstrated high levels of accuracy and consistency in similar tests.The limited amount of data which might result from such tests may make useful statistical analysis impossible.Rec. ITU-R BS.1285 34.3 Further simplification of the test methodThe test procedure may be further sim
20、plified by using a two-way comparison. In that way, the test subjects are requiredto make only one assessment between two signals, one of which is known to be the reference signal. The referencesignal may be impaired and it may then be appropriate to use the seven-point comparison scale (see Recomme
21、n-dation ITU-R BS.1284 Methods for the subjective assessment of sound quality General requirements). It should benoted that such a method can give only a limited indication of the reliability of the subjects.5 AttributesRecommendation ITU-R BS.1116, 5, includes descriptions of the audio quality attr
22、ibutes that might be assessed, formonophonic, two-channel and multichannel stereophonic systems.6 Programme materialFor the selection of test programme material, the considerations of Recommendation ITU-R BS.1116, 6, should befollowed.7 Choice of reproduction devicesFor the chosen reproduction devic
23、es, the considerations of Recommendation ITU-R BS.1116, 7, should be followed.8 Listening conditionsFor the relevant listening conditions, the considerations of Recommendation ITU-R BS.1116, 8, should be followed.9 Statistical analysisFor the statistical analysis of the data, Recommendation ITU-R BS
24、1116, 9, should be followed as far as is appropriate.It is important to note that, due to the methodology employed, the results from the pre-selection tests can only be used todetermine whether or not to include or reject audio systems for subsequent formal tests. Only a statement about whethera sy
25、stem has “passed” or “failed” can be made. No further conclusions regarding the relative performances of thesystems may be drawn from the results of the pre-selection tests.The decision threshold should be set so that marginal systems are included in the “pass” category.10 Presentation of resultsFor
26、 the presentation of the results, the considerations of Recommendation ITU-R BS.1116, 10, should be taken intoaccount as far as is appropriate.The objective of the tests is simply to distinguish between systems that produce large impairments and those that do not.The final result will be simply a statement of whether a system has “passed” or “failed”.11 Contents of test reportThe contents of the test report should follow the considerations of Recommendation ITU-R BS.1116, 11, as far as isappropriate.
