1、Rec. ITU-R BT.1129-2 1RECOMMENDATION ITU-R BT.1129-2SUBJECTIVE ASSESSMENT OF STANDARD DEFINITIONDIGITAL TELEVISION (SDTV) SYSTEMS(Question ITU-R 211/11)(1994-1995-1998)Rec. ITU-R BT.1129-2The ITU Radiocommunication Assembly,consideringa) that a number of administrations and organizations throughout
2、the world are currently evaluating digitalsystems and that, in many parts of the world, digital broadcasting is likely to become the primary medium of the nextcentury;b) that subjective assessments are a vital element in the design and comparison of digital systems;c) that Recommendation ITU-R BT.50
3、0 outlines a number of preferred methods for subjective assessments, manydetails of which are appropriate in the context of digital television;d) that implicitly, and by virtue of the data transport techniques used, digital systems offer the opportunity ofintroducing multiple programming or scalable
4、 or hierarchical coding schemes in the broadcast channel;e) that in digital television systems, program contents may have a significant influence on the picture quality,recommends1 that the general methods for the subjective assessment of digital standard definition systems, including thoseoffering
5、multiple programme streams or scalable or hierarchical coding schemes, be as described in Recommen-dation ITU-R BT.500;2 that specific procedures for the subjective assessment of digital systems, including those offering multipleprogramme streams or scalable or hierarchical coding schemes, be as giv
6、en in Annex 1.ANNEX 11 IntroductionThis Annex, which is intended to be used in conjunction with Recommendation ITU-R BT.500, provides detailsconcerning the application of the general methods given in the Recommendation to subjective assessments of digitalsystems offering levels of quality at, or nea
7、r, those of conventional television systems. The procedural details given here,together with relevant background information, pertain to tests of codecs (or systems) used to convey material originatedaccording to Recommendation ITU-R BT.601 in contribution and distribution applications as well as to
8、 those used inemission applications.For distribution applications, quality specifications can be expressed in terms of the subjective judgement of observers.Such codecs can in theory therefore be assessed subjectively against these specifications. The quality of a codecdesigned for contribution appl
9、ications however, could not in theory be specified in terms of subjective performanceparameters because its output is destined not for immediate viewing, but for studio post-processing, storing and/orcoding for further transmission. Because of the difficulty of defining this performance for a variet
10、y of post-processingoperations, the approach preferred has been to specify the performance of a chain of equipment, including a2 Rec. ITU-R BT.1129-2post-processing function, which is thought to be representative of a practical contribution application. This chain mighttypically consist of a codec,
11、followed by a studio post-processing function (or another codec in the case of basiccontribution quality assessment), followed by yet another codec before the signal reaches the observer. Adoption of thisstrategy for the specification of codecs for contribution applications means that the measuremen
12、t procedures given in thisRecommendation can also be used to assess them.Although progress is being made, there is currently insufficient experience to give details of objective picture qualityassessment methods for codecs. In the area of subjective assessment, where much experience exists, test con
13、ditions andmethodologies can be recommended. It must be remembered, however, when specifying quality or impairment targets,that existing methods cannot give absolute subjective ratings but rather results which are influenced to some extent bythe choice of the reference and/or anchor conditions. The
14、same methodologies may be adopted for both fixed andvariable word-length codecs, and for intrafield and interframe codecs although the choice of test images sequences maybe influenced.At the present time, the most completely reliable method of evaluating the ranking order of high-quality codecs is t
15、oassess all the candidate systems at the same time under identical conditions. Tests made independently, where finedifferences of quality are involved, should be used for guidance rather than as indisputable evidence of superiority.A useful subjective measure may be impairment determined as a functi
16、on of the bit error ratio which occurs in thetransmission link between coder and decoder. At present there is insufficient experimental knowledge of truetransmission error statistics to recommend parameters for a model which accounts for error clustering or bursts. Untilthis information becomes avai
17、lable Poisson-distributed errors may be used.2 Viewing conditionsThe general viewing conditions for subjective assessments are those given in Recommendation ITU-R BT.500, 2.1.Specific viewing conditions for subjective assessments of digital systems are given in the following paragraphs.2.1 Laborator
18、y environmentThe laboratory environment is intended to provide critical conditions to check systems. Specific viewing conditions forsubjective assessments in the laboratory environment are given in the following Table 1.TABLE 1Specific viewing conditions for subjective assessmentsof digital systems
19、in laboratory environmentCondition Item Valuesa Ratio of viewing distance to picture height 4 H and 6 H (1)b Peak luminance 70 cd/m2c Viewing angle subtended by that portion of the background that meetsspecifications 43 H 57 Wd Display High quality screen.Size 20“ (50 cm) (2)(1)6 H is the design vie
20、wing distance (DVD) for the assessment of digital standard definition systems, but using assessors at4 H is also acceptable, provided that the results are given separately.(2)Because there is some evidence that display size may influence the results of subjective assessments experimenters arerequest
21、ed to explicitly report the screen size and make and model of displays used in any experiments.Rec. ITU-R BT.1129-2 32.2 Home environmentThis environment is intended to provide a mean to evaluate quality at the consumer side of the digital TV chain. Specificviewing conditions for digital standard de
22、finition digital television (SDTV) subjective assessments in the homeenvironment are given in Table 2.TABLE 2Specific viewing conditions for subjective assessmentsof digital systems in home environment3 Assessment methods3.1 Evaluations of basic picture qualityWhere a codec is being assessed for dis
23、tribution applications, this quality refers to pictures decoded after a single passthrough a codec pair. For contribution codecs, basic quality may be assessed after several codecs in series, in order tosimulate a typical contribution application.Where the range of quality to be assessed is small, a
24、s will normally be the case for television codecs, the testingmethodology to be used is variant II of the double-stimulus continuous quality-scale described in Recommenda-tion ITU-R BT.500. The original source sequence will be used as the reference condition. Further consideration is beinggiven to t
25、he duration of presentation sequences. In the recent tests on codecs for 4:2:2 component video, it wasconsidered advantageous to modify the presentation from that given in Recommendation ITU-R BT.500. Compositepictures were used as an additional reference to provide a lower quality level against whi
26、ch to judge the codecperformance.It is recommended that at least six picture sequences be used in the assessment, plus an additional one to be used fortraining purposes prior to the start of the trial. The sequences should range between moderately critical and critical in thecontext of the bit-rate
27、reduction application being considered.In subjective assessments, still pictures and moving sequences may be selected from those listed inRecommendation ITU-R BT.1210, Annex 1. In this respect, it should be noted that digitally stored pictures andsequences, being the most reproducible source signals
28、, are the preferred sources for assessments. Material such as thatdescribed in Recommendation ITU-R BT.1210 can be exchanged among laboratories in order to make systemcomparisons more meaningful. The D-1 4:2:2 tape format described in Recommendation ITU-R BT.657 should providea basis for such exchan
29、ge when such machines are widely and economically available. Exchange via computer tapeformats also is possible.Throughout this Annex, the importance is stressed of testing digital codecs with picture sequences which are critical inthe context of television bit-rate reduction. It is therefore reason
30、able to ask how critical a particular image sequence isfor a particular bit-rate reduction task, or whether one sequence is more critical than another. A simple but not especiallyCondition Item Valuesa Ratio of viewing distance to picture height 6 Hb Screen size for a 4/3 format ratio From 25“ to 29
31、“(1)c Screen size for a 16/9 format ratio From 32“ to 36“(1)d Monitor standard SDTVe Peak luminance 200 cd/m2f Environmental Illuminance on the screen(Incident light from the environment falling on the screen should bemeasured perpendicularly on the screen)200 Lux(1)This screen size satisfies rules
32、of the preferred viewing distance (PVD) for a PVD = 6 H.4 Rec. ITU-R BT.1129-2helpful answer is that “criticality” means very different things to different codecs. For example, to an intrafield codec astill picture containing much detail could well be critical, while to an interframe codec which is
33、capable of exploitingframe-to-frame similarities, this same scene would present no difficulty at all. Some sequences employing movingtexture and complex motion will be critical to all classes of codec so these types of sequences are most useful to generateor identify. Complex motion may take the for
34、m of movements which are predictable to an observer but not to codingalgorithms, such as tortuous periodic motion.One examination of possible statistical measures of image criticality, such as by correlative methods, spectral methods,conditional entropy methods etc. has revealed a simple but useful
35、measure based on an intrafield/interframe adaptiveentropy measurement. This method was used to “calibrate” picture sequences proposed for use in the ITU-R trials ofcodecs for 34, 45 and 140 Mbit/s and proved useful for the selection of the sequences used. The making of suchmeasurements on picture se
36、quences is most easily accomplished by transferring them to image processing computers andsubjecting them to analysis by software.Where access to these techniques is not available, the following presents some general guidelines on how to choosecritical material.a) Fixed word-length intrafield codecs
37、While it is possible and valid to assess these codecs on still images, the use of moving sequences is recommendedsince coding noise processes are easier to observe and this is more realistic of television applications. If still imagesare used in computer simulations of codecs, processing should be p
38、erformed over the entire assessment sequence inorder to preserve temporal aspects of any source noise, for example. The scenes chosen should contain as many aspossible of the following details: static and moving textured areas (some with coloured texture); static and movingobjects with sharp high co
39、ntrast edges at various orientation (some with colour); static plain mid-grey areas. At leastone sequence in the ensemble should exhibit just perceptible source noise and at least one sequence should besynthetic (i.e. computer generated) so that it is free from camera imperfections such as scanning
40、aperture and lag.b) Fixed word-length interframe codecsThe test scenes chosen should all contain movement and as many as possible of the following details: movingtextured areas (some coloured); objects with sharp, high contrast edges moving in a direction perpendicular to theseedges and at various o
41、rientations (some coloured). At least one sequence in the ensemble should exhibit justperceptible source noise and at least one sequence should be synthetic.c) Variable word-length intrafield codecsIt is recommended that these codecs be tested with moving image sequence material for the same reasons
42、 as thefixed word-length codecs. It should be noted that by virtue of its variable word-length coding and associated bufferstore, these codecs can dynamically distribute coding bit-capacity throughout the image. Thus, for example, if halfof a picture consists of a featureless sky which does not requ
43、ire many bits to code, capacity is saved for the otherparts of the picture which can therefore be reproduced with high quality even if they are critical. The importantconclusion from this is that if a picture sequence is to be critical for such a codec, the content of every part of thescreen should
44、be detailed. It should be filled with moving and static texture, as much colour variation as possibleand objects with sharp, high contrast edges. At least one sequence in the text ensemble should exhibit justperceptible source noise and at least one sequence should be synthetic.d) Variable word-leng
45、th interframe codecsThis is the most sophisticated class of codec and the kind which requires the most demanding material to stress it.Not only should every part of the scene be filled with detail as in the intrafield variable word-length case, but thisdetail should also exhibit motion. Furthermore,
46、 since many codecs employ motion compensation methods, themotion throughout the sequence should be complex. Examples of complex motion are: scenes employingsimultaneous zooming and panning of a camera; a scene which has as a background a textured or detailed curtainblowing in the wind; a scene conta
47、ining objects which are rotating in the three dimensional world; scenescontaining detailed objects which accelerate across the screen. All scenes should contain substantial motion ofobjects with different velocities, textures and high contrast edges as well as a varied colour content. At least onese
48、quence in the test ensemble should exhibit just perceptible source noise, at least one sequence should havecomplex computer generated camera motion from a natural still picture (so that it is free from noise and cameralag), and at least one sequence should be entirely computer generated.Rec. ITU-R B
49、T.1129-2 53.2 Evaluations of picture quality after downstream processingThis assessment is intended to permit judgement to be made on the suitability of a codec for contribution applicationswith respect to a particular post-process e.g. colour matte, slow motion, electronic zoom. The minimum arrangement ofequipment for such an assessment is a single pass through the codec under test, followed by the post-process of interest,followed by the viewer. It may, however, be more representative of a contribution application to employ further codecsafter the post-process.The test methodology t
