ITU-T J 246-2008 Perceptual visual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference (S.pdf

1、 International Telecommunication Union ITU-T J.246TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (08/2008) SERIES J: CABLE NETWORKS AND TRANSMISSION OF TELEVISION, SOUND PROGRAMME AND OTHER MULTIMEDIA SIGNALS Measurement of the quality of service Perceptual visual quality measurement techniques for

2、 multimedia services over digital cable television networks in the presence of a reduced bandwidth reference Recommendation ITU-T J.246 Rec. ITU-T J.246 (08/2008) i Recommendation ITU-T J.246 Perceptual visual quality measurement techniques for multimedia services over digital cable television netwo

3、rks in the presence of a reduced bandwidth reference1Summary The term multimedia as defined in Recommendation ITU-T J.148 is the combination of multiple forms of media such as: video, audio, text, graphics, fax, and telephony in the communication of information. A three stage approach has been adopt

4、ed to recommending objective assessment methods for multimedia. The first two stages will identify perceptual quality tools appropriate for measuring video and audio individually. The third stage will identify objective assessment methods for the combined audiovisual media. This Recommendation conta

5、ins the first stage video only used in multimedia applications. Recommendation ITU-T J.246 provides guidelines on the selection of appropriate objective perceptual video quality measurement methods when a reduced reference signal is available. The following are example applications that can use this

6、 Recommendation: 1) Internet multimedia streaming 2) Video telephony and conferencing over cable and other networks 3) Progressive video television streams viewed on LCD monitors over cable networks including those transmitted over the Internet using Internet Protocol. (VGA was the maximum resolutio

7、n in the validation test.) 4) Mobile video streaming over telecommunications networks 5) Some forms of IPTV video payloads (VGA was the maximum resolution in this validation test.) 6) Video quality monitoring at the receiver when side-channels are available. Source Recommendation ITU-T J.246 was app

8、roved on 13 August 2008 by ITU-T Study Group 9 (2005-2008) under Recommendation ITU-T A.8 procedures. _ 1The name was changed to reflect the fact that this Recommendation covers video only. ii Rec. ITU-T J.246 (08/2008) FOREWORD The International Telecommunication Union (ITU) is the United Nations s

9、pecialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on th

10、em with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-

11、T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used

12、 for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure e.g. interoperability or applicability) and compliance with the Re

13、commendation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any

14、 party. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Proper

15、ty Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implemen

16、ters are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2010 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written p

17、ermission of ITU. Rec. ITU-T J.246 (08/2008) iii CONTENTS Page 1 Scope 1 1.1 Application . 2 1.2 Limitations 2 2 References. 3 2.1 Normative references 3 2.2 Informative references 3 3 Definitions 4 3.1 Terms defined elsewhere 4 3.2 Terms defined in this Recommendation. 4 4 Abbreviations and acronym

18、s 5 5 Conventions 6 6 Description of the reduced reference measurement method. 6 7 Findings of the Video Quality Experts Group (VQEG) . 7 Annex A Yonsei University Reduced Reference Method 10 A.1 Introduction 10 A.2 The EPSNR Reduced Reference Models . 10 A.3 Conclusions 22 Appendix I Optimal side-c

19、hannel bandwidths 23 Appendix II Excerpts from the Synopsis from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment, phase I 26 II.1 Introduction 26 II.2 Model performance evaluation techniques. 27 II.3 RR model performance. 27 II.4 Data analysis e

20、xecuted by ILG. 29 Appendix III Equations for Model Evaluation Metrics 30 III.1 Evaluation Metrics 30 Bibliography. 33 Rec. ITU-T J.246 (08/2008) 1 Recommendation ITU-T J.246 Perceptual visual quality measurement techniques for multimedia services over digital cable television networks in the presen

21、ce of a reduced bandwidth reference 1 Scope This Recommendation provides guidelines and recommendations on the selection of appropriate perceptual video quality measurement equipment for use in multimedia applications when the reduced reference measurement method can be used. The reduced reference m

22、easurement method can be used when features extracted from the unimpaired reference video signal are readily available at the measurement point, as may be the case of measurements on individual equipment or a chain in the laboratory or in a closed environment such as a cable television head-end. The

23、 estimation methods are based on processing video in VGA, CIF, and QCIF resolution. The validation test material contained both multiple coding degradations and various transmission error conditions (e.g., bit errors, dropped packets). In the case where coding distortions are considered in the video

24、 signals, the encoder can utilize various compression methods (e.g., MPEG-2, H.264, etc.). The models proposed in this Recommendation may be used to monitor the quality of deployed networks to ensure their operational readiness. The visual effects of the degradations may include spatial as well as t

25、emporal degradations (e.g., frame repeats, frame skips, frame rate reduction). The models in this Recommendation can also be used for lab testing of video systems. When used to compare different video systems, it is advisable to use a quantitative method (such as that in ITU-T J.149) to determine th

26、e models accuracy for that particular context. This Recommendation is deemed appropriate for telecommunications services delivered at 4 Mbit/s or less presented on mobile/PDA and computer desktop monitors. The following conditions were allowed in the validation test for each resolution: PDA/Mobile (

27、QCIF): 16 kbit/s to 320 kbit/s CIF: 64 kbit/s-2 Mbit/s (C01 has several 2 Mbit/s) VGA: 128 kbit/s-4 Mbit/s (V13 has one HRC with 6 Mbit/s) Table 1 Factors for which J.246 has been evaluated Test factors Transmission errors with packet loss Video resolution QCIF, CIF and VGA Video bit rates QCIF: 16

28、kbit/s to 320 kbit/s CIF: 64 kbit/s-2 Mbit/s VGA: 128 kbit/s-4 Mbit/s Temporal errors (pausing with skipping) of maximum 2 seconds Video frame rates from 5 fps to 30 fps Coding technologies H.264/AVC (MPEG-4 Part 10), VC-1, Windows Media 9, Real Video (RV 10), MPEG-4 Part 2. See Note 1. 2 Rec. ITU-T

29、 J.246 (08/2008) Table 1 Factors for which J.246 has been evaluated Applications Real-time, in-service quality monitoring at the source Remote destination quality monitoring when side-channels are available for features extracted from source video sequences Quality measurement for monitoring of a st

30、orage or transmission system that utilizes video compression and decompression techniques, either a single pass or a concatenation of such techniques Lab testing of video systems NOTE 1 The validation testing of models included video sequences encoded using 15 different video codecs. The five codecs

31、 listed in this table were most commonly applied to encode test sequences and any recommended models may be considered appropriate for evaluating these codecs. In addition to these five codecs a smaller proportion of test sequences were created using the following codecs: Cinepak, DivX, H.261, H.263

32、, H.263+ (Note 2), JPEG-2000, MPEG-1, MPEG-2, Sorenson, H.264 SVC, Theora. It can be noted that some of these codecs were used only for CIF and QCIF resolutions because they are expected to be used in the field mostly for these resolutions. Before applying a model to sequences encoded using one of t

33、hese codecs the user should carefully examine its predictive performance to determine whether the model reaches acceptable predictive performance. NOTE 2 H.263+ is a particular configuration of H.263 (1998). 1.1 Application This Recommendation provides video quality estimations for video classes TV3

34、 to MM5B, as defined in Annex B of ITU-T P.911. Note that the maximum resolution was VGA and the maximum bit rate covered well in the test was 4 Mbit/s. The applications for the estimation models described in this Recommendation include but are not limited to: 1) potentially real-time, in-service qu

35、ality monitoring at the source; 2) remote destination quality monitoring when side-channels are available for features extracted from source video sequences; 3) quality measurement for monitoring of a storage or transmission system that utilizes video compression and decompression techniques, either

36、 a single pass or a concatenation of such techniques; 4) lab testing of video systems. 1.2 Limitations The estimation models described in this Recommendation cannot be used to fully replace subjective testing. Correlation values between two carefully designed and executed subjective tests (i.e., in

37、two different laboratories) normally fall within the range 0.95 to 0.98. If this Recommendation is utilized to make video system comparisons (e.g., comparing two codecs), it is advisable to use a quantitative method (such as that in ITU-T J.149) to determine the models accuracy for that particular c

38、ontext. The models in this Recommendation were validated by measuring video that exhibits frame freezes up to 2 seconds. The models in this Recommendation were not validated for measuring video that has a steadily increasing delay (e.g., video which does not discard missing frames after a frame free

39、ze). Rec. ITU-T J.246 (08/2008) 3 It should be noted that in case of new coding and transmission technologies producing artifacts which were not included in this evaluation, the objective models may produce erroneous results. Here a subjective evaluation is required. NOTE The structure and content o

40、f this Recommendation have been organized for ease of use by those familiar with the original source material; as such, the usual style of ITU-T recommendations has not been applied. 2 References 2.1 Normative references The following ITU-T Recommendations and other references contain provisions whi

41、ch, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of a

42、pplying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. IT

43、U-T J.143 Recommendation ITU-T J.143 (2000), User requirements for objective perceptual video quality measurements in digital cable television. ITU-T P.910 Recommendation ITU-T P.910 (2008), Subjective video quality assessment methods for multimedia applications. ITU-T P.911 Recommendation ITU-T P.9

44、11 (1998), Subjective audiovisual quality assessment methods for multimedia applications. 2.2 Informative references ITU-T H.261 Recommendation ITU-T H.261 (1993), Video codec for audiovisual services at p x 64 kbits. ITU-T H.263 Recommendation ITU-T H.263 (1996), Video coding for low bit rate commu

45、nication. ITU-T H.263+ Recommendation ITU-T H.263 (1998), Video coding for low bit rate communication (H.263+). ITU-T H.264 Recommendation ITU-T H.264 (2003), Advanced video coding for generic audiovisual services. ITU-T J.144 Recommendation ITU-T J.144 (2001), Objective perceptual video quality mea

46、surement techniques for digital cable television in the presence of a full reference. ITU-T J.148 Recommendation ITU-T J.148 (2003), Requirements for an objective perceptual multimedia quality model. ITU-T J.149 Recommendation ITU-T J.149 (2004), Method for specifying accuracy and cross-calibration

47、of Video Quality Metrics (VQM). ITU-T J.244 Recommendation ITU-T J.244 (2008), Calibration methods for constant misalignment of spatial and temporal domains with constant gain and offset. ITU-T P.931 Recommendation ITU-T P.931 (1998), Multimedia communications delay, synchronization and frame rate m

48、easurement. ITU-R BT.500-11 Recommendation ITU-R BT.500-11 (in force), Methodology for the subjective assessment of the quality of television pictures. 4 Rec. ITU-T J.246 (08/2008) VQEG Final report from the video quality experts group on the validation of objective models of multimedia quality-Phas

49、e I, 2008. 3 Definitions 3.1 Terms defined elsewhere This Recommendation uses the following terms defined elsewhere: 3.1.1 objective perceptual measurement (picture): ITU-T J.144 3.1.2 Proponent: ITU-T J.144 3.1.3 subjective assessment (picture): ITU-T J.144 3.2 Terms defined in this Recommendation This Recommendation defines the following terms: 3.2.1 anomalous frame repetition: An event where the HRC outputs a single frame repeatedly in response to an unusual or out of the ordinary event. Ano