1、 International Telecommunication Union ITU-T J.247TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (08/2008) SERIES J: CABLE NETWORKS AND TRANSMISSION OF TELEVISION, SOUND PROGRAMME AND OTHER MULTIMEDIA SIGNALS Measurement of the quality of service Objective perceptual multimedia video quality measur
2、ement in the presence of a full reference Recommendation ITU-T J.247 Rec. ITU-T J.247 (08/2008) i Recommendation ITU-T J.247 Objective perceptual multimedia video quality measurement in the presence of a full reference Summary The term multimedia, as defined in Recommendation ITU-T J.148, is the com
3、bination of multiple forms of media such as video, audio, text, graphics, fax and telephony in the communication of information. A three stage approach has been adopted to recommending objective assessment methods for multimedia. The first two stages identify perceptual quality tools appropriate for
4、 measuring video and audio individually. The third stage will identify objective assessment methods for the combined audiovisual media. This Recommendation contains the first stage video only used in multimedia applications. Recommendation ITU-T J.247 provides guidelines on the selection of appropri
5、ate objective perceptual video quality measurement methods when a full reference signal is available. The following are example applications that can use this Recommendation: 1) Internet multimedia streaming. 2) Video telephony and conferencing over cable and other networks. 3) Progressive video tel
6、evision streams viewed on LCD monitors over cable networks, including those transmitted over the Internet using Internet Protocol (VGA was the maximum resolution in the validation test). 4) Mobile video streaming over telecommunications networks. 5) Some forms of IPTV video payloads (VGA was the max
7、imum resolution in this validation test). Source Recommendation ITU-T J.247 was approved on 13 August 2008 by ITU-T Study Group 9 (2005-2008) under Recommendation ITU-T A.8 procedures. ii Rec. ITU-T J.247 (08/2008) FOREWORD The International Telecommunication Union (ITU) is the United Nations specia
8、lized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them wi
9、th a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Rec
10、ommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for
11、conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure e.g., interoperability or applicability) and compliance with the Recomm
12、endation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any par
13、ty. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property R
14、ights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers
15、 are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2010 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permi
16、ssion of ITU. Rec. ITU-T J.247 (08/2008) iii CONTENTS Page 1 Scope 1 1.1 Application . 2 1.2 Model usage 2 1.3 Limitations 3 2 References. 4 2.1 Normative references 4 2.2 Informative references 4 3 Definitions 5 3.1 Terms defined elsewhere 5 3.2 Terms defined in this Recommendation . 5 4 Abbreviati
17、ons and acronyms 6 5 Conventions 7 6 Description of the full reference measurement method 7 7 Findings of the Video Quality Experts Group (VQEG) . 8 Annex A NTT full reference method 11 A.1 Introduction 11 A.2 Model overview 11 A.3 Video alignment module 11 A.4 Temporal/spatial feature amount deriva
18、tion module 15 A.5 Subjective video quality estimation module . 20 Annex B Description of OPTICOMs video quality measure PEVQ . 21 B.1 Description of the OPTICOM model PEVQ 21 Annex C Psytechnics full reference method . 51 C.1 Scope 51 C.2 Introduction 51 C.3 Overview of the Psytechnics full referen
19、ce video model . 51 C.4 Video registration . 52 C.5 Detection of perceptual features . 59 C.6 Kernel for the contrast sensitivity function 74 Annex D Yonsei University full reference method 80 D.1 Introduction 80 D.2 Full reference model based on edge degradation . 80 D.3 Conclusions 92 iv Rec. ITU-
20、T J.247 (08/2008) Page Appendix I Excerpts from the synopsis from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment, phase I 93 I.1 Introduction 93 I.2 Model performance evaluation techniques . 94 I.3 FR model performance . 94 I.4 Data analysis ex
21、ecuted by ILG . 96 Appendix II Equations for model evaluation metrics . 97 Rec. ITU-T J.247 (08/2008) 1 Recommendation ITU-T J.247 Objective perceptual multimedia video quality measurement in the presence of a full reference 1 Scope This Recommendation provides guidelines and recommendations on the
22、selection of appropriate perceptual video quality measurement equipment for use in multimedia applications when the full reference measurement method can be used. The full reference measurement method can be used when the unimpaired reference video signal is readily available at the measurement poin
23、t, as may be the case of measurements on individual equipment or a chain in the laboratory or in a closed environment such as a cable television head-end. The estimation methods are based on processing video in VGA, CIF and QCIF resolution. The validation test material contained both multiple coding
24、 degradations and various transmission error conditions (e.g., bit errors, dropped packets). In the case where coding distortions are considered in the video signals, the encoder can utilize various compression methods (e.g., MPEG-2, ITU-T H.264, etc.). The models proposed in this Recommendation may
25、 be used to monitor the quality of deployed networks to ensure their operational readiness. The visual effects of the degradations may include spatial as well as temporal degradations (e.g., frame repeats, frame skips, frame rate reduction). The models in this Recommendation can also be used for lab
26、 testing of video systems. When used to compare different video systems, it is advisable to use a quantitative method (such as that in ITU-T J.149) to determine the models accuracy for that particular context. This Recommendation is deemed appropriate for telecommunications services delivered at 4 M
27、bit/s or less presented on mobile/PDA and computer desktop monitors. The following conditions were allowed in the validation test for each resolution: PDA/mobile (QCIF): 16 kbit/s to 320 kbit/s. CIF: 64 kbit/s-2 Mbit/s (C01 has several 2 Mbit/s). VGA: 128 kbit/s-4 Mbit/s (V13 has one HRC with 6 Mbit
28、/s). Table 1 Factors for which this Recommendation has been evaluated Test factors Transmission errors with packet loss Video resolution QCIF, CIF and VGA Video bit rates: QCIF: 16 kbit/s to 320 kbit/s CIF: 64 kbit/s-2 Mbit/s VGA: 128 kbit/s-4 Mbit/s Temporal errors (pausing with skipping) of maximu
29、m 2 seconds Video frame rates from 5 fps to 30 fps Coding technologies H.264/AVC (MPEG-4 part 10), VC-1, Windows Media 9, Real Video (RV 10), MPEG-4 Part 2. (see Note below). 2 Rec. ITU-T J.247 (08/2008) Table 1 Factors for which this Recommendation has been evaluated Applications Real-time, in-serv
30、ice quality monitoring at the source Remote destination quality monitoring when a copy of the source is available Quality measurement for monitoring of a storage or transmission system that utilizes video compression and decompression techniques, either a single pass or a concatenation of such techn
31、iques Lab testing of video systems NOTE The validation testing of models included video sequences encoded using 15 different video codecs. The five codecs listed above were most commonly applied to encode test sequences and any recommended models may be considered appropriate for evaluating these co
32、decs. In addition to these five codecs, a smaller proportion of test sequences were created using the following codecs: Cinepak, DivX, ITU-T H.261, ITU-T H.263, ITU-T H.263+1, JPEG-2000, MPEG-1, MPEG-2, Sorenson, ITU-T H.264 SVC, Theora. It can be noted that some of these codecs were used only for C
33、IF and QCIF resolutions because they are expected to be used in the field mostly for these resolutions. Before applying a model to sequences encoded using one of these codecs, the user should carefully examine its predictive performance to determine whether the model reaches acceptable predictive pe
34、rformance. 1.1 Application This Recommendation provides video quality estimations for video classes TV3 to MM5B, as defined in Annex B of ITU-T P.911. Note that the maximum resolution was VGA and the maximum bit rate covered well in the test was 4 Mbit/s. The applications for the estimation models d
35、escribed in this Recommendation include, but are not limited to: 1) potentially real-time, in-service quality monitoring at the source; 2) remote destination quality monitoring when a copy of the source is available; 3) quality measurement for monitoring of a storage or transmission system that util
36、izes video compression and decompression techniques, either a single pass or a concatenation of such techniques; 4) lab testing of video systems. 1.2 Model usage All four models significantly outperform PSNR. The models from OPTICOM and Psytechnics tend to perform slightly better than the NTT and Yo
37、nsei models in some resolutions. The Psytechnics and OPTICOM models usually produce statistically equivalent results. For QCIF, the model from NTT is often statistically equivalent to the models of Psytechnics and OPTICOM. For VGA, the Yonsei model is typically statistically equivalent to the OPTICO
38、M and Psytechnics models. The tables below provide an overview on the models performance. Although all four models can be used to meet the industries needs adequately, for VGA, it is strongly advised that the models from OPTICOM, Psytechnics or Yonsei be used to obtain slightly better performance in
39、 most cases. For the same reason, it is strongly advised that the models from OPTICOM or Psytechnics be used for CIF. For the same reason, it is strongly advised that the models from NTT, OPTICOM or Psytechnics be used for QCIF. _ 1ITU-T H.263+ is a particular configuration of ITU-T H.263. Rec. ITU-
40、T J.247 (08/2008) 3 The OPTICOM model shows the best overall minimum correlation. The minimum correlation coefficients of the OPTICOM model are 0.68, of the NTT model 0.60, of the Yonsei model 0.59, and of the Psytechnics model 0.57, respectively. The Psytechnics model obtained the highest number of
41、 occurrences of being in the top group. The total number of occurrences in the top group is 37 for the Psytechnics, 34 for the OPTICOM, 25 for the NTT and 24 for the Yonsei model. Model performance overview VGA NTT OPTICOM Psytechnics Yonsei PSNR Avg. correlation 0.786 0.825 0.822 0.805 0.713 Min. c
42、orrelation 0.598 0.685 0.565 0.612 0.499 Occurrences at rank 1 8 10 11 10 3 Ranking analysis Second Best Best Best CIF NTT OPTICOM Psytechnics Yonsei PSNR Avg. correlation 0.777 0.808 0.836 0.785 0.656 Min. correlation 0.675 0.695 0.769 0.712 0.440 Occurrences at rank 1 8 13 14 10 0 Ranking analysis
43、 Second Best Best Second QCIF NTT OPTICOM Psytechnics Yonsei PSNR Avg. correlation 0.819 0.841 0.830 0.756 0.662 Min. correlation 0.711 0.724 0.664 0.587 0.540 Occurrences at rank 1 9 11 12 4 1 Ranking analysis Best Best Best Second 1.3 Limitations The estimation models described in this Recommendat
44、ion cannot be used to fully replace subjective testing. Correlation values between two carefully designed and executed subjective tests (i.e., in two different laboratories) normally fall within the range 0.95 to 0.98. If this Recommendation is utilized to make video system comparisons (e.g., compar
45、ing two codecs), it is advisable to use a quantitative method (such as that in ITU-T J.149) to determine the models accuracy for that particular context. The models in this Recommendation were validated by measuring video that exhibits frame freezes up to 2 seconds. The models in this Recommendation
46、 were not validated for measuring video that has a steadily increasing delay (e.g., video which does not discard missing frames after a frame freeze). It should be noted that in case of new coding and transmission technologies producing artefacts which were not included in this evaluation, the objec
47、tive models may produce erroneous results. Here a subjective evaluation is required. 4 Rec. ITU-T J.247 (08/2008) 2 References 2.1 Normative references The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Rec
48、ommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other refe
49、rences listed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. ITU-T J.143 Recommendation ITU-T J.143 (2000), User requirements for objective perceptual video quality measurements in digital cable television. ITU-T P.910 Recommendation ITU-T P.910 (2008), Subjective video quality assessment methods for multimedia applications. ITU-T P.91
