1、 Recommendation ITU-R BT.1867(03/2010)Objective perceptual visual quality measurement techniques for broadcasting applications using low definition television in the presence of a reducedbandwidth referenceBT SeriesBroadcasting service(television)ii Rec. ITU-R BT.1867 Foreword The role of the Radioc
2、ommunication Sector is to ensure the rational, equitable, efficient and economical use of the radio-frequency spectrum by all radiocommunication services, including satellite services, and carry out studies without limit of frequency range on the basis of which Recommendations are adopted. The regul
3、atory and policy functions of the Radiocommunication Sector are performed by World and Regional Radiocommunication Conferences and Radiocommunication Assemblies supported by Study Groups. Policy on Intellectual Property Right (IPR) ITU-R policy on IPR is described in the Common Patent Policy for ITU
4、-T/ITU-R/ISO/IEC referenced in Annex 1 of Resolution ITU-R 1. Forms to be used for the submission of patent statements and licensing declarations by patent holders are available from http:/www.itu.int/ITU-R/go/patents/en where the Guidelines for Implementation of the Common Patent Policy for ITU-T/I
5、TU-R/ISO/IEC and the ITU-R patent information database can also be found. Series of ITU-R Recommendations (Also available online at http:/www.itu.int/publ/R-REC/en) Series Title BO Satellite delivery BR Recording for production, archival and play-out; film for television BS Broadcasting service (sou
6、nd) BT Broadcasting service (television) F Fixed service M Mobile, radiodetermination, amateur and related satellite services P Radiowave propagation RA Radio astronomy RS Remote sensing systems S Fixed-satellite service SA Space applications and meteorology SF Frequency sharing and coordination bet
7、ween fixed-satellite and fixed service systems SM Spectrum management SNG Satellite news gathering TF Time signals and frequency standards emissions V Vocabulary and related subjects Note: This ITU-R Recommendation was approved in English under the procedure detailed in Resolution ITU-R 1. Electroni
8、c Publication Geneva, 2010 ITU 2010 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without written permission of ITU. Rec. ITU-R BT.1867 1 RECOMMENDATION ITU-R BT.1867 Objective perceptual visual quality measurement techniques for broadcasting applicatio
9、ns using low definition television*in the presence of a reduced bandwidth reference*(2010) Scope This Recommendation specifies methods for estimating the perceived video quality of broadcasting applications using low definition television (LDTV) when a reduced reference (RR) signal can be made avail
10、able, e.g. through an ancillary data channel, watermark, metadata, and so on. The ITU Radiocommunication Assembly, considering a) that the ability to automatically measure the quality of broadcast video has long been recognized as a valuable asset to the industry; b) that Recommendation ITU-R BT.168
11、3 describes objective methods for measuring the perceived video quality of standard definition digital broadcast television in the presence of a full reference; c) that Recommendation ITU-R BT.1833 describes multimedia systems for broadcasting of multimedia and data applications for mobile reception
12、 by handheld receivers; d) that low definition television (LDTV) is becoming widely used in the broadcasting of multimedia and data applications for mobile reception; e) that ITU-T Recommendation J.2461specifies objective measurement techniques of perceptual video quality applicable to LDTV applicat
13、ions in the presence of a reduced reference; f) that objective measurement of perceived video quality may usefully complement subjective assessment methods, recognizing a) that the use of LDTV is mainly intended for viewing on small screens, such as those available on handheld and mobile receivers,
14、*Low definition television (LDTV) refers to video resolutions having less number of pixels than the ones defined in Recommendation ITU-R BT.601. A pertinent ITU-R Recommendation on LDTV is under consideration. *The measurement method with reduced reference, for objective measurement of perceptual vi
15、deo quality, evaluates the performance of systems by making a comparison between features extracted from the undistorted input, or reference, video signal at the input of the system, and the degraded signal at the output of the system. 1ITU-T Recommendation J.246 is available at . 2 Rec. ITU-R BT.18
16、67 recommends 1 that the guidelines, scope, and limitations given in Annex 1 should be used in the application of the objective video quality measurement models identified in recommends 2; 2 that the objective perceptual video quality measurement model given in Annex 2 should be used for broadcastin
17、g applications using LDTV when a reduced reference signal, as described in Annex 2, is available. Annex 1 1 Introduction This Recommendation specifies methods for estimating the perceived video quality of broadcasting applications using LDTV when a reduced reference signal is available. The reduced
18、reference measurement method can be used when the features extracted from the reference video signal is readily available at the measurement point, as may be the case of measurements on individual equipment or a chain in the laboratory or in a closed environment. The estimation methods are based on
19、processing video in VGA, CIF, and QCIF resolution. The validation test material contained both multiple coding degradations and various transmission error conditions (e.g. bit errors, dropped packets). In the case where coding distortions are considered in the video signals, the encoder can utilize
20、various compression methods (e.g. MPEG-2, H.264, etc.). The models in this Recommendation may be used to monitor the quality of deployed networks to ensure their operational readiness. The visual effects of the degradations may include spatial as well as temporal degradations (e.g. frame repeats, fr
21、ame skips, frame rate reduction). The models in this Recommendation can also be used for lab testing of video systems. When used to compare different video systems, it is advisable to use a quantitative method (such as that in ITU-T Recommendation J.149) to determine the models accuracy for that par
22、ticular context. This Recommendation is deemed appropriate for services delivered at 4 Mbit/s or less presented on mobile receivers. The following conditions were allowed in the validation test for each resolution: QCIF (quarter common intermediate format (176 144 pixels): 16 kbit/s to 320 kbit/s. C
23、IF (common intermediate format (352 288 pixels): 64 kbit/s 2 Mbit/s. VGA (video graphics array (640 480 pixels): 128 kbit/s 6 Mbit/s. TABLE 1 Factors used in the evaluation of models Test factors Transmission errors with packet loss Video resolution QCIF, CIF and VGA Video bitrates QCIF: 16 kbit/s t
24、o 320 kbit/s CIF: 64 kbit/s 2 Mbit/s VGA: 128 kbit/s 4 Mbit/s Temporal errors (pausing with skipping) of maximum 2 s Rec. ITU-R BT.1867 3 TABLE 1 (end) Test factors Video frame rates from 5 fps to 30 fps Coding schemes H.264/AVC (MPEG-4 Part 10), MPEG-4 Part 2, and three other proprietary coding sch
25、emes. (See Note 1.) Applications Real-time, in-service quality monitoring at the source Remote destination quality monitoring when side-channels are available for features extracted from source video sequences Quality measurement for monitoring of a storage or transmission system that utilizes video
26、 compression and decompression techniques, either a single pass or a concatenation of such techniques Lab testing of video systems NOTE 1 The validation testing of models included video sequences encoded using 15 different video codecs. The five codecs listed in Table 1 were most commonly applied to
27、 encode test sequences and any recommended models may be considered appropriate for evaluating these codecs. In addition to these five codecs a smaller proportion of test sequences were created using the following codecs: H.261, H.263, H.263+2, JPEG-2000, MPEG-1, MPEG-2, H.264 SVC, and other proprie
28、tary systems. It can be noted that some of these codecs were used only for CIF and QCIF resolutions because they are expected to be used in the field mostly for these resolutions. Before applying a model to sequences encoded using one of these codecs the user should carefully examine its predictive
29、performance to determine whether the model reaches acceptable predictive performance. 2 Application The applications for the estimation models described in this Recommendation include, but are not limited to: 1 codec evaluation, specification, and acceptance testing, consistent with the limited accu
30、racy as described below; 2 real-time, in-service quality monitoring; 3 remote destination quality monitoring when side channels are available for features extracted from source video sequences; 4 quality measurement for monitoring of a storage or transmission system that utilizes video compression a
31、nd decompression techniques, either a single pass or a concatenation of such techniques; 5 lab testing of video systems. 3 Limitations The estimation models described in this Recommendation cannot be used to replace subjective testing. Correlation values between two carefully designed and executed s
32、ubjective tests (i.e. in two different laboratories) normally fall within the range 0.95 to 0.98. If this Recommendation is 2H.263+ is a particular configuration of H.263 (1998). 4 Rec. ITU-R BT.1867 utilized to make video system comparisons (e.g. comparing two codecs), it is advisable to use a quan
33、titative method (such as that in ITU-T Recommendation J.149) to determine the models accuracy for that particular context. The models in this Recommendation were validated by measuring video that exhibits frame freezes up to 2 s. The models in this Recommendation were not validated for measuring vid
34、eo that has a steadily increasing delay (e.g. video which does not discard missing frames after a frame freeze). It should be noted that in case of new coding and transmission technologies producing artefacts which were not included in this evaluation, the objective models may produce erroneous resu
35、lts. Here a subjective evaluation is required. 4 Model descriptions The following models are described in Annex 2: Model A (Annex 2) VQEG Proponent Yonsei University, Korea (Republic of). Appendix 1 to Annex 1 Findings of the Video Quality Experts Group (VQEG) Studies of perceptual video quality mea
36、surements are conducted in an informal group, called VQEG, which reports to ITU-T Study Groups 9 and 12 and Radiocommunication Study Group 6. The recently completed Multimedia Phase I test of VQEG assessed the performance of proposed reduced reference perceptual video quality measurement algorithms
37、for QCIF, CIF, and VGA formats. Based on present evidence, the following method can be recommended by ITU-R at this time: Model A (Annex 2) VQEG Proponent Yonsei University, Korea (Republic of). Tables 2, 3 and 4 provide informative details on the models performances in the VQEG Multimedia Phase I t
38、est. TABLE 2 VGA resolution: Informative description on the models performances in the VQEG Multimedia Phase I test: Averages over 13 subjective tests Statistic Yonsei RR10k Yonsei RR64k Yonsei RR128k PSNR(1)Correlation 0.803 0.803 0.803 0.713RMSE(2)0.599 0.599 0.598 0.714 Outlier ratio 0.556 0.553
39、0.552 0.615 (1)PSNR: peak signal-to-noise ratio. (2)RMSE: root mean square error. Rec. ITU-R BT.1867 5 TABLE 3 CIF resolution: Informative description on the models performances in the VQEG Multimedia Phase I test: Averages over 14 subjective tests Statistic Yonsei RR10k Yonsei RR64k PSNR Correlatio
40、n 0.780 0.782 0.656RMSE 0.593 0.590 0.720Outlier ratio 0.519 0.511 0.632 TABLE 4 QCIF resolution: Informative description on the models performances in the VQEG Multimedia Phase I test: Averages over 14 subjective tests Statistic Yonsei RR1k Yonsei RR10k PSNR Correlation 0.771 0.791 0. 662RMSE 0.604
41、 0.578 0.721Outlier ratio 0.505 0.486 0.596 The average correlations of the primary analysis for the RR VGA models were all 0.80, and PSNR was 0.71. Individual model correlations for some experiments were as high as 0.93. The average RMSE for the RR VGA models were all 0.60, and PSNR was 0.71. The a
42、verage outlier ratio for the RR VGA models ranged from 0.55 to 0.56, and PSNR was 0.62. All proposed models performed statistically better than PSNR for 7 of the 13 experiments. Based on each metric, each RR VGA model was in the group of top performing models the following number of times: Statistic
43、 Yonsei RR10k Yonsei RR64k Yonsei RR128k PSNR Correlation 13 13 13 7 RMSE 13 13 13 6 Outlier ratio 13 13 13 10 The average correlations of the primary analysis for the RR CIF models were 0.78, and PSNR was 0.66. Individual model correlations for some experiments were as high as 0.90. The average RMS
44、E for the RR CIF models were all 0.59, and PSNR was 0.72. The average outlier ratio for the RR CIF models were 0.51 and 0.52, and PSNR was 0.63. All proposed models performed statistically better than PSNR for 10 of the 14 experiments. Based on each metric, each RR CIF model was in the group of top
45、performing models the following number of times: Statistic Yonsei RR 10k Yonsei RR64k PSNR Correlation 14 14 5 RMSE 14 14 4 Outlier ratio 14 14 5 The average correlations of the primary analysis for the RR QCIF models were 0.77 and 0.79, and PSNR was 0.66. Individual model correlations for some expe
46、riments were as high as 0.89. 6 Rec. ITU-R BT.1867 The average RMSE for the RR QCIF models were 0.58 and 0.60, and PSNR was 0.72. The average outlier ratio for the RR QCIF models were 0.49 and 0.51, and PSNR was 0.60. All proposed models performed statistically better than PSNR for at least 9 of the
47、 14 experiments. Based on each metric, each RR QCIF model was in the group of top performing models the following number of times: Statistic Yonsei RR1k Yonsei RR10k PSNR Correlation 14 14 5 RMSE 14 14 4 Outlier ratio 12 13 4 Annex 2 Model A reduced reference methods*TABLE OF CONTENTS Page 1 Introdu
48、ction 7 2 The EPSNR reduced-reference models 7 2.1 Edge PSNR . 7 2.2 Selecting features from source video sequences 12 2.3 Spatial/temporal registration and gain/offset adjustment . 15 2.4 Computing EPSNR and post-processing 18 2.5 Optimal bandwidth of side channel 19 Appendix 20 *This model is iden
49、tical to that specified in Annex A of ITU-T Recommendation J.246. Rec. ITU-R BT.1867 7 1 Introduction Although PSNR has been widely used as an objective video quality measure, it is also reported that it does not well represent perceptual video quality. By analysing how humans perceive video quality, it is observed that the human visual system is sensitive to degradation around the edges. In other words, when the edge pixels of a video are blurred, evaluators tend to give low scores to the video, even though the PSNR is high. Based on th
