1、 TECHNICAL REPORT T1.TR.73-2001 Technical Report on Video Normalization Methods Applicable to Objective Video Quality Metrics Utilizing a Full Reference Technique Prepared by T1A1.1 Working Group on Multimedia Communications Coding and Performance Problem Solvers to the Telecommunications Industry A
2、 Word from ATIS and Committee T1 Established in February 1984, Committee T1 develops technical standards, reports and requirements regarding interoperability of telecommunications networks at interfaces with end-user systems, carriers, information and enhanced-service providers, and customer premise
3、s equipment (CPE). Committee T1 is sponsored by ATIS and is accredited by ANSI. T1.TR.73-2001 Published by Alliance for Telecommunications Industry Solutions 1200 G Street, NW, Suite 500 Washington, DC 20005 Committee T1 is sponsored by the Alliance for Telecommunications Industry Solutions (ATIS) a
4、nd accredited by the American National Standards Institute (ANSI). Copyright 2002 by Alliance for Telecommunications Industry Solutions All rights reserved. No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission
5、 of the publisher. For information contact ATIS at 202.628.6380. ATIS is online at . Printed in the United States of America. T1.TR.73-2001 Technical Report Video Normalization Methods Applicable to Objective Video Quality Metrics Utilizing a Full Reference Technique Alliance for Telecommunications
6、Industry Solutions Approved October 2001 Abstract This Technical Report has been developed as one of a group of Technical Reports designed to meet the industry need for documentation of video quality metrics utilizing the Full Reference technique. Most full reference methods require that the degrade
7、d video be normalized in gain and spatial/temporal parameters prior to calculating the video quality algorithm. This Technical Report describes the reasons for including normalization in video quality metrics and methods that may be used for normalization. Accuracy and cross calibration of video qua
8、lity metrics are defined in T1.TR.72-2001. T1.TR.73-2001 ii Foreword As part of the industry-wide effort to develop objective video quality of service measurements three methodological approaches have been defined. Full Reference (FR) - A method applicable when the full reference video signal is ava
9、ilable. This is a double-ended method and is the subject of this technical report. Reduced Reference (RR) - A method applicable when only reduced video reference information is available. This is also a double-ended method. No Reference (NR) - A method applicable when no reference video signal or in
10、formation is available. This is a single-ended method. This Technical Report has been developed as one of a group of Technical Reports designed to meet the industry need for documentation of objective video quality metrics (VQM) utilizing the Full Reference technique. Normalization means that time-i
11、nvariant systematic changes in the video from reference input to processed video output are removed prior to performing full reference video quality measurements. Typical parameters to be normalized are gain, black level and spatial alignment. Gain and black level may affect perceived picture qualit
12、y while spatial alignment, within reason, will not. However, spatial alignment is needed for most full reference objective methods since the calculation is done by comparison on a pixel-by-pixel basis. All objective methods require temporal alignment. The forward of this Technical Report describes t
13、he background of including normalization in objective full reference video quality metrics. Specific methods that may be used for normalization are in the normative body of the document. Accuracy and cross calibration of video quality metrics are defined in T1.TR.72-2001. While objective measurement
14、s with good correlation to subjective quality assessment are desirable in order to attain optimal quality of service, it must be realized that objective measurements are not a direct replacement for subjective quality assessment. Subjective quality assessments are carefully designed procedures inten
15、ded to determine the average opinion of human viewers to a specific set of video sequences for a given application. Results of such tests are valuable in basic system design and benchmark evaluations. Subjective quality assessments for a different application with different test conditions will stil
16、l provide meaningful results. However, opinion scores for the same set of video sequences are likely to have different values. Objective measurements are intended for use in a broad set of applications producing the same results with a given set of video sequences. The choice of video sequences to u
17、se and the interpretation of the resulting objective measurements are some of the factors varied for a specific application. Therefore, objective measurements and subjective quality assessment are complementary rather than interchangeable. Where subjective assessment is appropriate for research rela
18、ted purposes, objective measurements are required for equipment specifications and day-to-day system performance measurement and monitoring. Normalization used as part of the calculation in a VQM should have meaningful correspondence with both related subjective assessments and operational realities
19、 in a television system. The most fundamental issue is whether normalization (other than temporal) should be used at all. To some extent, this is a function of the capabilities of the VQM. A VQM that models the human visual system and response may require a less complete set of parameters to be norm
20、alized or perhaps require no normalization. Others, such as peak signal to noise ratio (PSNR), may require very precise and complete normalization in order to provide meaningful results. At the time this Technical Report was developed there were no subjective assessment results known to provide view
21、er opinion scores based on including gain and black level changes in the displayed pictures. Although some of the perceptual based objective video quality metrics output values might be responsive to gain and black level changes there has been no independent validation of such a capability. Therefor
22、e, normalization of processed video is considered to be an important aspect of objective video quality metrics. Subjective Assessments For subjective video quality assessment, some parameters relating to visual presentation are carefully specified and reported. (See Recommendation ITU-R Recommendati
23、on BT.500.) The parameters related to video display monitor characteristics and viewing conditions and are considered part of the experimental design. While the parameter values are necessary for evaluation of the subjective assessment results, they cannot be mathematically included in the presentat
24、ion of the subjective assessment scores. Video gain and black level changes between the reference video and the processed video are not controlled or reported in subjective assessment experiments defined by ITU-R Recommendation BT.500. However, it is well known that they may have a significant affec
25、t on perceived picture T1.TR.73-2001 iii quality and that these parameters are usually well controlled in broadcast television operations. One argument for controlling (normalizing) gain and black level in the processed video relates to the use of subjective testing for evaluation of compression cod
26、ecs. The question is, will the system be fairly evaluated if such adjustments are, or are not, made? Because adjusting gain and black level is always considered good practice in an operational environment and the mechanism for such adjustments is virtually always available (even in all digital syste
27、ms), it would be unfair to evaluate a system with incorrect gain and black level. Consider two similar systems with different picture quality. An engineer is going to decide which one to purchase. The one with the better picture quality has modest gain and black level errors. These errors are of suc
28、h a nature that the subjective assessment results will reverse the true ranking of picture quality for the two systems. Clearly, it would be better to properly adjust the gain and black level of all processed sequences or request that a manufacturer fix a design problem in order to make the best pur
29、chase decision. With this in mind, the following flowchart has been suggested to the Video Quality Experts Group (VQEG) for use with the double stimulus continuous quality scale (DSCQS) assessment method of ITU-R Recommendation BT.500. It requires maintaining gain and black level (offset) to within
30、2% of full amplitude throughout the sequence processing for subjective assessment. 2% accuracy will have little or no affect on subjective results while an improved accuracy may be required for some video quality metrics as described below in the discussion of PSNR. HRCSource Video (reference)color
31、bars on each sequenceProcessedVideoOpinionScoresObjectiveRatingsObjectiveRatingsN1Gain and Offset2% accuracy(Good TV Practice)Tape EditingandDistributionSubjectiveAssessmentObjectiveModelwith normalizationObjectiveModeloptionalnormalizationN2Spatial alignmentsub-pixel accuracyand DistributionTape Re
32、cordingvideo/dataSubjective viewing tapeswith color bars on leaderObjective tapes withcolor bars on each sequenceNormalized objective tapes withcolor bars on each sequence1. Each source sequence shall have associated color bars that accurately represent the correct amplitude and offset (black level)
33、 of the sequence. 2. At the time, the sequence is processed through the hypothetical reference circuit (HRC) the gain and offset shall be adjusted to an accuracy of 2% of full amplitude either by adjustment within the HRC or an external processing amplifier. (N1) 3. When the tape recording of the pr
34、ocessed sequence is made, it shall be replayed to verify the accuracy of the gain and offset. 4. Distribution copies from the tape editing shall be replayed prior to distribution to verify the accuracy of the gain and offset. 5. The color bars on the leader of the subjective tapes shall be used to a
35、djust the gain and offset of the viewing monitors to provide ITU-R Recommendation BT.500 viewing conditions. T1.TR.73-2001 iv 6. If resources are available for normalization and distribution of all sequences (N2), normalization of gain, offset and spatial alignment shall be implemented by an agreed
36、method and to an agreed accuracy. 7. PSNR shall be calculated based on normalization of gain, offset and spatial alignment implemented by an agreed method and to an agreed accuracy. 8. Proponents may use the color bars as an aid to the normalization for their model if desired. Peak Signal to Noise R
37、atio For many years PSNR has been a benchmark VQM. (See T1.801.03-1996.) This objective method requires rather precise normalization in order to give meaningful results. Shown below are values for single-frame luminance PSNR without normalization calculated based on a 127 by 127 pixel picture. Sampl
38、es were in the range of valid values per Recommendation ITU-R Recommendation BT.601 for 8-bit operation. Distortion Diagonal Ramp Diagonal Sine waves 5 cycles Diagonal Sine waves 10 cycles Black level shift 1.4% 37.8 dB 37.8 dB 37.8 dB Gain change 1.8 % 38.9 dB 37.9 dB 37.8 dB H picture shift 0.5 pi
39、xel 48.1 dB 33.8 dB 27.8 dB H picture shift 1 pixel 48.1 dB 27.8 dB 21.8 dB Similar PSNR values are obtained for gain change and black level shift for all picture types as would be expected since they represent similar changes to all pixel values. The equal values for the 1 and 0.5 pixel picture shi
40、fts for the ramp are due to the rounding method to keep the digital values as integers. PSNR reduction due to picture shift is greater in the picture with more detail, that is 10 cycles of sine wave. To put these values in perspective, the VQEG Phase 1 measurements gave PSNR values in the range of 2
41、0 dB to 45 dB for a wide variety of video sequences and test systems. (See ITU-T Com 9-80, June 2000.) The modest gain and black level changes are well within the range of good operating practice and may not provide visible changes in a subjective assessment. However they produce PSNR values are not
42、 indicative of nearly perfect pictures. Small amounts of horizontal shift would produce no visual affect however, if not corrected, they will make PSNR completely meaningless. Operational Practice In traditional broadcast operations, it is relatively common to monitor and adjust gain and black level
43、 using available test equipment and vertical interval test signals. The 2% accuracy suggested for subjective assessments is reasonable. Linear systems, analog or digital, tend to maintain those parameters as a constant regardless of picture content so care taken with each segment of a program will i
44、nsure consistent operation for the entire program. As broadcast and other operations using full resolution standard definition television (SDTV) incorporate compression systems both live and pre-recorded, the ease of maintaining gain and black level is reduced specifically due to the lack of meaning
45、ful vertical interval test signals. There is also the possibility that these parameters will change with picture content in a compression system. However, that is more of a design problem to be eliminated than an operational problem to be dealt with in an ongoing manner. There is still the need to m
46、aintain gain and black level in order to provide the viewer with acceptable results. How well this is accomplished, including how often adjustments are monitored and corrected, provides some operational requirements for a VQM with respect to normalization. Another operational consideration is in-ser
47、vice versus out-of-service measurements. Without the availability of vertical interval test signals, compressions system testing is more inclined toward out-of-service (intrusive) tests using known sequences of varying complexity. In this case gain and black level adjustment on a sequence-by-sequenc
48、e basis is easily accomplished and relates to normalization by the VQM. While this may be the most obvious use of a full reference VQM, the possibility for continuous in-service monitoring exists in many applications. Any normalization of gain and black level used by the VQM should appropriately rep
49、resent the operational aspects of the system. That is the VQM must separate the static, time invariant parameters, that are appropriate for normalization from those that should be included in the picture quality output results of the measurement system. T1.TR.73-2001 v Normalization Based on the above discussion it is apparent that there is no one set of parameters to be normalized for a VQM nor is there one specific normalization method to be used. There are two dimensions to be considered, use of artificial test signals related to the video sequence and length of the video sequence.
