1、 International Telecommunication Union ITU-T J.249TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (01/2010) SERIES J: CABLE NETWORKS AND TRANSMISSION OF TELEVISION, SOUND PROGRAMME AND OTHER MULTIMEDIA SIGNALS Measurement of the quality of service Perceptual video quality measurement techniques for
2、digital cable television in the presence of a reduced reference Recommendation ITU-T J.249 Rec. ITU-T J.249 (01/2010) i Recommendation ITU-T J.249 Perceptual video quality measurement techniques for digital cable television in the presence of a reduced reference Summary Recommendation ITU-T J.249 pr
3、ovides guidelines on the selection of appropriate objective perceptual video quality measurement methods when a reduced reference signal is available. The following are example applications that can use this Recommendation: 1) Interlaced video television streams over cable networks including those t
4、ransmitted over the Internet using Internet Protocol. 2) Some forms of Internet Protocol Television (IPTV) video payloads. 3) Video quality monitoring at the receiver when side-channels are available. 4) Video quality monitoring at measurement nodes located between the point of transmission and the
5、point of reception. History Edition Recommendation Approval Study Group 1.0 ITU-T J.249 2010-01-13 9 ii Rec. ITU-T J.249 (01/2010) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication techn
6、ologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Teleco
7、mmunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some are
8、as of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized ope
9、rating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shal
10、l“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the pra
11、ctice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation devel
12、opment process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore
13、strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2010 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T J.249 (01/2010) iii CONTENTS Page 1 Scope 1 1.1 Application
14、s 1 1.2 Model usage 1 1.3 Limitations 2 2 References. 3 3 Definitions 3 3.1 Terms defined elsewhere 3 3.2 Terms defined in this Recommendation . 3 4 Abbreviations and acronyms 4 5 Conventions 5 6 Description of the reduced reference measurement method . 5 7 Findings of the Video Quality Experts Grou
15、p (VQEG) . 6 Annex A Model_A: Yonsei University, HDSP Laboratory (15 kbit/s model, 80 kbit/s model, 256 kbit/s model) 9 A.1 Introduction 9 A.2 The EPSNR reduced reference models 9 A.3 Conclusions 21 Annex B Model_B: NEC (80 kbit/s, 256 kbit/s) 525-line format only 22 B.1 Summary . 22 B.2 Definitions
16、 22 B.3 Overview of the video quality metric (VQM) computation . 23 B.4 Detailed algorithm 24 B.5 Analysis of the results 29 B.6 Sample codes 29 B.7 Informative reference . 35 Annex C Model_C: NTIA (80 kbit/s model) 36 C.1 Background . 36 C.2 Introduction 36 C.3 Fast low bandwidth VQM description .
17、37 C.4 Informative references 44 C.5 Reference code for implementing the fast low bandwidth VQM . 44 Appendix I Transmission error analyses for the RRNR-TV validation tests . 161 Bibliography. 162 Rec. ITU-T J.249 (01/2010) 1 Recommendation ITU-T J.249 Perceptual video quality measurement techniques
18、 for digital cable television in the presence of a reduced reference 1 Scope This RRNR-TV (reduced reference and no reference objective models for standard definition television) test addresses two video formats (525-line and 625-line). Reduced reference (RR) models have limited bandwidth access to
19、the source video. For an RR model to operate correctly, the unimpaired source video should be available for the models to extract parameters. The hypothetical reference circuits (HRCs) in each experiment spanned both coding-only artefacts and coding with transmission errors. The coding schemes exami
20、ned were MPEG-2 and ITU-T H.264 (MPEG-4 part 10). The MPEG-2 encoders were run at a variety of bit rates from 1.0 to 5.5 Mbit/s. The ITU-T H.264 encoders were run at a variety of bit rates ranging from 1.0 to 3.98 Mbit/s. Each experiment included 12 source sequences, of which two were secret source.
21、 Each experiment included 34 HRCs, and 156 processed video sequences (PVSs). Of these PVSs, 40 contained transmission errors and 116 contained coding only. Although the models defined in this Recommendation may be used to compare video system performance (e.g., comparing two codecs), readers should
22、be aware that the models contained in this Recommendation were not validated for such applications. None of the evaluated models reached the accuracy of the normative subjective testing. 1.1 Applications This Recommendation provides video quality estimations for video classes TV 3 to MM 5b, as defin
23、ed in Annex B of ITU-T P.911. The applications for the estimation models described in this Recommendation include, but are not limited to: 1) potentially real-time, in-service quality monitoring at the source; 2) remote destination quality monitoring when side-channels are available for features ext
24、racted from source video sequences; 3) quality measurement for monitoring of a storage or transmission system that utilizes video compression and decompression techniques, either a single pass or a concatenation of such techniques; 4) lab testing of video systems. 1.2 Model usage This Recommendation
25、 includes the objective computational models shown in Table 1. Table 1 Model description Model index Name Country Annex Model_A Yonsei University Korea (Republic of) A Model_B NEC Japan B Model_C National Telecommunications and Information Administration/Institute for Telecommunication Sciences (NTI
26、A/ITS) United States of America C 2 Rec. ITU-T J.249 (01/2010) For the 525-line format, all the six models (Model_A 15k, Model_A 80k, Model_A 256k, Model_C 80k, Model_B 80k, Model_B 256k ) significantly outperform or are equivalent to peak signal-to-noise ratio (PSNR). These six models can be used f
27、or the 525-line format. For the 625-line format, four models (Model_A 15k, Model_A 80k, Model_A 256k, Model_C 80k) significantly outperform PSNR. These four models can be used for the 625-line format. Table 2 shows the recommended formats for which the models can be used. Table 3 shows the validated
28、 performance (reported as correlations) for each model. Table 2 Model usage Side-channel bandwidth 525-line format 625-line format 15 kbit/s Model_A Model_A 80 kbit/s Model_A, Model_C, Model_B Model_A, Model_C 256 kbit/s Model_A, Model_B Model_A Table 3 Model performance overview (correlations) Mode
29、l_A 15k Model_A 80k Model_A 256k Model_B 80k Model_B 256k Model_C 80k PSNR 525-line format 0.906 0.903 0.903 0.795 0.803 0.882 0.826 625-line format 0.894 0.899 0.898 NA NA 0.866 0.857 1.3 Limitations The estimation models described in this Recommendation cannot be used to fully replace subjective t
30、esting. Correlation values between two carefully designed and executed subjective tests (i.e., in two different laboratories) normally fall within the range 0.93 to 0.98. If this Recommendation is utilized to make video system comparisons (e.g., comparing two codecs), it is advisable to use a quanti
31、tative method (such as that in b-ITU-T J.149) to determine the models accuracy for that particular context. The models in this Recommendation were validated by measuring video that exhibits frame freezes of up to 2 seconds. The models were evaluated assuming that the first second and final second of
32、 each 8-second video sequence maintained temporal alignment (with 0.25-second margin). Users of this Recommendation should make sure that there are no temporal alignment problems at the beginning or end of the processed video sequence (PVS). The reduced reference information can not have any transmi
33、ssion impairments (i.e., the bottom feature extraction path shown in Figures 1 and 2). The models in this Recommendation were not validated for measuring video that has a steadily increasing delay (e.g., video which does not discard missing frames after a frame freeze). For detailed test conditions
34、and calibration limits under which the models were validated, users are referred to the RRNR-TV test plan published by VQEG. It should be noted that in case of new coding and transmission technologies producing artefacts which were not included in this evaluation, the objective models may produce er
35、roneous results. Here, a subjective evaluation is required. Rec. ITU-T J.249 (01/2010) 3 2 References The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editi
36、ons indicated were valid. All Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid
37、 ITU-T Recommendations is regularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. ITU-T J.143 Recommendation ITU-T J.143 (2000), User requirements for objective perceptual video quality measurements in d
38、igital cable television. ITU-T J.144 Recommendation ITU-T J.144 (2004), Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. ITU-T J.244 Recommendation ITU-T J.244 (2008), Full reference and reduced reference calibration methods
39、for video transmission systems with constant misalignment of spatial and temporal domains with constant gain and offset. ITU-T P.910 Recommendation ITU-T P.910 (2008), Subjective video quality assessment methods for multimedia applications. ITU-T P.911 Recommendation ITU-T P.911 (1998), Subjective a
40、udiovisual quality assessment methods for multimedia applications. ITU-R BT.500-11 Recommendation ITU-R BT.500-11 (2002), Methodology for the subjective assessment of the quality of television pictures. 3 Definitions 3.1 Terms defined elsewhere This Recommendation uses the following terms defined el
41、sewhere: 3.1.1 objective perceptual measurement (picture) ITU-T J.144: The measurement of the performance of a programme chain by the use of programme-like pictures and objective (instrumental) measurement methods to obtain an indication that approximates the rating that would be obtained from a sub
42、jective assessment test. 3.1.2 proponent ITU-T J.144: An organization or company that proposes a video quality model for validation testing and possible inclusion in an ITU Recommendation. 3.1.3 subjective assessment (picture) ITU-T J.144: The determination of the quality or impairment of programme-
43、like pictures presented to a panel of human assessors in viewing sessions. 3.2 Terms defined in this Recommendation This Recommendation defines the following terms: 3.2.1 frame rate: The number of unique frames (i.e., total frames repeated frames) per second. 3.2.2 simulated transmission errors: Err
44、ors imposed upon the digital video bit stream in a highly controlled environment. Examples include simulated packet loss rates and simulated bit errors. Parameters used to control simulated transmission errors are well defined. 3.2.3 transmission errors: Any error imposed on the video transmission.
45、Example types of errors include simulated transmission errors and live network conditions. 4 Rec. ITU-T J.249 (01/2010) 4 Abbreviations and acronyms This Recommendation uses the following abbreviations and acronyms: ACR Absolute Category Rating ACR-HR Absolute Category Rating with Hidden Reference A
46、TI Absolute Temporal Information AVI Audio Video Interleave DMOS Difference Mean Opinion Score EPSNR Edge Peak Signal-to-Noise Ratio FR Full Reference FRTV Full Reference TeleVision HRC Hypothetical Reference Circuit HV Horizontal and Vertical ILG VQEGs Independent Laboratory Group LPF Low Pass Filt
47、er MAD Mean Absolute Difference MB Macro-Block MM 5b MultiMedia video class 5b MOS Mean Opinion Score MOSp Mean Opinion Score, predicted MPEG Moving Picture Experts Group NFD Normalized Frame Difference NHFE Normalized High Frequency Energy NR No (or zero) Reference OR Outlier Ratio PSNR Peak Signal
48、-to-Noise Ratio PVS Processed Video Sequence RMSE Root Mean Square Error RR Reduced Reference RRNR-TV Reduced Reference and No Reference objective models for standard definition TeleVision SFR Source Frame Rate SI Spatial Information SNFD Source Normalized Frame Difference SNMPE Source Normalized Hi
49、gh Frequency Energy SRC Source Reference Channel or Circuit TV 3 TeleVision class 3 VQEG Video Quality Experts Group Rec. ITU-T J.249 (01/2010) 5 VQM Video Quality Metric 5 Conventions None. 6 Description of the reduced reference measurement method The double-ended measurement method with reduced reference, for objective measurement of perceptual video quality, evaluates the performance of systems by making a comparison between features extracted from the undistorted input, or reference, video signal at the input of the sy
