1、Rep. ITU-R BT.2017 REPORT ITU-R BT.2017 STEREOSCOPIC TELEVISION WEG-2 MULTI-VIEW PROFILE 1 (1998) 1 Overview of MPEG-2 multi-view profile (MVP) The extension of the MPEG-2 video standard (ITU-T Recommendation H.262 I ISO/CEI 13818-2: Information technol- ogy - Generic coding of moving pictures and a
2、ssociated audio information: Video) for multi-view applications (e.g. used for stereoscopic video) has been promoted to a final International Standard at the ISO/IEC JTC 1/SC 29/WG 11 meeting in September 1996 (Amendment 3, WG 11 N1366) the multi-view profile (MVP) is envisioned to be a profile appr
3、opriate for applications that require multiple viewpoints within the context the MPEG-2 video standard. MVP supports stereoscopic pictures as its source images for a wide range of picture resolution and quality as requested by the applications to be used. 1.1 Coding scheme for MVP A block diagram of
4、 the codec reference model for the MVP is shown in Fig. 1. Its main features are a monoscopic coding in its base layer for compatibility and a hybrid prediction of motion and disparity for compression efficiency. Temporal scalability tools are used for coding an enhancement layer. FIGURE 1 The codec
5、 reference model for the MVP Motion and disparity compensated DCT encoder Motion and disparity compensated DCT decoder DCT : discret cosine transform Rap 2Ol7-01 A configuration of prediction modes are shown in Fig. 2. A monoscopic coding with the same tools as main profile (MP), including the ISO/I
6、EC 11172-2 Standard is applied to the base layer. A base layer of MVP is assigned to a left view and an enhancement layer is assigned to a right view. An enhancement layer is coded using temporal scalability tools and a hybrid prediction of motions and disparity can be utilized in the enhanced layer
7、. It foresees higher compression of the right view of stereoscopic video by exploiting the similarity between the left and right views. MVP, one of the scalable profiles in terms of multiple viewpoint layers, has the same type of compatibility features; other scalable profiles have such compatibilit
8、y with MP. For example: - decoders compliant to MVP at a certain level are capable of decoding the bitstreams compliant to MP at the corresponding level (i.e. forward compatibility) - decoders compliant to MP at a certain level are capable of decoding the bitstream in the base layer of MVP (i.e. bac
9、kward compatibility). COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Handling Services2 Rep. ITU-R BT.2017 Right view Left view FIGURE 2 Prediction configuration example with M = 3 coding of left-view, right-view frame picture coded using disparity pr
10、ediction with respect to left-view and motion prediction with respect to itself 1.2 Parameter values for MVP The levels for the MVP are high, high-1440, main and low. Temporal scalability involves two layers, a base layer and an enhancement layer. Both the enhancement and base layers have the same s
11、patial resolution at the same frame rate. Tables 1 to 4 present bounds on sampling rates, luminance pel rates, bit rates and buffer sizes for the MVP. TABLE 1 Upper bounds for sampling density Level Spatial resolution layer I Multiview High High- 1440 Enhancement (right view) Sampledline Linedframe
12、Frameds 1 920 1152 60 Lower Linedframe (left view) Sampledline Frameds Enhancement Linedframe (right view) Sampledline Frameds Lower Linedframe (left view) Sampledline Frameds 1 920 1152 60 1 440 1152 60 1 440 1152 60 Main Enhancement (right view) Sampledline Linedframe Frameds 720 576 30 Lower (lef
13、t view) Sampledline Linedframe Frameds 720 576 30 Low Enhancement (right view) Sampledline Linedframe Frameds 352 288 30 Lower 30 Frameds 288 Linedframe (left view) 352 Sampledline COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Handling ServicesRep. I
14、TU-R BT.2017 3 TABLE 2 Upper bounds for luminance sample rate (samples/s) Level Spatial resolution Profile layer Lower (left view) 62 668 800 I High-1440 Enhancement (right view) I I 47 O01 600 I Lower (left view) 10 368 O00 Enhancement (right view) Main 47 O01 600 10 368 O00 Lower (left view) I Low
15、 I Enhancement (right view) I 3041 280 I Lower (left view) 3 041 280 TABLE 3 Upper bounds for bit rates (Mbit/s) Profile I Level Multiview High 130 both layers 80 base layer High-1440 100 both layers 60 base layer Main 25 both layers 15 base layer Low 8 both layers 4 base layer Level High High-1440
16、Main Low TABLE 4 Buffer size requirements (bits) Layer Enhancement B ase Enhancement B ase Enhancement B ase Enhancement B ase Profile Multiview 15 898 480 9 787 248 12 222 464 7 340 032 3 047 424 1 835 O08 950 272 475 136 1.3 Camera parameter extension An extension for camera information has been i
17、ntroduced in MVP. The extension specifies the height of image device, the focal length, the F-number, the vertical angle of the field of view, the position and the direction of the camera, and upper direction of the camera. COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLice
18、nsed by Information Handling Services4 Rep. ITU-R BT.2017 2 Assessment tests for MVP The verification tests for the MVP were carried out at three different test sites located in Japan, Germany and Canada. The results of tests were presented at the WG 11 Chicago meeting (WG 11 N1373, September 1996.
19、Test and video subgroup “Results of MPEG-2 multiview profile verification test”). The results of the different test sites are consistent with each other and show that in general, at the tested bit rates, the observers judged that the MPEG-2 multi-view profile coding scheme did not introduce annoying
20、 coding artifacts. 2.1 Test method The double stimulus impairment scale method (variant 11) in Recommendation ITU-R BTS00 was applied. Instead of the discrete scale recommended by ITU-R a continuous scale was used in order to obtain more precise evaluations. 2.2 Test conditions The test sequences ge
21、nerated during bit stream exchange were used. An overview of the test conditions is provided in Table 5. Different display systems were used at each test site. TABLE 5 Overview of the subjective test conditions Sequences Algorithms and bit-rates (leftlright view) Test method Stereoscopic display sys
22、tem (picture size, viewing distance) Observers “Street organ”, “Flower pot”, “Trapeze” (525160) “Fun fair” (625150) MVPML: 613 Mbitls, 914 Mbitls Simulcast of “ML: 4.514.5 Mbitls, 6.516.5 Mbitls Simulcast of “ML as lower anchor: 2.512.5Mbitls (for “Street organ”, “Fun fair”), 1.511.5 Mbitls (for “Fl
23、ower pot”, “Trapeze”) Originalloriginal as upper anchor The double-stimulus impairment scale method (variant II) described in Recommendation lTU-R BT.500, with a continuous scale HHI: two-mirror display system (19 cm X 14 cm, 5 H) CRC: time sequential display and LCD shutter eyeglasses (40.6 cm X 30
24、.5 cm, 4 H) NHK: LCD high definition television (HDTV) projectors and polarizing eyeglasses (82 cm X 57 cm, 5 H) HHI: 24 non-expert viewers CRC: 18 non-expert viewers NHK: 19 non-expert viewers (an observer was rejected by screening based on Recomnebdation TU-R BT.500) HHI: Heinrich-Hertz-Institut f
25、r Nachrichtentechnik (Germany) CRC: Communications Research Center (Canada) NHK: Nippon Hoso (Kyokai (Japan) 2.3 Results of subjective assessment tests The mean scores and the 95% confidence intervals of the means were calculated for each test condition. The test results of HHI, CRC and NHK are prov
26、ided in Table 6 and Fig. 3. HHIl and HH12 are results obtained at HHI on two different parts of the same sequence. HHI could not test the whole sequences because of limitations of display memory size. COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Han
27、dling ServicesRep. ITU-R BT.2017 TABLE 6 Mean scores and the 95% confidence intervals a) Sequence: Street organ Source MPx2 MVP MPx2 MVP (914 Mbitls) anchor Lower (4.514.5 Mbit/s) (613 Mbitls) (6.516.5 Mbit/s) NHK f 0.23 f 0.32 f 0.24 f0.13 f0.19 f0.13 1.80 4.24 4.44 4.85 4.68 4.86 HHI2 f0.19 f 0.35
28、 f 0.26 f 0.22 f 0.21 f0.12 1.30 3.63 4.23 4.58 4.55 4.89 HHIl f 0.35 f 0.35 f 0.34 f 0.29 f 0.33 f 0.37 2.19 4.07 4.27 4.33 4.19 4.24 CRC f 0.33 f 0.32 f 0.39 f 0.26 f 0.27 f0.17 1.74 3.5 1 4.06 4.40 4.18 4.71 b) Sequence: Flower pot c) Sequence: Trapeze Source anchor (4.514.5 Mbit/s) (613 Mbitls)
29、(6.516.5 Mbit/s) (914 Mbitls) Lower MPx2 MVP MPx2 MVP NHK f O. 14 f 0.28 f 0.27 f 0.25 f0.19 fO.ll 1.13 4.46 4.48 4.55 4.60 4.90 HHIl f 0.31 f 0.24 f 0.23 f0.14 f 0.24 f 0.22 1.78 4.36 4.37 4.62 4.38 4.48 CRC f0.18 f 0.23 f 0.24 f 0.38 f 0.25 f0.13 1.33 4.41 4.16 4.34 4.24 4.77 d) Sequence: Fun fair
30、 Source anchor (4.514.5 Mbit/s) (613 Mbitls) (6.516.5 Mbit/s) (914 Mbitls) Lower MPx2 MVP MPx2 MVP HHIl f 0.20 f 0.27 f 0.35 f 0.29 f 0.32 f O. 14 1.27 3.46 3.10 4.23 3.96 4.83 5 COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Handling Services6 Rep. I
31、TU-R BT.2017 FIGURE 3 Mean scores of subjective assessment NHK CRC HHIl HH12 a) Street organ NHK CRC HHIl HH12 b) Flower pot NHK CRC HHI c) Trapeze 0 Souce 0 MVP (6/3 Mbit/s) MVP (9/4 Mbit/s) MP x 2 (4.5/4.5 Mbitls) 0 MP x 2 (6.5/6.5 Mbitls) HL ower anchor HHI d) Fun fair Some aspects of the results
32、 of this test seem worth being mentioned: - Within each of the four sequences, the mean score of the MVP sequence at a bit rate of 9/4 Mbitls does not differ significantly from the mean score of the simulcast of MPs at a bit rate of 636.5 Mbitls. As well, the mean score of the MVP sequence at a bit
33、rate of 6/3 Mbitls does not differ significantly from the mean score of the simulcast of MPs at a bit rate of 434.5 Mbitls, except the one pair of the sequence “Street organ.” For “Street organ,” the quality of MVP is superior to that of simulcast of MPs. These results show that differences in subje
34、ctive evaluation between MVP and simulcast of MPs are very small at higher bit rate for pictures with slight motions (“Flower pot” and “Trapeze”) and/or with significant luminance difference between left and right views (“Fun fair“). - “Fun fair“ is the scene with the most differing mean scores. In
35、this scene the most movement (changes of the image content to the next frame) could be observed in comparison with the other scenes. Especially in “Fun fair“, the moving objects cover most of the image. 3 Future work on stereoscopic television Progress made up to now has provided evidence that stere
36、oscopic television is technically feasible. The recently approved MPEG multi-view profile provides a basis for coding and compression of stereoscopic video sequences. The quality assessment tests carried out also have brought evidence that, within the limits of the test parameters chosen, COPYRIGHT
37、International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Handling ServicesRep. ITU-R BT.2017 7 subjectively perceived satisfactory picture quality can be achieved. Nevertheless, still many issues remain unanswered. Some of the issues that require further information are
38、as follows: 3.1 Requirement - It will be desirable that any future stereoscopic television system is compatible with the currently emerging monoscopic digital television systems, and additional bit rate should be as small as possible. - The quality of the monoscopic main picture that may be viewed o
39、n a monoscopic television display should be as close to that of the quality of a monoscopic picture using the entire channel capacity. 3.2 Required information for both standard definition television (SDTV) and HDTV - The degree of asymmetric bit-rate allocation that is possible to the left- and rig
40、ht-view pictures for a stereoscopic video sequence to achieve minimal quality degradation for the base level picture. - The effect of asymmetric bit rate allocation to the left- and right-view pictures on the subjectively perceived coding and compression artefacts and overall quality of the stereosc
41、opic video sequence. - The factors that may lead to viewer fatigue; and mitigating measures that could reduce or eliminate such fatigue. - The required bit rate range to achieve subjectively perceived satisfactory quality for both the stereoscopic picture as well as the monoscopic picture provided b
42、y the base level picture, through additional assessment tests with a large number of video sequences representing a wide range of programming material and for a wide bit-rate range. - Appropriate assessment test methods for stereoscopic images. - Coding algorithms that enable more efficient compression of stereoscopic television signals. These studies should be carried out maintaining the liaison with WP 11B, JWP 10-11Q, and other relevant working parties and organizations. COPYRIGHT International Telecommunications Union/ITU RadiocommunicationsLicensed by Information Handling Services
