1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T P.915 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (03/2016) SERIES P: TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Audiovisual quality in multimedia services Subjective assessment methods for 3D video quali
2、ty Recommendation ITU-T P.915 ITU-T P-SERIES RECOMMENDATIONS TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Voice terminal characteristics Series P.30 P.300 Reference systems Series P
3、.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of speech quality Series P.80 P.800 Audiovisual quality in multimedia services Series P.900 Tr
4、ansmission performance and QoS aspects of IP end-points Series P.1000 Communications involving vehicles Series P.1100 Models and tools for quality assessment of streamed media Series P.1200 Telemeeting assessment Series P.1300 Statistical analysis, evaluation and reporting guidelines of quality meas
5、urements Series P.1400 Methods for objective and subjective assessment of quality of services other than voice services Series P.1500 For further details, please refer to the list of ITU-T Recommendations. Rec. ITU-T P.915 (03/2016) i Recommendation ITU-T P.915 Subjective assessment methods for 3D v
6、ideo quality Summary Recommendation ITU-T P.915 describes non-interactive subjective assessment methods for evaluating the one-way overall video quality for three-dimensional (3D) video applications such as 3D videoconferencing, and 3D cable television. These methods can be used for several differen
7、t purposes including, but not limited to, selection of algorithms, ranking of system performance and evaluation of the quality level during a video connection. ITU-T P.915 also outlines the characteristics of the source sequences to be used, such as duration, kind of content and number of sequences.
8、 Details within ITU-T P.915 are expected to change, based on experiments into how best to conduct 3DTV subjective tests. History Edition Recommendation Approval Study Group Unique ID* 1.0 ITU-T P.915 2016-03-15 9 11.1002/1000/12777 * To access the Recommendation, type the URL http:/handle.itu.int/ i
9、n the address field of your web browser, followed by the Recommendations unique ID. For example, http:/handle.itu.int/11.1002/1000/11830-en. ii Rec. ITU-T P.915 (03/2016) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunicati
10、ons, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunication
11、s on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure lai
12、d down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunica
13、tion administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mand
14、atory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws a
15、ttention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or ot
16、hers outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent t
17、he latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2016 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T P.915 (03/2016) iii
18、 Table of Contents Page 1 Scope . 1 2 References . 1 3 Definitions 2 3.1 Terms defined elsewhere 2 3.2 Terms defined in this Recommendation . 2 4 Abbreviations and acronyms 3 5 Conventions 4 6 Selection of 3D source content . 4 6.1 Visual comfort 4 6.2 Source signal recordings . 5 6.3 Spatial and te
19、mporal information . 5 6.4 Optional subjective methods for 3D reference scene selection: Visual experience and visual comfort requirements 5 6.5 Discrepancies between left and right images . 6 6.6 Duration of stimuli . 6 6.7 Number of source stimuli . 7 7 Test methods and experimental design . 7 7.1
20、 Single and multiple question experiments 7 7.2 Assessment methods . 7 7.3 Changes to the methods 10 8 Environment . 12 8.1 Maximum display crosstalk 12 8.2 Screen brightness 12 8.3 Viewing distance and angle 12 8.4 Viewing conditions . 13 8.5 Colour temperature of 3D displays . 13 8.6 Documentation
21、 of environment 13 9 Subjects . 13 10 Experimental design . 15 10.1 Inclusion of reference conditions within the experiment . 15 10.2 Size of the experiment and subject fatigue . 15 10.3 Special considerations for transmission error, rebuffering, and audiovisual synchronization impairments 15 11 Exp
22、eriment implementation 16 11.1 Informed consent 16 11.2 Viewer screening 17 11.3 Post-screening of subjects 18 11.4 Instructions and training . 18 iv Rec. ITU-T P.915 (03/2016) Page 11.5 Experiment sessions and breaks . 19 11.6 Questionnaire or interview . 19 12 Data analysis . 19 12.1 Calculate MOS
23、 or DMOS 20 12.2 Evaluating objective metrics 20 12.3 2AFC-PC analysis 20 12.4 Aggregation of scale data . 22 Annex A Method for post-experimental screening of subjects using Pearson linear correlation . 23 A.1 Equations 23 A.2 Screen by PVS 24 A.3 Screen by PVS and HRC 24 Annex B Pair selection for
24、 2AFC-PC . 25 B.1 Optimized rectangular design . 25 B.2 Adaptive rectangular design . 26 Appendix I Issues for further study. 28 Appendix II Sample informed consent form . 30 Appendix III Sample instructions . 31 Bibliography. 32 Rec. ITU-T P.915 (03/2016) v Introduction Stereoscopic three-dimension
25、al (3D) television attempts to emulate the response of the human binocular visual system to the relative depth perception of objects. This Recommendation applies to stereoscopic imaging that directs a different view of the same scene to each eye. The images of the objects depicted in the scene have
26、different relative positions in the left-and right-view. 3D television viewing does not perfectly recreate the real viewing experience, because the normal formulae for accommodation (i.e., focus) and vergence (i.e., eye angle) do not apply. A variety of displays produce this effect, including stereo
27、scopic and autostereoscopic displays, using glasses with polarized lenses or shutters; and 2D televisions using complementary colour anaglyphs and glasses with coloured filters. Assessment factors generally applied to monoscopic (two-dimensional or 2D) television pictures can be applied to stereosco
28、pic television systems. In addition, there are many factors unique to stereoscopic television systems. These include factors such as depth resolution, which is the spatial resolution in depth direction, depth motion (i.e., motion or movements along the depth direction), and visual comfort. Rec. ITU-
29、T P.915 (03/2016) 1 Recommendation ITU-T P.915 Subjective assessment methods for 3D video quality 1 Scope This Recommendation describes subjective evaluation of 3D video. Topics include assessment methods, subjective scales, environmental conditions, viewing distance, display size and data analysis.
30、 These experiments can answer different questions, such as video quality, depth quality, naturalness, visual discomfort, quality of experience, viewing experience and presence. The applications of the subjective assessment methods for 3D video quality described in this Recommendation include, but ar
31、e not limited to obtaining perceptual 3D video quality. This Recommendation contains insufficient information for the following applications. While most of the information provided herein applies, additional constraints are required: medical applications; immersive, virtual reality environments (e.g
32、., gaming, caves, head mounted displays); augmented reality. 2 References The following ITU-T Recommendations and other references contain provisions, which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. A
33、ll Recommendations and other references are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is r
34、egularly published. The reference to a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. ITU-T P.78 Recommendation ITU-T P.78 (1996), Subjective testing method for determination of loudness ratings in accordance with Recommendation P.76.
35、 ITU-T P.800 Recommendation ITU-T P.800 (1996), Methods for subjective determination of transmission quality. ITU-T P.800.2 Recommendation ITU-T P.800.2 (2013), Mean opinion score interpretation and reporting. ITU-T P.914 Recommendation ITU-T P.914 (2016), Display requirements for 3D video quality a
36、ssessment. ITU-T P.916 Recommendation ITU-T P.916 (2016), Information and guidelines for assessing and minimizing visual discomfort and visual fatigue from 3D video. ITU-T P.1401 Recommendation ITU-T P.1401 (2012), Methods, metrics and procedures for statistical evaluation, qualification and compari
37、son of objective quality prediction models. ITU-R BT.500-13 Recommendation ITU-R BT.500-13 (2012), Methodology for the subjective assessment of the quality of television pictures. ITU-R BT.1788 Recommendation ITU-R BT.1788 (2007), Methodology for the subjective assessment of video quality in multime
38、dia applications. 2 Rec. ITU-T P.915 (03/2016) ITU-R BT.2021-1 Recommendation ITU-R BT.2021 (2015), Subjective methods for the assessment of stereoscopic 3DTV systems. ITU-R BT.2160-2 Report ITU-R BT.2160-2 (10/2011), Features of three-dimensional television video systems for broadcasting. 3 Definit
39、ions 3.1 Terms defined elsewhere This Recommendation uses the following terms defined elsewhere: 3.1.1 double stimulus b-ITU-T P.913: A quality rating method where the subject is presented with two stimuli; the subject then rates both stimuli in the context of the joint presentation (e.g., a rating
40、that compares the quality of one stimulus to the quality of the other). 3.1.2 hypothetical reference circuit (HRC) b-ITU-T P.913: A fixed combination of a video encoder operating at a given bit rate, network condition and video decoder. The term HRC is preferred when vendor names should not be ident
41、ified. 3.1.3 least distance of distinct vision (reference seeing distance) b-ITU-T P.913: The closest distance at which someone with normal vision (20/20 vision) can comfortably look at something 3.1.4 processed b-ITU-T P.913: The reference stimuli presented through a system under test. 3.1.5 proces
42、sed video sequence (PVS) b-ITU-T P.913: The impaired version of a video sequence. 3.1.6 reference b-ITU-T P.913: The original version of each source stimulus. This is the highest quality version available of the audio sample, video clip or audiovisual sequence. 3.1.7 sequence b-ITU-T P.913: A contin
43、uous sample of audio, video or audiovisual content. 3.1.8 single stimulus b-ITU-T P.913: A quality rating method where the subject is presented with one stimulus and rates that stimulus in isolation (e.g., a viewer watches one video clip and then rates it). 3.1.9 source b-ITU-T P.913: The content ma
44、terial associated with one particular audio sample, video clip or audiovisual sequence (e.g., a video sequence depicting a ship floating in a harbour). 3.1.10 stimulus b-ITU-T P.913: Audio sequence, video sequence or audiovisual sequence. 3.1.11 subject b-ITU-T P.913: A person who evaluates stimuli
45、by giving an opinion. 3.1.12 terminal b-ITU-T P.913: Device or group of devices used to play the stimuli during a subjective experiment e.g., a laptop with earphones, or a Blu-ray player with an liquid crystal display (LCD) monitor and speakers. 3.2 Terms defined in this Recommendation This Recommen
46、dation defines the following terms: 3.2.1 depth quality: The ability of the system to deliver a sensation of depth. The presence of monocular cues (such as linear perspective, blur and gradients) conveys some sensation of depth even in standard 2D images. Stereoscopic 3D images contain also disparit
47、y information which provides additional depth information and thus an enhanced sense of depth as compared to 2D. 3.2.2 frame effect: The effect of 3D pictures appearing highly unnatural when objects positioned in front of the screen approach the screen frame. The effect is generally reduced with a l
48、arger screen, because observers are less conscious of the existence of the frame when the screen is larger. Rec. ITU-T P.915 (03/2016) 3 3.2.3 naturalness: The perception of the stereoscopic image as being a truthful representation of reality (i.e., perceptual realism). The stereoscopic image may pr
49、esent different types of distortion that make it less natural. For example, stereoscopic objects are sometimes perceived as unnaturally large or small (puppet theatre effect) or they appear unnaturally thin (cardboard effect). 3.2.4 picture quality: The quality of rendering of texture and motion, the level of visibility of visual artifacts and rendering details. The perceived quality of the video provided by the system. This is a main determinant of the performance of any video system. Picture quality
