1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T P.1203.3 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (12/2016) SERIES P: TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Models and tools for quality assessment of streamed media Parametric
2、bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport Quality integration module Recommendation ITU-T P.1203.3 ITU-T P-SERIES RECOMMENDATIONS TELEPHONE TRANSMISSION QUALITY, TELEPHONE INSTALLATIONS, LOCAL LINE NETWORKS Vocabula
3、ry and effects of transmission parameters on customer opinion of transmission quality Series P.10 Voice terminal characteristics Series P.30 P.300 Reference systems Series P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related
4、to speech loudness Series P.70 Methods for objective and subjective assessment of speech quality Series P.80 Methods for objective and subjective assessment of speech and video quality Series P.800 Audiovisual quality in multimedia services Series P.900 Transmission performance and QoS aspects of IP
5、 end-points Series P.1000 Communications involving vehicles Series P.1100 Models and tools for quality assessment of streamed media Series P.1200 Telemeeting assessment Series P.1300 Statistical analysis, evaluation and reporting guidelines of quality measurements Series P.1400 Methods for objective
6、 and subjective assessment of quality of services other than speech and video Series P.1500 For further details, please refer to the list of ITU-T Recommendations. Rec. ITU-T P.1203.3 (12/2016) i Recommendation ITU-T P.1203.3 Parametric bitstream-based quality assessment of progressive download and
7、adaptive audiovisual streaming services over reliable transport Quality integration module Summary Recommendation ITU-T P.1203.3 specifies the quality integration module for Recommendation ITU-T P.1203. The ITU-T P.1203 series of ITU-T Recommendations specify modules for monitoring the audio, video
8、and audiovisual quality of video services such as adaptive bitrate video streaming. The respective ITU-T work item has formerly been referred to as P.NATS (parametric non-intrusive assessment of TCP-based multimedia streaming quality). The ITU-T P.1203.3 part of Recommendation ITU-T P.1203 can be ap
9、plied to the monitoring of performance and quality of experience (QoE) of video services such as adaptive bitrate video streaming. This Recommendation includes an electronic attachment containing the full 20 trees described in clause 8.4. History Edition Recommendation Approval Study Group Unique ID
10、* 1.0 ITU-T P.1203.3 2016-12-22 12 11.1002/1000/13161 Keywords Adaptive streaming, audio, audiovisual, IPTV, mean opinion score (MOS), mobile video, mobile TV, monitoring, multimedia, progressive download, QoE, TV, video. * To access the Recommendation, type the URL http:/handle.itu.int/ in the addr
11、ess field of your web browser, followed by the Recommendations unique ID. For example, http:/handle.itu.int/11.1002/1000/11830-en. ii Rec. ITU-T P.1203.3 (12/2016) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, in
12、formation and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a
13、worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down
14、in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication ad
15、ministration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory p
16、rovisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws attentio
17、n to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others ou
18、tside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the late
19、st information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2017 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T P.1203.3 (12/2016) iii Tab
20、le of Contents Page 1 Scope . 1 2 References . 2 3 Definitions 2 3.1 Terms defined elsewhere 2 3.2 Terms defined in this Recommendation . 2 4 Abbreviations and acronyms 3 5 Conventions 3 6 Building blocks . 3 7 Model input . 4 7.1 I.14 input specification . 4 8 Model output . 5 8.1 Media parameter e
21、xtraction 5 8.2 Model output O.34 10 8.3 Model output O.35 11 8.4 Model output O.46 12 8.5 Model output O.23 13 Rec. ITU-T P.1203.3 (12/2016) 1 Recommendation ITU-T P.1203.3 Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliabl
22、e transport Quality integration module 1 Scope This Recommendation1 describes an objective parametric quality assessment model as part of the ITU-T P.1203 series of ITU-T Recommendations. The model specified herein predicts the impact of audiovisual quality variations and stalling events on quality
23、experienced by the end user in multimedia mobile streaming and fixed network applications using adaptive bitrate streaming, based on a previous estimation of audio and video quality and information on stalling events during the media session. The model predicts mean opinion scores (MOS) on a 5-point
24、 ACR scale (see ITU-T P.910) as a final audiovisual quality MOS score (as defined in ITU-T P.911, for instance). The model also outputs a perceptual stalling quality indicator, a final audiovisual compression quality score and a list of integrated audiovisual quality scores for diagnostic purposes.
25、This model cannot provide a comprehensive evaluation of transmission quality as perceived by a particular end user because its scores reflect the impairments on the IP network being measured, which may only be part of the end-to-end connection. Furthermore, the scores predicted by a parametric model
26、 necessarily reflect an average perceptual impairment. Note also that the model is developed with a specific encoder and decoder pair. If a different encoder and decoder pair is used in a monitoring situation the scores will not reflect that. Note that user interactions (such as pausing, seeking, us
27、er initiated quality change, user initiated play or user initiated end) are NOT considered in this model either. The effects of audio level, noise, delay (and corresponding similar video factors) and other impairments related to the payload are not reflected in the scores computed by this model. The
28、refore, it is possible to have high scores with this model, yet have a poor quality stream overall. Moreover, the scores predicted by a parametric model (i.e., without access to payload information) necessarily reflect a somewhat simplified representation of the perceptual impairment of the consider
29、ed stream. The application ranges of the ITU-T P.1203.3 model are summarized in Table 1. Table 1 Factors and application ranges of the ITU-T P.1203.3 model algorithms Video sequence duration 60 seconds - 5 minutes Initial-buffering 0-10 seconds Maximum number of stalling events 5 Maximum length of a
30、 single stalling event 15 seconds Total stalling duration 30 seconds Other details No stalling within 5 seconds of the start of the video playing. 1 This Recommendation includes an electronic attachment containing the full 20 trees described in clause 8.4. 2 Rec. ITU-T P.1203.3 (12/2016) 2 Reference
31、s The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of
32、 this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to a document within this Recommendatio
33、n does not give it, as a stand-alone document, the status of a Recommendation. ITU-T P.800.1 Recommendation ITU-T P.800.1 (2016), Mean opinion score (MOS) terminology. ITU-T P.910 Recommendation ITU-T P.910 (2008), Subjective video quality assessment methods for multimedia applications. ITU-T P.911
34、Recommendation ITU-T P.911 (1998), Subjective audiovisual quality assessment methods for multimedia applications. ITU-T P.1201 Recommendation ITU-T P.1201 (2012), Parametric non-intrusive assessment of audiovisual media streaming quality. ITU-T P.1203 Recommendation ITU-T P.1203 (2016), Parametric b
35、itstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport. ITU-T P.1203.1 Recommendation ITU-T P.1203.1 (2016), Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over r
36、eliable transport Video quality estimation module. ITU-T P.1203.2 Recommendation ITU-T P.1203.2 (2016), Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport Audio quality estimation module. ITU-T P.1401 Recommendati
37、on ITU-T P.1401 (2012), Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models. 3 Definitions 3.1 Terms defined elsewhere This Recommendation uses the following term defined elsewhere: 3.1.1 mean opinion score (MOS): ITU-T P.80
38、0.1 3.2 Terms defined in this Recommendation This Recommendation defines the following terms: 3.2.1 bitstream: The part of an IP-based transmission where the actual audiovisual, video or audio content is available in encoded and packetized form. 3.2.2 compression artefacts: Artefacts introduced due
39、to lossy compression of the encoding process. 3.2.3 model, model algorithm: An algorithm with the purpose of estimating the subjective (perceived) quality of a media sequence. Rec. ITU-T P.1203.3 (12/2016) 3 3.2.4 stalling (rebuffering) artefacts: Artefacts coming from stalling (rebuffering) events
40、at the client side, which could be a result of video data arriving late. Usually, stalling events are indicated to the viewer, e.g., in the form of a spinning wheel. This is also referred to as “freezing without skipping“. 3.2.5 initial buffering: Refers to the time in seconds between the initiation
41、 of video playback by the user and the actual start of the playback. 3.2.6 media adaptation: Events where the player switches video playback between a known set of media quality levels while adapting to network conditions. 4 Abbreviations and acronyms This Recommendation uses the following abbreviat
42、ions and acronyms: MOS Mean Opinion Score 5 Conventions None. 6 Building blocks The overall ITU-T P.1203 architecture is shown in Figure 6-1. The model described in this Recommendation is highlighted as the “Pq: Quality integration module“. The model takes as input 1) audio coding quality per output
43、 sampling interval (O.21), 2) video coding quality per output sampling interval (O.22) scores and 3) stalling positions and their length (I.14). An internal media parameter extraction module extracts audio- and video-related parameters. The model outputs 1) final media session quality score (O.46),
44、2) audiovisual segment coding quality per output sampling interval (O.34), 3) final audiovisual coding quality score (O.35) and 4) perceptual buffering indication (O.23). The model details are described in clauses 7 and 8. 4 Rec. ITU-T P.1203.3 (12/2016) Figure 6-1 Pq module in context of building b
45、locks of the ITU-T P.1203 model 7 Model input The model must receive information about the estimated quality of the audio and video streams and the occurrence of stalling events during playback. In various modes of operation, the following inputs may be extracted or estimated in different ways, whic
46、h are outside the scope of this Recommendation: The model must receive the following input signals regardless of the mode of operation: O.21: audio quality per output sampling interval, as specified in ITU-T P.1203.2. O.22: video quality per output sampling interval, as specified in ITU-T P.1203.1.
47、I.14: stalling events, as described in clause 7.1. I.GEN: device type (either “PC“ or “Mobile“), as specified in ITU-T P.1203. The audio and video coding quality estimations (O.21 and O.22) may be generated using ITU-T P.1203.2 and ITU-T P.1203.1 modules, respectively, but may also be generated usin
48、g other audiovisual quality prediction models, provided that they 1) operate on the same output score range and 2) output sampling interval as defined in ITU-T P.1203. 7.1 I.14 input specification I.14 consists of a vector of stalling events observed in the media session. Each event consists of a tu
49、ple of start time and duration, both measured in seconds. The start time is expressed in terms of a “normalized time“/“media time“ which is calculated from the start of the original sequence (i.e., the sequence without any stalling). Initial buffering (i.e., stalling occurring at the beginning of the sequence before playback) has a start time of 0. All other stalling events have a start time greater than zero. In practice, I.14 input information may be represented as a file containing one line per stalling event, with eac
