1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T J.301 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (10/2014) SERIES J: CABLE NETWORKS AND TRANSMISSION OF TELEVISION, SOUND PROGRAMME AND OTHER MULTIMEDIA SIGNALS Application for Interactive Digital Television Part 2 Req
2、uirements for augmented reality smart television systems Recommendation ITU-T J.301 Rec. ITU-T J.301 (10/2014) i Recommendation ITU-T J.301 Requirements for augmented reality smart television systems Summary Recommendation ITU-T J.301 specifies requirements that should be considered for augmented re
3、ality (AR) smart television system (AR-STV). AR technology integrates 2D/3D graphics into real-world situations in order to enhance the user experience and enrich information. As smart devices have become more common, AR has been adopted in various applications such as AR-books, AR shopping, AR-SNS,
4、 etc. Depending on trends of AR technology, TV broadcasting can be positioned as a more advanced and smart service platform by applying this new technology. Usually, mobile devices such as smartphones and tablets have various sensors to determine the location, direction and speed of the device. Howe
5、ver TVs do not have such sensors and cannot provide the same user experience as a smartphone, which can also offer automatic addition of virtual visual components. AR on TV focuses on overlaying visual components onto broadcast content in real time, optionally manipulating components by using a smar
6、t device. AR-STV is TV broadcasting using AR technology to blend augmented content (2D/3D graphic objects) with broadcast content in real time on receiving terminals. The augmented content overlaid in the augmentation region is fixed statically or can move along a trajectory similar to what could be
7、 provided by an AR-based location application on a smart phone that shows path information on the real location from a smartphones camera. AR-STV provides that TV viewers can watch the mixed broadcasting contents whenever they want only the service. History Edition Recommendation Approval Study Grou
8、p Unique ID* 1.0 ITU-T J.301 2014-10-29 9 11.1002/1000/12314 Keywords Augmented Reality, Augmented Reality Smart Television. _ * To access the Recommendation, type the URL http:/handle.itu.int/ in the address field of your web browser, followed by the Recommendations unique ID. For example, http:/ha
9、ndle.itu.int/11.1002/1000/11830-en. ii Rec. ITU-T J.301 (10/2014) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector
10、(ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every fo
11、ur years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, th
12、e necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary.
13、However, the Recommendation may contain certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the nega
14、tive equivalents are used to express requirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the us
15、e of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation
16、, ITU had not received notice of intellectual property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/ww
17、w.itu.int/ITU-T/ipr/. ITU 2015 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T J.301 (10/2014) iii Table of Contents Page 1 Scope . 1 2 References . 1 3 Definitions 1 3.1 Terms defined elsewhere 1 3.
18、2 Terms defined in this Recommendation . 1 4 Abbreviations and acronyms 1 5 Conventions 2 6 Augmented reality smart television (AR-STV) system 2 6.1 System conceptual model . 2 6.2 System architecture 3 6.3 The relationship between broadcast provider and content provider . 5 7 Functional requirement
19、s . 6 7.1 General requirements 6 7.2 System requirements 7 Appendix I Scenarios for augmented reality smart television 8 I.1 Augmentation region based application . 8 I.2 Interactive application with manipulation of the 3D object . 8 I.3 Location based application . 9 Appendix II AR-STV service plat
20、form with 2D/3D graphics rendering engine 10 Bibliography. 11 Rec. ITU-T J.301 (10/2014) 1 Recommendation ITU-T J.301 Requirements for augmented reality smart television systems 1 Scope This Recommendation defines functional requirements of metadata, its delivery and synchronization between broadcas
21、ting content and augmented content for augmented reality smart television services (AR-STV). This Recommendation also describes the use cases as the background of those requirements. 2 References None. 3 Definitions 3.1 Terms defined elsewhere None. 3.2 Terms defined in this Recommendation This Reco
22、mmendation defines the following terms: 3.2.1 augmented broadcasting: Broadcasting service or programme to realize augmented reality smart television (AR-STV). 3.2.2 augmented broadcasting metadata: Formatted data describing augmented content. 3.2.3 augmented broadcast provider: A broadcaster who pr
23、ovides augmented broadcasting services, or an entity allowed to develop and provide augmented broadcasting service as a broadcast provider. 3.2.4 augmented content: A binary object, such as 2D images, 3D animated models or audio/video streaming files, to be augmented into a predefined augmentation r
24、egion. 3.2.5 augmented content provider: An entity allowed to develop and provide augmented content. 3.2.6 augmented reality (AR): A type of mixed reality where graphical elements are integrated into the real world in order to enhance user experience and enrich information. 3.2.7 augmented reality s
25、mart television (AR-STV): AR-based TV broadcasting service to implement the mixed content of a broadcast programme and augmented object at a targeted position and time in real time on a receiving terminal such as a TV or set-top box. 3.2.8 augmentation region: The targeted area to augment any object
26、 or region in a TV scene. It provides the parameters of a static or animated 2D region related to natural media. 3.2.9 augmentation time: The appointed time to augment any object or region in a TV scene. It provides the parameters of start presentation time and time duration for augmented content. 4
27、 Abbreviations and acronyms This Recommendation uses the following abbreviations and acronyms: AR Augmented Reality AR-STV Augmented Reality Smart Television STB Set-Top Box 2 Rec. ITU-T J.301 (10/2014) 5 Conventions In this Recommendation: The keywords “is required to“ indicate a requirement which
28、must be strictly followed and from which no deviation is permitted if conformance to this document is to be claimed. The keywords “is recommended“ indicate a requirement which is recommended but which is not absolutely required. Thus this requirement need not be present to claim conformance. The key
29、words “is prohibited from“ indicate a requirement which must be strictly followed and from which no deviation is permitted if conformance to this document is to be claimed. The keywords “can optionally“ indicate an optional requirement which is permissible, without implying any sense of being recomm
30、ended. This term is not intended to imply that the vendors implementation must provide the option and the feature can be optionally enabled by the network operator/service provider. Rather, it means the vendor may optionally provide the feature and still claim conformance with the specification. In
31、the body of this Recommendation and its annexes, the words shall, shall not, should, and may sometimes appear, in which case they are to be interpreted, respectively, as is required to, is prohibited from, is recommended and can optionally. The appearance of such phrases or keywords in an appendix o
32、r in material explicitly marked as informative are to be interpreted as having no normative intent. 6 Augmented reality smart television (AR-STV) system 6.1 System conceptual model Figure 1 shows a conceptual model for AR-STV. By this novel TV broadcasting service, the user is provided with addition
33、al information alongside the regular broadcast TV programme, and with a more interactive TV experience. The broadcast provider defines an augmentation region so that audio/video content and additional information can be added to that region. The information, formatted as augmented broadcast metadata
34、, is generated by an authoring tool. The specific form of metadata is from MPEG-4 BIFS b-ISO/IEC 23000-13 with some modification, etc. The metadata includes the unique name, object type, position, presentation/life time and resource location of the augmented objects and contains rendering attributes
35、 for mixing augmented objects according to the augmented broadcasting scenario. The service scenario for augmented broadcasting is similar to an acting script for the augmented objects, describing what to augment, when to present, where to locate, how to move and so forth. Content providers manage a
36、n augmented content database harmonized with the scenario. A broadcast programme is transmitted together with metadata to set-top boxes or smart TVs. When viewers wish to view the augmented broadcast service while watching TV, they can launch/control the service via a TV remote control or an applica
37、tion on a second smart mobile device. Therefore, viewers can watch the mixed broadcast contents on demand and only when they want only the service. The system will still provide the original broadcasting content. Moreover, multiple content providers can exist and viewers can be given the choice to s
38、elect the augmented content from a preferred content provider. Rec. ITU-T J.301 (10/2014) 3 Figure 1 Conceptual model for AR-STV 6.2 System architecture The AR-STV system is composed of four essential parts: an authoring tool, the transport multiplexer, a content-providing server and an AR-STV platf
39、orm, such as a TV or TV set-top box equipped with a 2D/3D-graphics rendering engine. A second smart device on which an AR-STV mobile application is installed is needed to allow interactions between the user and the augmented content. In addition, a broadcasting network is needed to transmit TV progr
40、ammes and information for the real-time rendering of an augmented content. Also, augmented content will be delivered over a telecommunication network. Figure 2 shows the overall system framework of an AR-STV service. Figure 2 System architecture of AR-STV Authoring tool An authoring tool is used for
41、 generating information for real-time rendering of augmented content on a TV or set-top box. The information is generated as metadata, which includes the regions where the 4 Rec. ITU-T J.301 (10/2014) augmented content will be identified and presented; the time schedules for when the augmented conte
42、nt will be downloaded; the uniform resource identifiers (URIs) for downloading the augmented content; the types of augmented content to be presented; and the attributes of the augmented content. This information allows a seamless blending of AR graphic images and video. The following figure shows th
43、e augmentation region, augmentation time, its related attributes and their relationships. The augmentation region defines a targeted area for the augmented object to be overlaid onto background video. The augmentation time defines the appointed time for the presentation of augmented content to start
44、 and stop on the TV screen. The augmentation region during the augmentation time can move or rotate, and each event affects the presentation of the augmented content. The rendering environment defines rendering attributes (e.g., lighting position and colours) for naturally matching the augmented con
45、tent with a background video. The TV viewers interaction defines is shown to represent a user generated event for interaction, for example, turn on service, drag object, zoom, etc. Figure 3 Definition of augmentation region and augmentation time The metadata is fragmented into constant-size frames i
46、n order to effectively transmit via the broadcast channel. In addition, the reference point of the fragmentation is the time during which the regions or scenes used for presenting augmented content are created, or the time during which the attributes of the augmented content or the regions or scenes
47、, become altered. Each unit of fragmentation, or “instruction“, may include all or part of the information used for the presentation of an augmented content. To create a region or scene for presenting an augmented content, a reference instruction is generated, which includes all of the information n
48、eeded for the presentation of the augmented content. Subsequent instructions may include only information on variations of the region, scene or augmented content. Each time an augmentation region or scene is created, a reference instruction for the region or scene is generated. Rec. ITU-T J.301 (10/
49、2014) 5 Transport multiplexer In an AR-STV system, a process for rendering augmented content in real-time and blending it into a broadcast video is required. The augmented broadcasting metadata can be multiplexed with a video stream based on an MPEG-2 transport stream. In this process, the packet ID (PID) for the augmented broadcasting metadata needs to be defined, and the programme-specific information (PSI) indicating the relation between the TV programme and the metadata is required to be transmitted through the broadcasting network. I
