ITU-T J 302-2016 System specifications of augmented reality smart television service (Study Group 9).pdf

1、 I n t e r n a t i o n a l T e l e c o m m u n i c a t i o n U n i o n ITU-T J.302 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (10/2016) SERIES J: CABLE NETWORKS AND TRANSMISSION OF TELEVISION, SOUND PROGRAMME AND OTHER MULTIMEDIA SIGNALS Application for Interactive Digital Television Part 2 Sys

2、tem specifications of augmented reality smart television service Recommendation ITU-T J.302 Rec. ITU-T J.302 (10/2016) i Recommendation ITU-T J.302 System specifications of augmented reality smart television service Summary Augmented reality (AR) technology is a kind of mixed reality in which 2D/3D

3、graphics are integrated into the real world in order to enhance user experience and enrich information. Augmented reality smart television (AR-STV) is TV broadcasting using AR technology to blend augmented content (2D/3D graphic object) with broadcasting content in real-time on receiving terminals.

4、The augmented content overlaid in the augmentation region is fixed statically or moves along the trajectory as if an AR-based location application shows path information on the real location from a smart-phones camera. AR-STV provides that TV viewers can choose whether to turn on the augmented conte

5、nt or watch the original TV content only (without the augmented content). Recommendation ITU-T J.302 specifies the related technologies that should be implemented for an augmented reality smart television system. History Edition Recommendation Approval Study Group Unique ID* 1.0 ITU-T J.302 2016-10-

6、14 9 11.1002/1000/13054 Keywords Augmented reality, augmented reality smart television. * To access the Recommendation, type the URL http:/handle.itu.int/ in the address field of your web browser, followed by the Recommendations unique ID. For example, http:/handle.itu.int/11.1002/1000/11830-en. ii

7、Rec. ITU-T J.302 (10/2016) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICTs). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. IT

8、U-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication Standardization Assembly (WTSA), which meets every four years, establishes the topics for st

9、udy by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of information technology which fall within ITU-Ts purview, the necessary standards are prepared on a

10、 collaborative basis with ISO and IEC. NOTE In this Recommendation, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency. Compliance with this Recommendation is voluntary. However, the Recommendation may contain

11、 certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the Recommendation is achieved when all of these mandatory provisions are met. The words “shall“ or some other obligatory language such as “must“ and the negative equivalents are used to express re

12、quirements. The use of such words does not suggest that compliance with the Recommendation is required of any party. INTELLECTUAL PROPERTY RIGHTSITU draws attention to the possibility that the practice or implementation of this Recommendation may involve the use of a claimed Intellectual Property Ri

13、ght. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the Recommendation development process. As of the date of approval of this Recommendation, ITU had received notice of intellectu

14、al property, protected by patents, which may be required to implement this Recommendation. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2017 All rights r

15、eserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. Rec. ITU-T J.302 (10/2016) iii Table of Contents Page 1 Scope . 1 2 References . 2 3 Definitions 2 3.1 Terms defined elsewhere 2 3.2 Terms defined in this Recommendation . 2

16、4 Abbreviations and acronyms 2 5 Conventions 3 6 Signalling protocol for AR-STV 3 7 Synchronization scheme for AR-STV 5 8 Metadata specification for AR-STV . 7 8.1 Initial instruction 8 8.2 Instruction . 8 8.3 Augmented object . 11 8.4 Environment info 11 8.5 User interaction 12 Appendix I Syntax of

17、 XML-based AR-STV metadata . 13 I.1 Namespace 13 I.2 Schema wrapper . 13 I.3 Root element . 13 I.4 Initial instruction 13 I.5 Instruction . 14 I.6 Reference resource . 14 I.7 Augmentation region 15 I.8 Augmented object . 16 I.9 Environment info 16 I.10 User interaction 18 Appendix II Use case exampl

18、e of haptic device . 19 Bibliography. 20 Rec. ITU-T J.302 (10/2016) 1 Recommendation ITU-T J.302 System specifications of augmented reality smart television service 1 Scope Figure 1 shows the reference architecture of the augmented reality smart television (AR-STV) system. Figure 1 System architectu

19、re of AR-STV Augmented reality smart television (AR-STV) provides an AR-based TV broadcasting service to implement the mixed content of a broadcasting programme and augmented content at a targeted position (augmentation region) and time (augmentation time) in real-time on a receiving terminal such a

20、s a smart TV or set-top-box (STB). AR-STV overlays augmented contents provided by the broadcasting provider itself or by the content providers which are authorised by the broadcasting provider. When AR-STV is provided, the broadcasting signal includes the metadata which contains the permissions abou

21、t timing and the area for the overlay. Authorisation information for an AR content provider is also provided in the broadcasting signal. This mechanism does not permit any illegal overlay. The AR-STV is realized by the combination of a specific signal on the broadcasting programme, metadata, augment

22、ed content and terminal side synchronization. The terminal functionality can be implemented as the extension of the existing integrated broadcast-broadband (IBB) systems. The scope of this Recommendation comprises the following technologies to realize AR-STV: Synchronization scheme for AR-STV Metada

23、ta for AR-STV Signalling protocol for AR-STV This Recommendation specifies three technologies for AR-STV, (1) synchronized representation between a broadcasting programme and augmented content, (2) metadata and (3) signalling protocol according to functional requirements defined in ITU-T J.301. 2 Re

24、c. ITU-T J.302 (10/2016) 2 References The following ITU-T Recommendations and other references contain provisions which, through reference in this text, constitute provisions of this Recommendation. At the time of publication, the editions indicated were valid. All Recommendations and other referenc

25、es are subject to revision; users of this Recommendation are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published. The reference to

26、a document within this Recommendation does not give it, as a stand-alone document, the status of a Recommendation. ITU-T J.301 Recommendation ITU-T J.301 (2014), Requirements for augmented reality smart television systems. 3 Definitions 3.1 Terms defined elsewhere This Recommendation uses the follow

27、ing terms defined elsewhere: 3.1.1 augmented broadcasting ITU-T J.301: Broadcasting service or programme to realize augmented reality smart television (AR-STV). 3.1.2 augmented content ITU-T J.301: A binary object, such as 2D images, 3D animated models or audio/video streaming files, to be augmented

28、 into a predefined augmentation region. 3.1.3 augmented reality (AR) ITU-T J.301: A type of mixed reality where graphical elements are integrated into the real world in order to enhance user experience and enrich information. 3.1.4 augmented reality smart television (AR-STV) ITU-T J.301: AR-based TV

29、 broadcasting service to implement the mixed content of a broadcast programme and augmented object at a targeted position and time in real time on a receiving terminal such as a TV or set-top box. 3.1.5 augmentation region ITU-T J.301: The targeted area to augment any object or region in a TV scene.

30、 It provides the parameters of a static or animated 2D region related to natural media. 3.1.6 augmentation time ITU-T J.301: The appointed time to augment any object or region in a TV scene. It provides the parameters of start presentation time and time duration for augmented content. 3.2 Terms defi

31、ned in this Recommendation This Recommendation defines the following term: 3.2.1 reference image: A resource to be referenced in the tracking process of acquiring coordinates generated by a particular regions movement in the video scene. 4 Abbreviations and acronyms This Recommendation uses the foll

32、owing abbreviations and acronyms: ABM Augmented Broadcasting Metadata AR Augmented Reality AR-STV Augmented Reality Smart Television EIT Event Information Table FoV Field of View IBB Integrated Broadcast-Broadband MPEG Moving Picture Experts Group Rec. ITU-T J.302 (10/2016) 3 PMT Program Map Table P

33、SI Program Specific Information PSIP Program and System Information Protocol STB Set-Top-Box TS Transport Stream VCT Virtual Channel Table 5 Conventions In this Recommendation: The keywords “is required to“ indicate a requirement which must be strictly followed and from which no deviation is permitt

34、ed if conformance to this document is to be claimed. The keywords “is recommended“ indicate a requirement which is recommended but which is not absolutely required. Thus this requirement need not be present to claim conformance. The keywords “is prohibited from“ indicate a requirement which must be

35、strictly followed and from which no deviation is permitted if conformance to this document is to be claimed. The keywords “can optionally“ indicate an optional requirement which is permissible, without implying any sense of being recommended. This term is not intended to imply that the vendors imple

36、mentation must provide the option and the feature can be optionally enabled by the network operator/service provider. Rather, it means the vendor may optionally provide the feature and still claim conformance with the specification. In the body of this document and its annexes, the words shall, shal

37、l not, should and may sometimes appear, in which case they are to be interpreted, respectively, as is required to, is prohibited from, is recommended and can optionally. The appearance of such phrases or keywords in an appendix or in material explicitly marked as informative are to be interpreted as

38、 having no normative intent. 6 Signalling protocol for AR-STV The real-time broadcast stream to be provided to the receiving terminal should include signalling and metadata for the AR-STV service. The receiving terminal may determine whether to provide an AR-STV service by checking an AR-STV descrip

39、tor contained in the program initialization information of the broadcast stream. Table 1 illustrates an example of the syntax of an AR-STV descriptor including descriptor_tag, descriptor_length and augmented_broadcasting_service_type. More specifically, the descriptor_tag is information that identif

40、ies an associated descriptor as an AR-STV descriptor. Descriptor_length indicates the total length of the descriptor. The ARSTV_app_type is to distinguish application types of AR-STV service. ContentProvider_Info includes content providers information. The descriptor may be located in a table of a s

41、ection packet, such as a virtual channel table (VCT) or event information table (EIT) of program and system information protocol (PSIP) and/or a program map table (PMT) of program specific information (PSI) in MPEG-2 TS. Table 1 shows the ARSTV descriptor syntax. 4 Rec. ITU-T J.302 (10/2016) Table 1

42、 ARSTV descriptor syntax Syntax Semantics ARSTV_descriptor() Descriptor_tag Descriptor identifier Descriptor_length Descriptor length ARSTV_app_type Application Service Type ContentProvider_Info ContentProvider (N) For(i=0;i Additionally, the following line should be appended to the resulting schema

43、 document in order to obtain a well-formed XML document. I.3 Root element I.4 Initial instruction 14 Rec. ITU-T J.302 (10/2016) I.5 Instruction I.6 Reference resource Rec. ITU-T J.302 (10/2016) 15 I.7 Augmentation region 16 Rec. ITU-T J.302 (10/2016) I.8 Augmented object Rec. ITU-T J.302 (10/2016) 1

44、7 18 Rec. ITU-T J.302 (10/2016) I.10 User interaction Rec. ITU-T J.302 (10/2016) 19 Appendix II Use case example of haptic device (This appendix does not form an integral part of this Recommendation.) With a haptic device, TV users can feel or enjoy the intensity/vibration of the content. Figure II.

45、1 An example of AR-STV service connected with haptic device AR-STV service can be extended beyond the TV screen to an external device such as a haptic device. On receiving a haptic signal from an AR-STV terminal, a TV viewer with haptic gloves, as shown in the case of Figure II.1, would have the tac

46、tile feel of feathers by touching the AR content via user motion sensing. 20 Rec. ITU-T J.302 (10/2016) Bibliography b-ITU-T H.222.0 Recommendation ITU-T H.222.0 (2014) | ISO/IEC 13818-1:2015, Information technology Generic coding of moving pictures and associated audio information: Systems. b-ISO/I

47、EC 23000-13 ISO/IEC 23000-13:2014, Information technology Multimedia application format (MPEG-A) Part 13: Augmented reality application format. Printed in Switzerland Geneva, 2017 SERIES OF ITU-T RECOMMENDATIONS Series A Organization of the work of ITU-T Series D Tariff and accounting principles and

48、 international telecommunication/ICT economic and policy issues Series E Overall network operation, telephone service, service operation and human factors Series F Non-telephone telecommunication services Series G Transmission systems and media, digital systems and networks Series H Audiovisual and

49、multimedia systems Series I Integrated services digital network Series J Cable networks and transmission of television, sound programme and other multimedia signals Series K Protection against interference Series L Environment and ICTs, climate change, e-waste, energy efficiency; construction, installation and protection of cables and other elements of outside plant Series M Telecommunication management, including telecommunication network management and network maintenance Series N Maintenance: international s