SMPTE RDD 34-2015 LLVC C Low Latency Video Codec for Network Transfer.pdf

1、 The attached document is a Registered Disclosure Document prepared by the proponent identified below. It has been examined by the appropriate SMPTE Technology Committee and is believed to contain adequate information to satisfy the objectives defined in the Scope, and to be technically consistent.

2、This document is NOT a Standard, Recommended Practice or Engineering Guideline, and does NOT imply a finding or representation of the Society. Errors in this document should be reported to the proponent identified below, with a copy to engsmpte.org. All other inquiries in respect of this document, i

3、ncluding inquiries as to intellectual property requirements that may be attached to use of the disclosed technology, should be addressed to the proponent identified below. Proponent contact information: Toshiaki Kojima Sony Corporation 4-14-1 Asahi-cho, Atsugi Kanagawa, 243-0014 Japan Email: Toshiak

4、i.K Page 1 of 32 pages SMPTE RDD 34:2015 SMPTE REGISTERED DISCLOSURE DOCUMENT LLVC Low Latency Video Codec for Network Transfer Copyright 2015 by THE SOCIETY OF MOTION-PICTURE AND TELEVISION ENGINEERS 3 Barker Avenue, White Plains, NY 10601 (914) 761-1100 Approved September 1, 2015 SMPTE RDD 34:2015

5、 Page 2 of 32 pages Table of Contents Page Introduction 3 1 Scope . 3 2 Mathematical Operators 4 2.1 Arithmetic Operators . 4 2.2 Mathematical Functions 4 3 Terms and Definitions 5 3.1 Codestream . 5 3.2 Coefficient . 5 3.3 Component 5 3.4 CU . 5 3.5 Decomposition level 5 3.6 Entropy decoding 5 3.7

6、Entropy encoding 5 3.8 HB CU . 5 3.9 LB CU 5 3.10 Picture . 5 3.11 Precision 5 3.12 Quantization 6 3.13 Sub-band . 6 3.14 TU 6 3.15 Word 6 3.16 VLC . 6 3.17 VLD . 6 4 Overview of Codec Characteristics . 7 4.1 Codec Key Technologies . 7 4.2 Encoding/Decoding Block Diagram . 8 5 Structure of Codestrea

7、m 9 5.1 Overall . 9 5.2 Picture . 10 5.3 TU (Transmission Unit) 11 5.4 CU (Coding Unit) . 12 6 Video Decoder . 15 6.1 Video Decoder Flow 15 6.2 Scanning of Coefficients 16 6.3 VLD (Variable Length Decoding) . 17 6.3.1 VLD Parameters . 18 6.3.2 Initialization 18 6.3.3 DPT Value Decoding 19 6.3.4 SGN

8、Value Decoding . 20 6.3.5 ABS Value Decoding 20 6.3.6 Decoding Example . 21 6.4 Inverse Quantization . 22 6.5 Inverse Wavelet Transform . 23 6.6 Inverse Wavelet Transform and Post-Inverse Wavelet Transform 26 6.7 Post Inverse Wavelet Transform and Coding Unit 27 7 IP Mapping of Codestream 29 7.1 Ove

9、rall . 29 7.2 Frame (Field) Support of Media Payload 29 7.3 RTP Packetization of Codestream 30 Annex A Encoding, Packetizing, De-Packetizing and Decoding (Informative). 31 Annex B Bibliography (Informative) 32 SMPTE RDD 34:2015 Page 3 of 32 pages Introduction The highest speed network environment th

10、at is readily available is 10GB Ethernet. Considering the bandwidth required for high resolution HD and UHDTV video, 10GB is not necessarily sufficient and it is desirable to implement some degree of video compression during transfer across the network. The major requirements for such a video compre

11、ssion scheme are low latency, and high picture quality. However, there is a trade-off between low latency, compression ratio, and picture quality. The video compression described in this RDD provides low latency of less than one video frame and is able to offer visually lossless quality. 1 Scope Thi

12、s RDD describes the Low Latency Video Codec (LLVC) and related technical information; in particular, a description is given of an example decoder implementation with sub-sampling scheme of 4:2:2 and 4:4:4. The following items are four main elements described in this RDD: a) The codec characteristics

13、: brief introduction to the technologies which offer low memory/low latency and high quality simultaneously. Block diagrams of the encoding and decoding processes are also shown. b) The codestream: a compressed image data representation which includes all necessary data to allow a (full or approxima

14、te) reconstruction of the sample values of a digital image. Additional data might be required that define the interpretation of the sample data, such as the spatial dimensions of the samples. c) A decoder: the decoder takes as input a codestream, and by means of a specified set of procedures generat

15、es as output digital reconstructed image data. Sufficient information is provided to enable an expert skilled in the art of wavelet-based image compression to be able to construct a compatible decoder. d) IP mapping: a simple introduction to the IP mapping of the codestream. SMPTE RDD 34:2015 Page 4

16、 of 32 pages 2 Mathematical Operators 2.1 Arithmetic Operators + Addition Subtraction (as a binary operator) or negation (as a unary prefix operator) 2 Step 2: b = b + (a + c) 1 Two levels of the transform are illustrated in Figure 6.13. For example, the Step 1 calculation computes coefficient c, th

17、en the Step 2 calculation computes coefficient b. Out-of-boundary coefficients shall be symmetrically extended as shown in Figure 6.13. The advantage of this method is to be able to directly compute the inverse wavelet transform with a small number of coefficients in memory. Figure 6.13 Lifting Oper

18、ations of 5-3 Inverse Wavelet Transform SMPTE RDD 34:2015 Page 26 of 32 pages 6.6 Inverse Wavelet Transform and Post-Inverse Wavelet Transform As shown in Figure 6.1, the video decoder employs an inverse wavelet transform to LB CU. A post-inverse wavelet transform is employed to generate decoded ima

19、ge correspondents to the TU. Figure 6.14 shows the overview of the one-dimensional inverse wavelet transform with 3 composition levels, followed by the post-inverse wavelet transform. The lifting operations in Figure 6.14 are the same as shown in Figure 6.13 between level 2 and level 1. The same pro

20、cedure is used between level 3 and level 2. The post-inverse wavelet transform generates decoded images. Figure 6.14 Inverse Wavelet Transform (IWT) and Post-Inverse Wavelet Transform (Post IWT) SMPTE RDD 34:2015 Page 27 of 32 pages 6.7 Post Inverse Wavelet Transform and Coding Unit In the case of t

21、he 4:2:2 format, CU0, CU3, CU4 and CU5 are inverse wavelet transformed as shown in Figure 6.15 so that component(0) is generated. Similarly, CU1 and CU6 are inverse wavelet transformed and component(1) is generated. Finally, CU2 and CU7 are inverse wavelet transformed and component(2) is generated.

22、Figure 6.15 Post-Inverse Wavelet Transform and CU - 4:2:2 Format SMPTE RDD 34:2015 Page 28 of 32 pages In the case of the 4:4:4 format, CU0, CU3, CU4 and CU5 are inverse wavelet transformed as shown in Figure 7.16 so that component(0) is generated. Similarly, CU1 , CU6, CU7 and CU8 are inverse wavel

23、et transformed and component(1) is generated. Finally, CU2, CU9, CU10 and CU11 are inverse wavelet transformed and component(2) is generated. Figure 6.16 Post Inverse Wavelet Transform and CU - 4:4:4 Format SMPTE RDD 34:2015 Page 29 of 32 pages 7 IP Mapping of Codestream 7.1 Overall In order to achi

24、eve correct delivery of the video content from transmitters to receivers, an advanced method is required to split the compressed codestreams into network packets to be sent via an RTP protocol such as multicast. To this purpose, three main procedures are needed: 1) Preparation of the compressed code

25、stream (low latency video codec) 2) Splitting of codestreams into network packets (such as RTP packets) 3) Adaptation of the IP protocol packet headers to the codestream 7.2 Frame (Field) Support of Media Payload Each video frame (or field in the case of interlaced video) of consecutive data of the

26、compressed codestream consists of a Picture_info header followed by multiple Transmission Units (TUs), as illustrated in Figure 7.1. The codestream is divided into a number of fixed-size Media Payload blocks. Zero padding shall be added as required to the final Media Payload at the end of the frame

27、(or field). Figure 7.1 Relation between Video Frame (or Field) and Compressed Video Media Payload SMPTE RDD 34:2015 Page 30 of 32 pages 7.3 RTP Packetization of Codestream Figure 7.2 shows one example of RTP packetization of a compressed codestream. The compressed codestream will be separated into multiple numbers of RTP packets containing the Media Payload. Each Media Payload is equipped with UDP header, RTP header and Payload Header. Figure 7.2 RTP Packetization of Compressed Codestream