ANSI INCITS ISO IEC 14496-2-2001 Information technology - Coding of audio-visual objects Part 2 Visual.pdf

1、INTERNATIONAL STANDARD ISO/IEC 14496-2 Second edition 2001-12-01 Information technology - Coding of audio-visual objects - Part 2: Visual Technologies de linformation - Codage des objets audiovisuels - Partie 2: Codage visuel Adopted by INCITS (InterNational Committee for Information Technology Stan

2、dards) as an American National Standard. Date of ANSI Approval: 8/29/02 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2002 by Information Technology Industry Council (ITI). All rights reserved. These materials are subject to copyright cla

3、ims of International Standardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Council (ITI). Not for resale. No part of this publication may be reproduced in any form, including an electroni

4、c retrieval system, without the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1250 Eye Street NW, Washington, DC 20005. Printed in the United States of America Reference number ISOIIEC 14496-2:2001 (E) O ISOIIEC 2001 ISO/IEC 14496-2:2001 (E) PD

5、F disclaimer This PDF file may contain embedded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, pa

6、rties accept therein the responsibility of not infringing Adobes licensing policy. The IS0 Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relativ

7、e to the file; the PDFcreation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by IS0 member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. O ISO/IEC

8、 2001 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from either IS0 at the address below or ISOs member body in the

9、country of the requester. IS0 copyright office Case postale 56 CH-1 21 1 Geneva 20 Tel. + 41 22 749 O1 11 Fax +41 227490947 E-mail copyrightiso.ch Web www.iso.ch Printed in Switzerland ii O ISOIIEC 2001 -All rights reserved ISOll EC 14496-2:2001 (E) Contents 1 2 3 4 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 5

10、 5.1 5.2 5.2.1 5.2.2 5.2.3 5.2.4 5.2.5 5.2.6 5.2.7 5.2.8 5.3 5.4 6 6.1 6.1.1 6.1.2 6.1.3 6.1.4 6.1.5 6.1.6 6.2 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.2.6 6.2.7 6.2.8 6.2.9 6.2.10 6.2.1 1 6.2.12 6.3 6.3.1 6.3.2 6.3.3 Scope 1 Normative references 1 Terms and definitions . 2 Abbreviations and symbols 12 Arith

11、metic operators . 12 Logical operators . 13 Relational operators 13 Bitwise operators . 13 Conditional operators . 13 Assignment 13 Mnemonics . 14 Constants . 14 Conventions . 14 Method of describing bitstream syntax . 14 Definition of functions . 15 Definition of next-bits() function . 15 Definitio

12、n of bytealigned0 function . 15 Definition of nextbits-bytealignecl() function . 15 Definition of next-start-code() function . 16 Definition of ne-resync-marker() function 16 Definition of transparent-mb() function 16 Definition of transparent-block() function 16 Definition of bytealign-for-upstream

13、() function 16 Arithmetic precision 17 Reserved. forbidden and marker-bit . 16 Visual bitstream syntax and semantics 17 Structure of coded visual data . 17 Visual object . 18 Video object . 18 Mesh object 24 FBA object 25 3D Mesh Object 29 Visual bitstream syntax . 30 Start codes . 30 Visual Object

14、Sequence and Visual Object . 34 Video Object Layer 35 Macroblock . 53 Block . 58 Still Texture Object 59 Mesh Object . 73 FBA Object . 75 3D Mesh Object 85 Upstream message 103 Visual bitstream semantics 104 Semantic rules for higher syntactic structures 104 Visual object sequence . 18 Group of Vide

15、o Object Plane 40 Video Object Plane and Video Plane with Short Header . 40 Visual Object Sequence and Visual Object . 104 Video Object Layer 109 O ISO/IEC 2001 -All rights reserved iii ISO/IEC 14496-2:2001 (E) 6.3.4 6.3.5 6.3.6 6.3.7 6.3.8 6.3.9 6.3.1 O 6.3.1 1 6.3.1 2 7 7.1 7.2 7.3 7.4 7.4.1 7.4.2

16、 7.4.3 7.4.4 7.4.5 7.4.6 7.5 7.5.1 7.5.2 7.5.3 7.5.4 7.5.5 7.5.6 7.6 7.6.1 7.6.2 7.6.3 7.6.4 7.6.5 7.6.6 7.6.7 7.6.8 7.6.9 7.6.10 7.7 7.7.1 7.7.2 7.8 7.8.1 7.8.2 7.8.3 7.8.4 7.8.5 7.8.6 7.8.7 7.9 7.9.1 7.9.2 7.10 7.10.1 7.10.2 7.10.3 7.10.4 7.10.5 7.10.6 Group of Video Object Plane 120 Video Object

17、Plane and Video Plane with Short Header . 120 Macroblock related 131 Block related 134 Still texture object 135 Mesh object 142 FBA object 144 3D Mesh Object 151 Upstream message 162 The visual decoding process . 164 Video decoding process . 165 Higher syntactic structures 166 VOP reconstruction . 1

18、66 Texture decoding . 166 Variable length decoding 167 Inverse scan . 168 DC and AC prediction for intra macroblocks 169 Inverse quantisation 172 Inverse DCT 175 Upsampling of the Inverse DCT output for Reduced Resolution VOP . 176 Shape decoding . 177 Higher syntactic structures 177 Macroblock deco

19、ding 178 Arithmetic decoding 187 Spatial scalable binary shape decoding 189 Grayscale Shape Decoding 198 Multiple Auxiliary Component Decoding 201 Motion compensation decoding . 201 Padding process 201 Sample interpolation for non-integer motion vectors 205 General motion vector decoding process . 2

20、07 Unrestricted motion compensation . 209 Vector decoding processing and motion-compensation in progressive P- and S(GMC)-VOP . 210 Overlapped motion compensation . 212 Temporal prediction structure 213 Vector decoding process of non-scalable progressive B-VOPs 214 Motion compensation in non-scalabl

21、e progressive B-VOPs 214 Motion Compensation Decoding of Reduced Resolution VOP 219 Interlaced video decoding 224 Field DCT and DC and AC Prediction 224 Motion compensation 225 Sprite decoding 234 Higher syntactic structures 234 Sprite Reconstruction . 235 Low-latency sprite reconstruction . 235 Spr

22、ite reference point decoding . 236 Warping . 237 Sample reconstruction 239 GMC decoding 240 Generalized scalable decoding 241 Temporal scalability 241 Spatial scalability . 246 Still texture object decoding . 251 Decoding of the DC subband . 251 ZeroTree Decoding of the Higher Bands . 252 Inverse Qu

23、antisation . 257 Still Texture Error Resilience 265 Wavelet Tiling . 268 Scalable binary shape object decoding 270 iv O ISO/IEC 2001 -All rights reserved ISO/IEC 14496-2:2001 (E) 7.1 1 7.11.1 7.1 1.2 7.12 7.12.1 7.12.2 7.12.3 7.12.4 7.12.5 7.12.6 7.12.7 7.13 7.13.1 7.13.2 7.13.3 7.1 3.4 7.13.5 7.13.

24、6 7.13.7 7.13.8 7.13.9 Mesh object decoding . 276 Mesh geometry decoding . 276 Decoding of mesh motion vectors . 279 FBA object decoding . 281 Frame based face object decoding 281 DCT based face object decoding . 282 Decoding of the viseme parameter fap 1 284 Decoding of the viseme parameter fap 2 2

25、84 Fap masking . 285 Frame Based Body Decoding . 285 DCT based body object decoding 286 3D Mesh Object Decoding 287 Start codes and bit stuffing 288 The Topological Surgery decoding process . 288 The Forest Split decoding process 291 Header decoder 292 partition type 293 Vertex Graph Decoder . 294 T

26、riangle Tree Decoder . 298 Triangle Data Decoder . 299 Forest Split decoder 303 7.1 3.1 O Arithmetic decoder 309 7.14 NEWPRED mode decoding . 314 7.14.1 Decoder Definition . 314 7.14.2 Upstream message 314 7.15 Output of the decoding process 314 7.15.1 Video data . 315 7.15.2 2D Mesh data 315 7.15.3

27、 Face animation parameter data 315 8 Visual-Systems Composition Issues . 315 8.1 Temporal Scalability Composition . 315 8.2 Sprite Composition 316 8.3 Mesh Object Composition 317 8.4 Spatial Scalability composition 318 9 Profiles and Levels 318 9.1 Visual Object Types . 318 9.2 Visual Profiles 321 9

28、.3 Visual ProfilesLevels . 322 9.3.1 Natural Visual . 322 9.3.2 Synthetic Visual . 322 9.3.3 Synthetic/Natural Hybrid Visual . 324 Annex A (normative) Coding transforms . 326 A.l Discrete cosine transform for video texture . 326 A.2 Discrete wavelet transform for still texture . 327 A.2.1 Adding the

29、 mean 327 A.2.2 Wavelet filter 327 A.2.3 Symmetric extension . 328 A.2.4 Decomposition level 329 A.2.5 Shape adaptive wavelet filtering and symmetric extension . 329 A.3 Shape-Adaptive DCT (SA-DCT) 330 A.3.1 Definition of Forward SA-DCT 330 A.3.2 Definition of Inverse SA-DCT . 332 A.4 SA-DCT with DC

30、 Separation and ADC Correction (ADC-SA-DCT) 333 A.4.1 Definition of Forward ADC-SA-DCT . 334 A.4.2 Definition of Inverse ADC-SA-DCT . 334 Annex B (normative) Variable length codes and arithmetic decoding . 336 B.l Variable length codes 336 B.l.l Macroblock type 336 O ISO/IEC 2001 -All rights reserve

31、d V ISO/IEC 14496-2:2001 (E) B.1.2 Macroblock pattern 338 B.1.3 Motion vector 340 B.1.4 DCT coefficients . 342 B.1.5 Shape Coding . 352 B.1.6 Sprite Coding . 357 B.1.7 DCT based facial object decoding . 358 B.1.8 Shape decoding for still texture object . 367 B.2 Arithmetic Decoding 368 B.2.1 Aritmet

32、ic decoding for still texture object 368 B.2.2 Arithmetic decoding for shape decoding 371 B.2.3 FBA Object Decoding 374 Annex C (normative) Face and body object decoding tables and definitions . 376 Annex D (normative) Video buffering verifier 409 D.l Introduction 409 D.2 Video Rate Buffer Model Def

33、inition 409 D.3 Comparison between ISO/IEC 14496-2 VBV and the ISO/IEC 13818-2 VBV (Informative) 412 D.4 Video Complexity Model Definition 413 D.5 Video Reference Memory Model Definition . 415 D.6 Interaction between VBV, VCV and VMV (informative) 416 D.7 Video Presentation Model Definition (informa

34、tive) 416 Annex E (informative) Features supported by the algorithm 418 E.l Error resilience . 418 E.l.l Resynchronization . 418 E.1.2 Data Partitioning 419 E.1.3 Reversible VLC . 419 E.1.4 Decoder Operation . 420 E.1.5 Adaptive Intra Refresh (AIR) Method . 423 E.1.6 NEWPRED . 425 E.2 Complexity Est

35、imation 427 E.3 Resynchronization in Case of Unknown Video Header Format 427 Annex F (informative) Preprocessing and postprocessing . 428 F.l VOP Generation Tools: Automatic and Semi-automatic Segmentations 428 F.l . 1 Automatic Segmentation 428 F.1.2 Semi-automatic Segmentation . 438 F.1.3 Referenc

36、es 446 F.2 Bounding Rectangle of VOP Formation 447 F.3 Postprocessing for Coding Noise Reduction . 448 F.3.1 Deblocking filter . 448 F.3.2 Deringing filter . 450 F.3.3 Further issues 452 F.4 Chrominance Decimation and Interpolation Filtering for Interlaced Object Coding 452 Annex G (normative) Profi

37、le and level indication and restrictions . 454 Annex H (informative) Patent statements 457 H.l Patent statements for ISO/IEC 14496 Version 1 . 457 H.2 Patent statements forthe extensions provided in ISO/IEC 14496 Version 2 458 Annex I (informative) Encoder Complexity Reduction Based on Intelligent P

38、re-Quantisation 460 1.1 Introduction 460 1.2 Feature Selection and Prequantisation 460 1.3 Model Verification and Threshold Setting . 462 1.3.1 H.263 Quantiser 462 1.3.2 MPEG-4 Quantiser . 462 Annex 5 (normative) View dependent object scalability . 464 5.1 Introduction 464 5.2 Decoding Process of a

39、View-Dependent Object . 464 5.2.1 General Decoding Scheme . 464 5.2.2 Computation of the View-Dependent Scalability parameters . 466 vi O ISO/IEC 2001 -All rights reserved ISO/IEC 14496-2:2001 (E) 5.2.3 VD mask computation . 468 5.2.4 Differential mask computation . 469 5.2.5 DCT coefficients decodi

40、ng . 469 5.2.6 Texture update . 469 5.2.7 IDCT . 470 Annex K (normative) Decoder Configuration Information . 471 K.l Introduction 471 K.2 Description of the set up of a visual decoder (informative) 471 K.2.1 Processing of decoder configuration information . 472 K.3 Specification of decoder configura

41、tion information 473 K.3.1 VideoObject 473 K.3.2 StillTextureObject 473 K.3.3 Meshobject 474 K.3.4 Faceobject . 474 K.3.5 3DMeshObject 474 Annex L (informative) Rate control 475 L.l Frame Rate Control . 475 L.l.l Introduction 475 L.1.2 Description . 475 L.1.3 Summary 479 L.2 Multiple Video Object Ra

42、te Control . 479 L.2.1 Initialization 479 L.2.2 Quantisation Level Calculation for I-frame and first P-frame . 479 L.2.3 Update Rate-Distortion Model 482 L.2.4 Post-Frameskip Control 482 L.3 Macroblock Rate Control 484 L.3.1 Rate-Distortion Model . 484 L.3.2 Target Number of Bits for Each Macroblock

43、 485 L.3.3 Macroblock Rate Control 485 Annex M (informative) Binary shape coding 487 M.l Introduction 487 M.2 Context-Based Arithmetic Shape Coding . 487 M.2.1 Intra Mode . 488 M.2.2 Inter Mode . 488 M.3 Texture Coding of Boundary Blocks . 489 M.4 Encoder Architecture 489 M.5 Encoding Guidelines . 4

44、90 M.5.1 Lossy Shape Coding . 491 M.5.2 Coding Mode Selection . 491 M.6 Conclusions . 491 M.7 References . 492 Annex N (normative) Visual profileslevels . 493 Annex O (informative) 3D Mesh Coding . 497 0.1 Introduction 497 0.2 Topological Surgery Representation . 497 0.2.1 Simple Polygon Representat

45、ion 498 0.2.2 Vertex Graph representation 499 0.3 Encoding guidelines for 3D Mesh Coding 500 0.3.1 Topological Surgery Encoding 500 0.3.2 Support for non-manifolds and Non-orientable manifolds . 501 0.3.3 Support for Error Resilience 503 0.4 Encoder considerations for efficient compression of Vertex

46、 Properties 507 0.5 Progressive Forest Split Representation 508 0.5.1 Encoding the Forest 508 0.5.2 Support for meshes with polygonal faces 509 0.5.3 Method for generating of a PFS Representation of a Triangular 3D Mesh 509 0.5.4 Topological Tests 510 O ISO/IEC 2001 -All rights reserved vi i ISO/IEC

47、 14496-2:2001 (E) 0.5.5 Geometric Tests . 512 0.6 Complexity estimation for Computational Graceful Degradation 512 0.7 QoS for SNHC through upstream . 514 0.8 References 516 Bibliography 517 viii O ISO/IEC 2001 -All rights reserved ISO/IEC 14496-2:2001 (E) Foreword IS0 (the International Organizatio

48、n for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of IS0 or IEC participate in the development of International Standards through technical committees established by the respectiv

49、e organization to deal with particular fields of technical activity. IS0 and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with IS0 and IEC, also take part in the work. In the field of information technology, IS0 and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3. The main task of the joint technical committee is to prepare International Standards. Draft Internationa