1、Digital Video Compression Fundamentals and Standards,Speaker: Wei-Yi Wei Advisor: Prof. Jian-Jung DingDigital Image and Signal Processing Lab GICE, National Taiwan University,2008/12/26,Digital Video Compression Fundamentals and Standards,2,Outline,Introduction Video Compression Standards Simulation
2、 Reference Software Future Work and Conclusions,2008/12/26,Digital Video Compression Fundamentals and Standards,3,Outline,Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions,2008/12/26,Digital Video Compression Fundamentals and Standards,4,Introduction
3、(1/2),Why video compression technique is important ? One movie video without compression 720 x 480 pixels per frame 30 frames per second Total 90 minutes Full color,The total quantity of data = 167.96 G Bytes !,2008/12/26,Digital Video Compression Fundamentals and Standards,5,Introduction (2/2),What
4、 is the difference between video compression and image compression? Temporal Redundancy Coding method to remove redundancy Intraframe Coding Remove spatial redundancy Interframe Coding Remove temporal redundancy,2008/12/26,Digital Video Compression Fundamentals and Standards,6,The most intuitive met
5、hod to remove Spatiotemporal redundancy,3-Dimensional DCT Remove spatiotemporal correlation Good for low motion video Bad for high motion video,2008/12/26,Digital Video Compression Fundamentals and Standards,7,The most popular method to remove temporal redundancy,The Block-Matching Algorithm,2008/12
6、/26,Digital Video Compression Fundamentals and Standards,8,Matching Function,The dissimilarity between two blocks andThe matching criteria Mean square error (MSE)High precision is needed Mean absolute difference (MAD)Low precision is enough,2008/12/26,Digital Video Compression Fundamentals and Stand
7、ards,9,The Exhaustive Block-Matching Algorithm,Reference Frame,Current Frame,Search Range,Motion Vector,11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88,12 2
8、2 33 44 55 66 77 88 12 22 33 44 55 66 77 88 12 22 33 44 55 66 77 88 12 22 33 44 55 66 77 88 11 23 34 44 55 66 77 88 11 23 34 44 55 66 77 88 11 23 34 44 55 66 77 88 11 23 34 44 55 66 77 88,1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0
9、0 1 1 0 0 0 0 0,|A|=12,2008/12/26,Digital Video Compression Fundamentals and Standards,10,Fast Block-Matching Algorithms,EBMA needs Intensive computationFast Algorithm is needed Find the possible local optimal,Reference Frame,Current Frame,2008/12/26,Digital Video Compression Fundamentals and Standa
10、rds,11,Fast Block-Matching Algorithms,The characteristics of fast algorithm Not accurate as EBMA Save large computation Two famous fast algorithm 2-D logarithm Search Method Three Steps Search Method,2008/12/26,Digital Video Compression Fundamentals and Standards,12,2-D logarithm Search Method,1,1,1
11、,1,1,2,2,2,3,3,4,4,4,4,4,1,2,3,4,2008/12/26,Digital Video Compression Fundamentals and Standards,13,Three Step Search Method,1,1,1,1,1,2,2,2,2,1,1,1,1,2,2,2,2,3,3,3,3,3,3,3,3,1,2,3,2008/12/26,Digital Video Compression Fundamentals and Standards,14,Multiresolution Motion Estimation (1/3),The number o
12、f levels is L l-th level images of the target frameswhere is set of pixels at level L At the l-th level, the MV is At the l-th level, the estimated MV is Determine update such that error is minimizedThe new motion vector is,2008/12/26,Digital Video Compression Fundamentals and Standards,15,Multireso
13、lution Motion Estimation (2/3),Variable block size method,2008/12/26,Digital Video Compression Fundamentals and Standards,16,Multiresolution Motion Estimation (3/3),2008/12/26,Digital Video Compression Fundamentals and Standards,17,Outline,Introduction Video Compression Standards Simulation Referenc
14、e Software Future Work and Conclusions,2008/12/26,Digital Video Compression Fundamentals and Standards,18,The Development of Video Compression Standards,2008/12/26,Digital Video Compression Fundamentals and Standards,19,The MPEG-1 Standard,Group of Pictures Motion Estimation Motion Compensation Diff
15、erential Coding DCT Quantization Entropy Coding,2008/12/26,Digital Video Compression Fundamentals and Standards,20,Group of Pictures (1/2),I-frame (Intracoded Frame) Coded in one frame such as DCT. This type of frame do not need previous frame P-frame (Predictive Frame) One directional motion predic
16、tion from a previous frame The reference can be either I-frame or P-frame Generally referred to as inter-frameB-frame (Bi-directional predictive frame) Bi-directional motion prediction from a previous or future frame The reference can be either I-frame or P-frame Generally referred to as inter-frame
17、,2008/12/26,Digital Video Compression Fundamentals and Standards,21,Group of Pictures (2/2),The distance between two nearest P-frame or P-frame and I-frame denoted by M The distance between two nearest I-frames denoted by N,2008/12/26,Digital Video Compression Fundamentals and Standards,22,The MPEG-
18、1 Encoder (1/4),DCT,Q,Entropy Coding,Q-1,IDCT,Motion Compensation,Frame Memory,Motion Estimation,DCT,Q,Entropy Coding,DCT,Q,Q-1,IDCT,Motion Compensation,Frame Memory,Motion Estimation,Intra-frame,Entropy Coding,Residue,Motion Vector,Inter-frame,2008/12/26,Digital Video Compression Fundamentals and S
19、tandards,23,The MPEG-1 Encoder (2/4),Differential Codingis the input image is the predictive image DCT,2008/12/26,Digital Video Compression Fundamentals and Standards,24,The MPEG-1 Encoder (3/4),Quantization Intra quantization matrix Inter quantization matrix,2008/12/26,Digital Video Compression Fun
20、damentals and Standards,25,The MPEG-1 Encoder (4/4),Motion Compensation Exploit motion vector and the previous reconstructed frame to generate the predictive frameis the compensated image is the previous imageis the motion vector,Reference frame,Target frame,2008/12/26,Digital Video Compression Fund
21、amentals and Standards,26,The MPEG-2 Standard,Field/Frame DCT Coding Field/Frame Prediction Mode Selection Alternative Scan Order Various Picture Sampling Formats User Defined Quantization Matrix,2008/12/26,Digital Video Compression Fundamentals and Standards,27,Progressive Scan and Interlaced Scan,
22、Progressive Scan,Interlaced Scan,2008/12/26,Digital Video Compression Fundamentals and Standards,28,Field/Frame DCT Coding,The field type DCT Fast motion video The frame type DCT Slow motion video,2008/12/26,Digital Video Compression Fundamentals and Standards,29,Alternative Scan Order,Zigzag scan o
23、rder Frame DCT Alternative scan order Field DCT,2008/12/26,Digital Video Compression Fundamentals and Standards,30,The MPEG-2 Encoder (2/2),Quantization User can change the quantization if necessary Intra quantization matrix Inter quantization matrixVarious picture sampling formats 4:4:4 4:2:2 4:2:0
24、,2008/12/26,Digital Video Compression Fundamentals and Standards,31,The MPEG-2 Encoder (1/2),Base Layer Basic quality requirement For SDTV Enhanced Layer High quality service For HDTV,DCT,Q,Entropy Coding,Q-1,IDCT,Motion Compensation,Frame Memory,Motion Estimation,Q,Entropy Coding,Q-1,+,+,+,+,+,Q-1,
25、Bits Enhance,SNR Enhanced Layer,+,Base Layer,Bits Base,2008/12/26,Digital Video Compression Fundamentals and Standards,32,H.264/AVC,Variable Block Size Multiple Reference Frames Integer Transform Intra Prediction In-loop Deblocking Filtering 1/4-pel Resolution Motion Estimation CAVLC,2008/12/26,Digi
26、tal Video Compression Fundamentals and Standards,33,Variable Block Size,The fixed block size may not be suitable for all motion objects Improve the flexibility of comparison Reduce the error of comparison 7 types of blocks for selection,2008/12/26,Digital Video Compression Fundamentals and Standards
27、,34,Multiple Reference Frames,The neighboring frames are not the most similar in some cases The B-frame can be reference frame B-frame is close to the target frame in many situations,2008/12/26,Digital Video Compression Fundamentals and Standards,35,Integer Transform for Reducing The Spatial Redunda
28、ncy (1/2),The transform matrix C44 Block Size Separable Integer Transform The transform coefficients are CXCT,2008/12/26,Digital Video Compression Fundamentals and Standards,36,Integer Transform for Reducing The Spatial Redundancy (2/2),C,X,CT,E,2008/12/26,Digital Video Compression Fundamentals and
29、Standards,37,Intra Prediction,Predict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform coding to remove the redundancy.,Vertical right,2008/12/26,Digital Video Compression Fundamentals and Standards,38,Remove Perceptual Redundancy,In-lo
30、op deblocking filtering Remove blocking artifact Result from block based motion compensation Result from block based transform coding,QP,2008/12/26,Digital Video Compression Fundamentals and Standards,39,1/4-pel Resolution Motion Estimation,2008/12/26,Digital Video Compression Fundamentals and Stand
31、ards,40,The H.264/AVC Encoder,Transform/ Quantization,Entropy Coding,Inverse Transform/ De-Quantization,Motion Compensation,De-blocking Filter,Motion Estimation,Residue,Motion Vector,Intra-frame Prediction,Coder Controller,Control Data,2008/12/26,Digital Video Compression Fundamentals and Standards,
32、41,Outline,Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions,2008/12/26,Digital Video Compression Fundamentals and Standards,42,H.264 Reference Software,JM Current software version: JM 15 Benchmark,http:/iphome.hhi.de/suehring/tml/,2008/12/26,Digital
33、Video Compression Fundamentals and Standards,43,Reference Software Demo,2008/12/26,Digital Video Compression Fundamentals and Standards,44,Outline,Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions,2008/12/26,Digital Video Compression Fundamentals and
34、Standards,45,Future Work,Fast Mode Decision Algorithm Interpolation Filter Design Deblocking Filter Design DCT-Based Motion Estimation Implementation Based on TI DSP,2008/12/26,Digital Video Compression Fundamentals and Standards,46,Fast Mode Decision Algorithm,The computational cost of H.264 is lar
35、ge Variable block-size ME Multiple reference frames Fast mode decision is needed for reducing the computation time,2008/12/26,Digital Video Compression Fundamentals and Standards,47,Interpolation Filter Design,In order to estimate and compensate the fractional-pel displacements Adaptive Interpolatio
36、n filter for replacing the fixed coefficient filter,2008/12/26,Digital Video Compression Fundamentals and Standards,48,Deblocking Filter Design,Block based ME and Transform result in the annoying blocking artifact Reduce the blocking artifact can increase the quality of compressed video,2008/12/26,D
37、igital Video Compression Fundamentals and Standards,49,DCT Based Motion Estimation,Robust even in noisy environment Complexity comparison DCT Based ME O(M2) Block Based ME O(N2M2) Concept Pseudo Phase Similar to,2008/12/26,Digital Video Compression Fundamentals and Standards,50,Implementation Based
38、on TI DSP,TMS320C6416TI DM642,2008/12/26,Digital Video Compression Fundamentals and Standards,51,Conclusions,Thank You Q & A,2008/12/26,Digital Video Compression Fundamentals and Standards,52,References (1/2),1 Yun Q.Shi and Huifang Sun, “Image and Video Compression for Multimedia Engineering: Funda
39、mentals, Algorithms, and Standards”, CRC press, 2000. 2 Yao Wand, Jorn Ostermann and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2007. 3 Richardson, Lain E. G., “Video Codec Design: Developing Image and Video Compression Systems”, John Wiley & Sons Inc, 2002. 4 Barry G, Haske
40、ll, Atul Puri and Arun N. Netravali, “Digital Video : An Introduction to MPEG-2”, Boston : Kluwer Academic, 1999. 5 T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard”, IEEE Trans. on Circuits and systems for video Technology, vol. 13, no. 7,
41、pp. 560-576, July 2003. 6 G. Sullivan and T. Wiegand, “Video Compression - From Concepts to the H.264/AVC Standard”, Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, December 2004. 7 酒井善則、吉田俊之 共著，白執善 編譯，“影像壓縮技術”，全華，2004.,2008/12/26,Digital Video Compression Fundamenta
42、ls and Standards,53,References (2/2),8 Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding”, IEEE Trans. on Circuits and Systems For Video Technology, vol. 9 Dong-Hwan Kim, Hwa-Yong Oh, Oguzhan Urhan, Sarp Ertrk and Tae-Gyu Chang, “Optimal Post-Process/In-
43、Loop Filtering for Improved Video Compression Performance”, IEEE Trans. on Consumer Electronics, vol. 53, no. 4, Nov. 2007. 10 Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, ” Fast Multi-Frame Motion Estimation for H.264 and Its Applications to Complexity-Aware Streaming”, IEEE International Symposium on Circuits and Systems, 2005. 11 Kai-Ting Cheng and Hsueh-Ming Hang, “Acceleration and Implementation of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP Platform”, master thesis, June 2007,
