1、 TSB-169 April 2012cdma2000 Multimedia Services Evaluation Methodology NOTICE TIA Engineering Standards and Publications are designed to serve the public interest through eliminating misunderstandings between manufacturers and purchasers, facilitating interchangeability and improvement of products,
2、and assisting the purchaser in selecting and obtaining with minimum delay the proper product for their particular need. The existence of such Standards and Publications shall not in any respect preclude any member or non-member of TIA from manufacturing or selling products not conforming to such Sta
3、ndards and Publications. Neither shall the existence of such Standards and Publications preclude their voluntary use by Non-TIA members, either domestically or internationally. Standards and Publications are adopted by TIA in accordance with the American National Standards Institute (ANSI) patent po
4、licy. By such action, TIA does not assume any liability to any patent owner, nor does it assume any obligation whatever to parties adopting the Standard or Publication. This Standard does not purport to address all safety problems associated with its use or all applicable regulatory requirements. It
5、 is the responsibility of the user of this Standard to establish appropriate safety and health practices and to determine the applicability of regulatory limitations before its use. (From Project No. 3-0241-A, formulated under the cognizance of the TIA TR-45 Mobile (b) there is no assurance that the
6、 Document will be approved by any Committee of TIA or any other body in its present or any other form; (c) the Document may be amended, modified or changed in the standards development or any editing process. The use or practice of contents of this Document may involve the use of intellectual proper
7、ty rights (“IPR”), including pending or issued patents, or copyrights, owned by one or more parties. TIA makes no search or investigation for IPR. When IPR consisting of patents and published pending patent applications are claimed and called to TIAs attention, a statement from the holder thereof is
8、 requested, all in accordance with the Manual. TIA takes no position with reference to, and disclaims any obligation to investigate or inquire into, the scope or validity of any claims of IPR. TIA will neither be a party to discussions of any licensing terms or conditions, which are instead left to
9、the parties involved, nor will TIA opine or judge whether proposed licensing terms or conditions are reasonable or non-discriminatory. TIA does not warrant or represent that procedures or practices suggested or provided in the Manual have been complied with as respects the Document or its contents.
10、If the Document contains one or more Normative References to a document published by another organization (“other SSO”) engaged in the formulation, development or publication of standards (whether designated as a standard, specification, recommendation or otherwise), whether such reference consists
11、of mandatory, alternate or optional elements (as defined in the TIA Engineering Manual, 4thedition) then (i) TIA disclaims any duty or obligation to search or investigate the records of any other SSO for IPR or letters of assurance relating to any such Normative Reference; (ii) TIAs policy of encour
12、agement of voluntary disclosure (see Engineering Manual Section 6.5.1) of Essential Patent(s) and published pending patent applications shall apply; and (iii) Information as to claims of IPR in the records or publications of the other SSO shall not constitute identification to TIA of a claim of Esse
13、ntial Patent(s) or published pending patent applications. TIA does not enforce or monitor compliance with the contents of the Document. TIA does not certify, inspect, test or otherwise investigate products, designs or services or any claims of compliance with the contents of the Document. ALL WARRAN
14、TIES, EXPRESS OR IMPLIED, ARE DISCLAIMED, INCLUDING WITHOUT LIMITATION, ANY AND ALL WARRANTIES CONCERNING THE ACCURACY OF THE CONTENTS, ITS FITNESS OR APPROPRIATENESS FOR A PARTICULAR PURPOSE OR USE, ITS MERCHANTABILITY AND ITS NONINFRINGEMENT OF ANY THIRD PARTYS INTELLECTUAL PROPERTY RIGHTS. TIA EX
15、PRESSLY DISCLAIMS ANY AND ALL RESPONSIBILITIES FOR THE ACCURACY OF THE CONTENTS AND MAKES NO REPRESENTATIONS OR WARRANTIES REGARDING THE CONTENTS COMPLIANCE WITH ANY APPLICABLE STATUTE, RULE OR REGULATION, OR THE SAFETY OR HEALTH EFFECTS OF THE CONTENTS OR ANY PRODUCT OR SERVICE REFERRED TO IN THE D
16、OCUMENT OR PRODUCED OR RENDERED TO COMPLY WITH THE CONTENTS. TIA SHALL NOT BE LIABLE FOR ANY AND ALL DAMAGES, DIRECT OR INDIRECT, ARISING FROM OR RELATING TO ANY USE OF THE CONTENTS CONTAINED HEREIN, INCLUDING WITHOUT LIMITATION ANY AND ALL INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES (INC
17、LUDING DAMAGES FOR LOSS OF BUSINESS, LOSS OF PROFITS, LITIGATION, OR THE LIKE), WHETHER BASED UPON BREACH OF CONTRACT, BREACH OF WARRANTY, TORT (INCLUDING NEGLIGENCE), PRODUCT LIABILITY OR OTHERWISE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE FOREGOING NEGATION OF DAMAGES IS A FUNDAMENT
18、AL ELEMENT OF THE USE OF THE CONTENTS HEREOF, AND THESE CONTENTS WOULD NOT BE PUBLISHED BY TIA WITHOUT SUCH LIMITATIONS. 3GPP2 C.R1008-0 v2.0 Revision History Revision Description of Changes Date Rev 0 v1.0 Initial Publication January 2007 Rev 0 v2.0 Removal of non-public video test sequences The ch
19、ange marks are not displayed in this version. December 2011 3GPP2 C.R1008-0 v2.0 i Contents 1 Foreword.iv 2 1 Introduction 1 3 1.1 Scope 1 4 1.2 Requirements Language . 1 5 1.3 Informative References 1 6 1.4 Definitions, Symbols and Abbreviations 2 7 1.4.1 Definitions . 3 8 1.4.2 Symbols and Abbrevi
20、ations . 3 9 2 Source Material 5 10 2.1 Video 5 11 2.1.1 3GPP2 Database Sequence 1: Little Bugs . 5 12 2.1.2 3GPP2 Database Sequence 2: Large Bugs 5 13 2.2 Speech 6 14 2.3 Audio 6 15 3 Codec . 7 16 3.1 General Description . 7 17 3.2 Speech 7 18 3.3 Video 7 19 3.4 Audio 7 20 4 Channel 9 21 4.1 Dedica
21、ted Channel . 9 22 4.1.1 Description of Traces 9 23 4.1.2 Simulation Methodology . 9 24 4.2 HRPD Shared Channels . 11 25 4.2.1 Reverse-link Model . 12 26 4.2.2 Forward-link Model 15 27 4.2.3 End-to-end Channel Model for VoIP 18 28 4.3 Broadcast Channel . 21 29 4.3.1 Simulation Procedure 21 30 5 Simu
22、lator 23 31 5.1 Network Simulator . 23 32 5.2 S/W Usage Description 23 33 5.3 Simulator Description 23 34 5.3.1 Frame Dropping Modification . 23 35 5.3.2 Support for Multiple PDU Sizes 24 36 5.3.3 Forbidden Bit Mode for Error Detection . 24 37 5.3.4 Timing Modification for Multimedia 25 38 6 Quality
23、 Evaluation . 31 39 6.1 Traditional Objective Measure . 31 40 6.2 Objective Metrics For Error-prone Conditions 31 41 6.2.1 Percentage Degraded Video Duration (pDVD) . 31 42 6.2.2 Average PSNR in Clean and Error-Propagated Durations 32 43 6.2.3 Standard Deviation of PSNR . 33 44 7 Services 35 45 7.1
24、Packet Switched Video Telephony (PSVT) . 35 46 7.2 Multimedia Streaming Service (MSS) . 35 47 7.3 Broadcast Multicast Service (BCMCS) . 35 48 7.4 Voice over IP (VoIP) . 35 49 50 3GPP2 C.R1008-0 v2.0 ii List of Figures 1 Figure 1 Interlace Structure and RTP Packet Fragmentation in the HRPD Reverse-li
25、nk Model 13 2 Figure 2 Token Bucket Mechanism in the HRPD Reverse-link Model . 14 3 Figure 3 Probabilistic Model for Early Termination Determination . 15 4 Figure 4 Interlace Structure and RTP Packet Fragmentation in the HRPD Forward-Link Model 17 5 Figure 5 Scheduling Delay Determination in the For
26、ward-Link Model 18 6 Figure 6 Network Simulator Operation. 24 7 Figure 7 Test Setup for Multimedia Clients 25 8 Figure 9 Testing Methodology for VoIP Codecs . 27 9 Figure 10 Descriptive Example NCIM Input File . 28 10 Figure 11 Example of PSNR Trace in Error-Prone Environment 32 11 12 13 3GPP2 C.R10
27、08-0 v2.0 iii List of Tables 1 Table 1 Propagation Channel Models 10 2 Table 2 Multipath Profiles . 11 3 Table 3 Reverse-Link Packet Index and Application Payload Size . 14 4 Table 4 DRC Index and the Corresponding Span and Application Payload Size 17 5 Table 5 Format of Input Files 20 6 Table 6 For
28、mat of Encoded Speech File 26 7 Table 7 Format of VoIP Client Output File . 30 8 9 10 3GPP2 C.R1008-0 v2.0 iv Foreword 1 (This foreword is not part of this document.) 2 This technical report provides common simulation conditions for cdma20001multimedia services 3 such as Broadcast Multicast Services
29、 (BCMCS), Packet Switched Video Telephony/ Multimedia 4 Conversational Services (PSVT/MCS), Multimedia Messaging Services (MMS) 18 and Multimedia 5 Streaming Services (MSS) 19. The common simulation conditions include source material, pointers 6 to multimedia codecs used in cdma2000 multimedia servi
30、ces, an abstraction of layers below Real 7 Time Transport Protocol (RTP) in wireless Internet Protocol (IP) networks, and suitable objective 8 metrics. 9 .10 1cdma2000is the trademark for the technical nomenclature for certain specifications and standards of the Organizational Partners (OPs) of 3GPP
31、2. Geographically (and as of the date of publication), cdma2000 is a registered trademark of the Telecommunications Industry Association (TIA-USA) in the United States. 3GPP2 C.R1008-0 v2.0 1 1 Introduction 1 The purpose of this technical report is to document the simulation methodology for cdma2000
32、 2 multimedia services. The simulations are designed to capture performance characteristics of source 3 codecs, including appropriate RTP packetization schemes used for cdma2000 multimedia services. 4 The layers below the RTP layer are abstracted in a software tool with RTP packets as input, RTP 5 p
33、ackets as output and appropriate control/configuration parameters to capture packet loss and delay 6 behavior of a given wireless IP network. Appropriate metrics to capture user experience under packet 7 losses and variable delays are also provided. Common source material is proposed for use in the
34、8 simulations. 9 1.1 Scope 10 The scope of this technical report is to provide common simulation conditions to enable 11 repeatable evaluations of source coding and RTP packetization options. It is possible to perform cross 12 verifications. The document includes 13 Common source material (e.g., vid
35、eo clips) 14 Pointers to multimedia codecs used in cdma2000 multimedia services, including technical 15 descriptions and reference software when available 16 An abstraction of layers below RTP in wireless IP networks 17 Suitable objective metrics 18 1.2 Requirements Language 19 “Shall” and “shall no
36、t” identify requirements to be followed strictly to conform to this document and 20 from which no deviation is permitted. “Should” and “should not” indicate that one of several 21 possibilities is recommended as particularly suitable, without mentioning or excluding others, that a 22 certain course
37、of action is preferred but not necessarily required, or that (in the negative form) a 23 certain possibility or course of action is discouraged but not prohibited. “May” and “need not” 24 indicate a course of action permissible within the limits of the document. “Can” and “cannot” are used 25 for st
38、atements of possibility and capability, whether material, physical or causal. 26 1.3 Informative References 27 The following standards are referenced in this text. At the time of publication, the editions indicated 28 were valid. All standards are subject to revision, and parties to agreements based
39、 upon this document 29 are encouraged to investigate the possibility of applying the most recent editions of the standards 30 indicated below. American National Standards Institute (ANSI) and TIA maintain registers of 31 currently valid national standards published by them. 32 3GPP2 C.R1008-0 v2.0 2
40、 1 ITU-T VCEG-M77, Common Test Conditions for RTP/IP over 3GPP/3GPP2. 1 2 Parag Agashe, et al, cdma2000 High Rate Broadcast Packet Data Air Interface Design, IEEE 2 Communication Magazine, February 2004. 3 3 3GPP2 C.S0024, cdma2000 High Rate Packet Data Air Interface Specification. 4 4 3GPP2 C.R1002
41、, cdma2000 Evaluation Methodology. 5 5 Reserved. 6 6 3GPP2 C.S0014-B v1.0, Enhanced Variable Rate Codec, Speech Service Option 3 and 68 for 7 Wideband Spread Spectrum Digital Systems, May 2006. 8 7 3GPP2 C.S0052-A v1.0, Source-Controlled Variable-Rate Multimode Wideband Speech 9 Codec (VMR-WB), Serv
42、ice Options 62 and 63 for Spread Spectrum Systems, April 2005. 10 8 3GPP2 C.S0020-A v1.0, High Rate Speech Service Option 17 for Wide Band Spread Spectrum 11 Communication Systems, April 2004. 12 9 3GPP2 C.R1009, cdma2000 Multimedia Services Evaluation Methodology: Software Tools. 13 10 3GPP2 C.S005
43、4-0 v2.0, cdma2000 High Rate Broadcast-Multicast Packet Data Air Interface 14 Specification, August 2005. 15 11 3GPP2 C.S0054-A v1.0, cdma2000 High Rate Broadcast-Multicast Packet Data Air Interface 16 Specification, March 2006. 17 12 ITU-T P.800, Methods for subjective determination of transmission
44、 quality. 18 13 J.P. Egan, Articulation Testing Methods, Laryngoscope, Vol. 58, No. 9, 955-991, 1948. 19 14 ITU-T Recommendation H.263, Video Coding for Low Bitrate Communication. 20 15 ISO/IEC 14496-2:2004, Information Technology Generic Coding of Audio-Visual Object 21 Part 2: Visual. 22 16 ITU-T
45、Recommendation H.264 (2003), Advanced video coding for generic audiovisual 23 services, or ISO/IEC 14496-10:2003, Information technology Coding of audio-visual objects 24 Part 10: Advanced Video Coding. 25 17 ISO/IEC 14496-3:2005, Information technology - Coding of audio-visual objects - Part 26 3:A
46、udio. 27 18 3GPP2 C.S0045-A v1.0, Multimedia Messaging Service (MMS) Media Format and Codecs for 28 cdma2000 Spread Spectrum Systems, April 2006. 29 19 3GPP2 C.S0046-0 v1.0, 3G Multimedia Streaming Services, March 2006. 30 1.4 Definitions, Symbols and Abbreviations 31 This section contains definitio
47、ns, symbols and abbreviations that are used throughout the document. 32 3GPP2 C.R1008-0 v2.0 3 1.4.1 Definitions 1 codec: a system component that encodes and decodes data (usually audio, video, etc.) from one 2 representation to another, often with the goal of saving memory space or transmission ban
48、dwidth 3 (compression). 4 multimedia: a combination of multiple media elements used in a service to enrich the user experience. 5 1.4.2 Symbols and Abbreviations 6 3GPP2 Third Generation Partnership Project 2 7 AAC Advanced Audio Coding 8 ARQ Automatic Repeat Request 9 AT Access Terminal 10 BCMCS Br
49、oadcast-Multicast Services 11 CIF Common Intermediate Format 12 C/I Carrier to Interference (Ratio) 13 DCCH Dedicated Control Channel 14 DRC Data Rate Control 15 EVRC Enhanced Variable Rate Codec 16 FER Frame Error Rate 17 FL Forward Link 18 HE AAC High Efficiency Advanced Audio Coding 19 HRPD High Rate Packet Data 20 IP Internet Protocol 21 ITU International Telecommunications Union 22 LSB Least Significant Bit 23 MAC Medium Access Control 24 MCS Multimedia Conversational Services 25 MMS Multimedia Messaging Service 26 MPEG Motion Pictu
