1、ETSI TR 126 91 2 V4.0.0 (2001-03) Technical Report Universal Mobile Telecommunications System (UMTS); QoS for Speech and Multimedia Codec; Quantitative performance evaluation of H.324 Annex C over 3G (3GPP TR 26.912 version 4.0.0 Release 4) COPYRIGHT European Telecommunications Standards InstituteLi
2、censed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 1 Reference RTWTSGS-0426912Uv4 Keywords UMTS ETSI 650 Route des Lucioles F-O6921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 O0 Fax: +33 4 93 65 47 16 Siret No 348 623 562 O0017 - NAF 742 C Association but non luc
3、ratif enregistre la Sous-prfecture de Grasse (06) No 7803/88 Important notice ETSI TR 126 912 V4.0.0 (2001-03) Individual copies of the present document can be downloaded from: htttx/w.etsi. org The present document may be made available in more than one electronic version or in print. In any ase of
4、 existing c perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF). In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present
5、document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at hi%:i/wc%lw.esi.or/b/slatis/ If you find errors in the present document, send your comment to: ed itoretsi .fr Copyright No ti
6、fication No part may be reproduced except as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in all media. O European Telecommunications Standards Institute 2001. All rights reserved. ETSI COPYRIGHT European Telecommunications Standards InstituteL
7、icensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 2 ETSI TR 126 912 V4.0.0 (2001-03) Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, i
8、s publicly available for ETSI members and non-members, and can be found in ETSI SR O00 3 14: “Intellectual Property Rights (IPRs); Essential, orpotentially Essential, IPRs notjed to ETSI in respect OfETSIstandards“, which is available from the ETSI Secretariat. Latest updates are available on the ET
9、SI Web server (-). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR O00 3 14 (or the updates on the ETSI Web server) which are, or may be, or may become, esse
10、ntial to the present document. Foreword This Technical Report (TR) has been produced by the ETSI 3d Generation Partnership Project (3GPP). The present document may refer to technical specifications or reports using their 3GPP identities, UMTS identities or GSM identities. These should be interpreted
11、 as being references to the corresponding ETSI deliverables. The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under www.etsi.ordkey . ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Releas
12、e 4 3 ETSI TR 126 912 V4.0.0 (2001-03) Contents Foreword 5 1 Scope . 6 2 References . 6 3 Definitions and abbreviations 6 Definitions . . 6 3.1 3.2 Abbreviations . 7 4 3GPP Configuration of H.324 annex C . 8 5 Perfomiance . 9 Audio . 9 5.1 . 5.1.1 Introduction . 9 5.1.2 Results . 9 5.2 Video . . 11
13、5.2.1 Introduction . 11 5.2.2 Test environment . . 11 5.2.3 Results . 12 5.2.3.1 Subjective results . 12 5.2.3.2 Objective results . 14 . . Annex A: Quality degradation as a function of the FER and RBER in presence of background noise . 17 A . 1 A.2 A.3 Results in Car Noise: . 17 Results in Street N
14、oise: . 18 Results in Office Noise 19 Annex B: B . 1 Introduction . 20 B.2 Test Procedure . 21 Simulation test of a video multimedia codec 20 B.2.1 B.2.2 B.2.3 B.2.3.1 B.2.3.2 B.2.4 B.2.5 B.2.5.1 B.2.5.2 B.2.5.3 B.2.6 B.2.7 B.2.8 B.2.8.1 B.2.8.1.1 B.2.9 B.2.9.1 B.2.9.2 B.2.9.3 B.2.9.4 B.2.9.4.1 B.2.
15、9.4.2 B.2.9.4.3 . B.2.9.4.4 I I“ L . 21 Simulation Model Source materials . 22 Source encoding . 22 Speech . 22 Video . 23 Multiplexing . 23 Bit error injection . 23 Error pattern files . 23 Error pattern segments to be used at the simul . 24 Injection of bit errors . . 25 De-multiplexing . . 25 Vid
16、eo decoding . 25 Constraints and regulations . . 25 . 25 Video . . 26 Statistical data to be reported . 26 Video coding bitrate %6, 2 verhead %6, 2fkbpsl 26 Speech coding bitrate %6, 2f kbps and frame length %d ms 26 Video initial delay %6, lfms . 26 PSNR related data . 26 Total average PSNR, i.e.,
17、PSNRtOtal %6.2f dB 26 . Delay Average PSNR for representative run, i.e., PSNRk* %6.2f dB . 26 Average PSNR in error-free case, i.e., PSNR?,., %6.2f dB . 27 Standard deviation of PSNR . i.e Sigma r%6.2f dB1 27 ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handl
18、ing Services3GPP TR 26.912 version 4.0.0 Release 4 ETSI TR 126 912 V4.0.0 (2001-03) 4 B.2.9.5 B.2.9.6 B.2.9.7 B.2.9.8 B.2.9.9 Coding frame rate %5.2f framedsec 27 Average dropframe rate %6.2f % . 27 Out of delay constraints rate %6.2f % 27 Definition of video stationary delay 27 Decoded video of rep
19、resentative run 28 B.3.1 B.3.1.1 B.3.1.2 B.3.1.3 B.3.1.4 B.3.1.5 B.3.2 B.3.2.1 B.3.2.2 B.3.3 B.3.3.1 B.3.3.2 B.3.3.3 B.3.4 B.3.4.1 B.3.4.2 B.3 Subjective quality evaluation 28 Structure of test 28 Program . 28 Training session 28 Scoring session . 28 Video sequence . 28 Structure of program . 29 Edi
20、ting process . 29 Producing training session 29 Randomization 29 Assessment . 29 Test subjects 29 Facilities and equipment for test . 30 Score sheet 30 Data processing 30 MOS %4.2fl 31 Standard deviation of OS, i.e., oOs %4.2 fl 31 . B.4 Test results and observations . 31 B.4.1 Test results 31 B.4.2
21、 Observations 31 B.5 List of videohpeech codecs and multiplexers employed in the simulation . 32 B.6 Test results . 33 Annex C: Change history . 38 ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 5 ETSI TR
22、126 912 V4.0.0 (2001-03) Foreword This Technical Report has been produced by the 3d Generation Partnership Project (3GPP). The contents of the present document are subject to continuing work within the TSG and may change following formal TSG approval. Should the TSG modiSl the contents of the presen
23、t document, it will be re-released by the TSG with an identiSling change of release date and an increase in version number as follows: Version x.y.z where: x the first digit: 1 presented to TSG for information; 2 presented to TSG for approval; 3 or greater indicates TSG approved document under chang
24、e control. y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections, updates, etc. z the third digit is incremented when editorial only changes have been incorporated in the document. ETSI COPYRIGHT European Telecommunications Standards InstituteLicens
25、ed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 6 ETSI TR 126 912 V4.0.0 (2001-03) 1 Scope The present document is meant to function as guidance in the work of other 3GPP work groups or work items. Such work may include conclusion on how to achieve detailed Stage 1 service
26、requirements or suggestion of a set of recommended RAB parameters giving satisfactory user-to-user quality for a circuit switched multimedia service using 3G-324M. 2 Re fe re nces The following documents contain provisions which, through reference in this text, constitute provisions of the present d
27、ocument. References are either specific (identified by date of publication, edition number, version number, etc.) or non-specific. For a specific reference, subsequent revisions do not apply. For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (inc
28、luding a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same Release as theyresent document. il “Volume 3; Specifications of air-interface for 3G mobile system (Version 1 .O-0.3)“, IMT-2000 Study Committee, Air-Interface WG/SWG2, Nov. 1 Sth, 1
29、998. “Volume 8; Codec specification for use in a 3G mobile system (Version 0.5.2)“, IMT-2000 Study Committee, Codec WG (ARIB), July 2lSt, 1998. International Standard ISO/IEC 14494-2: “Information technology - Generic coding of audio- visual object - Part 2: Visual, 1999“. ITU-T Recommendation H.263
30、 : “Video coding for low bit rate communication“. ITU-T Recommendation H.245: “Control protocol for multimedia communication“ ITU-T Recommendation H.223 : “Multiplexing protocol for low bitrate multimedia communication“. 21 31 41 51 61 71 SI ITU-T Recommendation H.324: “Terminal for low bit rate mul
31、timedia communication“. 3GPP TR 21.905: “3G Vocabulary“. 91 3GPP TR 26.975: “Performance characterization of the AMR speech codec Version 1.1.0. Release 1999“. 3 3.1 Definitions and abbreviations De fi nit ions For the purposes of the present document, the following terms and definitions apply. Code
32、c: a single media codeddecoder, or a multimedia system specific coder ETSI TR 126 912 V4.0.0 (2001-03) Video Codec D H.263, MPEG-4, 8 I I 4 I C,H.223 3GPP Configuration of H.324 annex C d a; Video I/O Equipment Data Protocols D V. 14, LAFM, I Audio I/O Equipment - 4 User Data Application S - H.245 -
33、b CCSRL -b NSRP LAPMN.4 4- Syste m Contro 1 Scope of TS 26.11 1 . Multiplex/ Demultiplex H.223. H.223 Annex A, H.223 Annex B, H.223 Annex Annex Dl * 4 . i Scope of TS 26.1 12 Figure 1 3GPP Network Figure 1 shows the main building block of a 3G-324M terminal. The standards inside . . . are not mandat
34、ory. The configuration actually used for each of the four performances evaluations (Audio, Video, Control and Data) will be described under each heading. ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 9 ET
35、SI TR 126 912 V4.0.0 (2001-03) 5 5.1 Performance Audio 5.1.1 I n trod uction This clause provides a subset of the AMR Characterisation test results expressed as speech quality degradation (in AMOS or ADMOS) compared to the EFR speech codec in error free conditions, as a function of the FER (Frame Er
36、ror Rate) and RBER( Residual Bit Error Rate as defined for GSM). It is believed that these results would also apply to H.324M channel with equivalent error conditions. Additional results are provided in Annex A for test conditions under background noise (Car noise, Street noise and Office noise). Th
37、e original test results are included in the AMR characterisation report (3GPP TR 26.975 9). They relate to a channel condition GSM TU3 IFH (Typical Urban 3 kdh Ideal Frequency Hopping). These results could be updated once the AMR 3G Characterisation tests are completed. Quality performances of audio
38、 codecs in H.324M channels should be included in future versions of this document, as these results become available. 5.1.2 Results The following diagrams present the speech quality degradation in clean speech (expressed in AMOS) for the different AMR codec modes, as a function of the FER and RBER,
39、when compared to the EFR speech codec in error free condition. In all cases, the results represent the average scores obtained over all tests performed for each experiment as compiled in the GSM AMR Characterization report (3GPP TR 26.975 9). The EFR reference is taken from the score obtained by the
40、 EFR speech codec in error free in the same experiment. The actual results were slightly altered to smoothen the curves shape. Finally, it should also be noted that the diagrams function of the FER are actually affected by the Residual Bit Error Rate for each test condition, while the diagrams funct
41、ion of the RBER are also function of the FER present for each test condition. The two sets of diagrams cannot be considered totally independent. They are a reflection of the channel coding scheme selected for the GSM radio channels. Finally, it should be pointed out that the FER and RBER estimates u
42、sed to derive these diagrams are based on the limited number of error patterns used for the AMR characterization phase. These could be affected by some inaccuracies that could explain the difference in shapes between the different speech codec modes. WARNING: These results are representative of the
43、test conditions used for the GSM AMR characterization phase and may not be representative of the codec performances in other test conditions. When analyzing the original experiments, it was usually found that a difference in MOS lower than 0.2 was not statistically significant. For the following res
44、ults, the confidence interval should also be increased by the uncertainty introduced in the estimation of the FER or RBER. ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 10 ETSI TR 126 912 V4.0.0 (2001-03)
45、 Perceived aualitv(M0S) dearadation as afunction ofthe AR A 3 50 3 O0 3 50 1 O0 150 2 O0 2 50 3 O0 0001% o 010% o 100% 1000% 1 o 000% 100 O009 Figure 2a: Quality Degradation function of FER 1 Mils Perceived quality(M0S) degradation as a function of the RBER o 001% 0010% o 100% 1000% 10 O009 Figure 2
46、b: Quality Degradation function of RBER ETSI COPYRIGHT European Telecommunications Standards InstituteLicensed by Information Handling Services3GPP TR 26.912 version 4.0.0 Release 4 Layer Application Layer Mux Layer P hvs i cal ETSI TR 126 912 V4.0.0 (2001-03) Entity Instance Video Codec Speech Code
47、c Dummy data Multiplexer De-multiplexer Simulated Error Dattern files IS0 MPEG-4 Simple Profile or ITU-T H.263 4 Ver.2 H.223 6/M (mobile extension of ITU-T H.223 6 multiplexing protocol) 11 Wideband CDMA-channel Comments on the previous results: bitrate: 32 Kbis, 64 Kbit/s and 128 Kbis channel error
48、 condition BER: le-3, le-4 and le-6 velocitv (model: 3km/h (Vehicular-A and 120km/h (Vehicular-A In clean speech, it appears that all AMR codec modes do not show any significant quality degradation when the Frame Erasure Rate is lower than 0.5%. In some instances, the range can even be extended to 1
49、% FER without any quality degradation. It is also interesting to note that at 1% FER degradation, the highest codec modes (12.2 and 10.2) are still equivalent to the second tier of codec modes (7.95 to 5.9) in error free. Similarly, the middle range codec modes (7.95 to 5.9) present the same quality at 1% FER than the lower rate codec modes (5.15 and 4.75) in error free conditions. The results as a function of the RBER are quite similar with a different range of acceptable RBER. The AMR codec modes do not present any
