ETSI TR 126 936-2017 Digital cellular telecommunications system (Phase 2+) Universal Mobile Telecommunications System (UMTS) LTE Performance characterization of 3GPP audio codecs (.pdf

1、 ETSI TR 126 936 V14.0.0 (2017-04) Digital cellular telecommunications system (Phase 2+) (GSM); Universal Mobile Telecommunications System (UMTS); LTE; Performance characterization of 3GPP audio codecs (3GPP TR 26.936 version 14.0.0 Release 14) floppy3TECHNICAL REPORT ETSI ETSI TR 126 936 V14.0.0 (2

2、017-04)13GPP TR 26.936 version 14.0.0 Release 14Reference RTR/TSGS-0426936ve00 Keywords GSM,LTE,UMTS ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sou

3、s-Prfecture de Grasse (06) N 7803/88 Important notice The present document can be downloaded from: http:/www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall

4、not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Sec

5、retariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https:/portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document

6、, please send your comment to one of the following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by writte

7、n permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2017. All rights reserved. DECTTM, PLUGTESTSTM, UMTST

8、Mand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI

9、TR 126 936 V14.0.0 (2017-04)23GPP TR 26.936 version 14.0.0 Release 14Intellectual Property Rights IPRs essential or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non

10、-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https:/ipr.etsi.org/). Pur

11、suant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present do

12、cument. Foreword This Technical Report (TR) has been produced by ETSI 3rd Generation Partnership Project (3GPP). The present document may refer to technical specifications or reports using their 3GPP identities, UMTS identities or GSM identities. These should be interpreted as being references to th

13、e corresponding ETSI deliverables. The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under http:/webapp.etsi.org/key/queryform.asp. Modal verbs terminology In the present document “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be i

14、nterpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in ETSI deliverables except when used in direct citation. ETSI ETSI TR 126 936 V14.0.0 (2017-04)33GPP TR 26.936 version 14.0.0 Release 14Contents In

15、tellectual Property Rights 2g3Foreword . 2g3Modal verbs terminology 2g3Foreword . 5g31 Scope 6g32 References 6g33 Abbreviations . 6g34 Subjective test results . 7g34.1 Organization of the subjective test results 7g34.2 Subjective test method 9g35 Performance Characterization for Audio Content 10g35.

16、1 Results from characterization phase 1 10g35.2 Results from Characterization phase 2 . 11g35.3 Results from Selection Test 13g36 Performance characterization over bit-rate 15g37 Performance based on Selection Tests . 16g37.1 Intrinsic quality comparison in lower bit-rate range (up to 24 kbps) . 16g

17、37.2 Quality comparison under stressed operating conditions at lower bit-rates (up to 24 kbps) 18g37.3 Performance evaluation in higher bit-rate range (up to 48 kbps) . 19g38 Performance characterization for error conditions . 19g39 Results of verification tests 22g39.1 Complexity verification 22g39

18、.1.1 AMR-WB+ complexity 22g39.1.2 Eaac+ Complexity 23g39.2 Frequency response verification . 23g39.2.1 Frequency response of AMR-WB+ 23g39.2.1.1 Characterization phase configuration 23g39.2.1.2 Selection phase configuration . 27g39.2.2 Frequency response of Eaac+ . 30g39.3 Codec delay verification .

19、 34g39.3.1 AMR-WB+ codec delay . 34g39.3.2 Eaac+ codec delay 35g39.4 Performance verification with 3D audio signal 36g39.5 Verification of rate switching performance 41g39.5.1 Rate switching performance of AMR-WB+ . 41g39.5.2 Rate switching performance of Eaac+ 41g39.6 Verification of source code 42

20、g39.6.1 Source code verification of AMR-WB+ . 42g39.6.2 Source code verification of Eaac+ 45g39.6.3 General discussion 48g39.7 Content dependency . 49g3Annex A: Test results from other bodies 50g3A.1 Test results from ITU-T standardization of G.722-1, annex C 50g3Annex B: Documents for information.

21、52g3B.1 Official testing documents (file TR26.936 Annex B1 Official Test Documents.zip) . 52g3B.2 Additional information documents (file TR26.936 Annex B2 Additional Documents.zip) 52g3ETSI ETSI TR 126 936 V14.0.0 (2017-04)43GPP TR 26.936 version 14.0.0 Release 14B.3 Verification test documents (fil

22、e TR26.936 Annex B3 Verification test documents.zip) 53g3Annex C: Change history . 54g3History 55g3ETSI ETSI TR 126 936 V14.0.0 (2017-04)53GPP TR 26.936 version 14.0.0 Release 14Foreword This Technical Report has been produced by the 3rdGeneration Partnership Project (3GPP). The contents of the pres

23、ent document are subject to continuing work within the TSG and may change following formal TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an identifying change of release date and an increase in version number as follows: Version x.y.

24、z where: x the first digit: 1 presented to TSG for information; 2 presented to TSG for approval; 3 or greater indicates TSG approved document under change control. y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections, updates, etc. z the third digi

25、t is incremented when editorial only changes have been incorporated in the document. ETSI ETSI TR 126 936 V14.0.0 (2017-04)63GPP TR 26.936 version 14.0.0 Release 141 Scope The present document comprises the Technical Report for the Characterization of the 3GPP Audio Codecs, Enhanced aacPlus (Eaac+)

26、and Extended AMR-WB (AMR-WB+), standardized by 3GPP in Release 6 for Packet-switched Streaming Service (PSS), Multimedia Messaging Service (MMS), Multimedia Broadcast and Multicast Service (MBMS), and IMS Messaging Service and Presence Service. 2 References The following documents contain provisions

27、 which, through reference in this text, constitute provisions of the present document. References are either specific (identified by date of publication, edition number, version number, etc.) or non-specific. For a specific reference, subsequent revisions do not apply. For a non-specific reference,

28、the latest version applies. In the case of a reference to a 3GPP document (including a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same Release as the present document. 1 ITU-R Recommendation BS.1534-1 (2003): “Method for the subjective ass

29、essment of intermediate quality level of coding systems“. 2 ISO/IEC JTC1/SC29/WG11/N5571 (2003): “Report on Informal MPEG-4 Extension 1 (Bandwidth Extension) Verification Tests“. 3 Tech 3296 (2003): “EBU Subjective listening tests on low bit-rate audio codecs“. 4 3GPP TS 26.273: “ANSI-C code for the

30、 fixed-point Extended Adaptive Multi-Rate - Wideband (AMR-WB+) speech codec“. 5 3GPP TS 26.290: “Audio codec processing functions; Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Transcoding functions“. 6 3GPP TS 26.410: “General audio codec audio processing functions; Enhanced aacPlus gene

31、ral audio codec; Floating-point ANSI-C code“. 7 3GPP TS 26.411: “General audio codec audio processing functions; Enhanced aacPlus general audio codec; Fixed-point ANSI-C code“. 3 Abbreviations For the purposes of the present document, the following abbreviations apply: ACR Absolute Category Rating m

32、ethod AMR-WB+ Extended AMR-WB BLER Block Error Rate DCR Degradation Category Rating method DMOS Degradation Mean Opinion Score Eaac+ Enhanced aacPlus EBU European Broadcasting Union EGPRS Enhanced General Packet Radio Service FEC Forward Error Correction FER Frame Error Rate HRTF Head Related Transf

33、er Function ISF Internal Sampling FrequencyITU-T International Telecommunications Union - Telecommunications MBMS Multi-media Broadcast and Multicast Services ETSI ETSI TR 126 936 V14.0.0 (2017-04)73GPP TR 26.936 version 14.0.0 Release 14MMS Multi-media Messaging Service MOS Mean Opinion Score MPEG

34、Moving Picture Experts Group MUSHRA Multiple Stimulus with Hidden Reference and Anchors method PLR Packet Loss RatePSS Packet-switched Streaming Service ROHC Robust Header Compression RTP Real-time transport protocol RLC-PDU Radio Link Control - Protocol Data Unit SEAQ System for the Evaluation of A

35、udio Quality UTRAN UMTS Terrestrial Radio Access Network 4 Subjective test results The main body of the present document summarizes the results from official tests conducted by 3GPP. Further subjective test results were provided into the standardization process by various parties. Some of these resu

36、lts were part of the verification tasks. Two other contributions from codec proponents assessing the codecs across different bit rates in the low rate segment up to 24 kbps were provided in earlier phases of the process and are also attached to the present document (see S4-040439 and S4-040710 in at

37、tachment “TR 26.936 Annex B2 Additional Documents.zip“). In addition, there are further reports from other bodies. Clause A.1 comprises test results for the two standardized codecs carried out by ITU-T, where the 3GPP audio codecs served as reference codecs. The Annex also contains references to tes

38、ts on Eaac+ carried out in MPEG and EBU, which provide information for configurations using the Eaac+ decoder in combination with a non-standardized encoder. In the reporting of subjective test results, it is generally agreed that comparisons of results are valid only for conditions conducted within

39、 the same experiment. It is not valid, for example, to directly or statistically compare subjective test results for one codec across two bit-rates when those results have been obtained from different experiments. In general, this principle will be observed in the subjective test results presented i

40、n the following clauses. 4.1 Organization of the subjective test results This report comprises data from subjective tests derived from standardization exercises organized in 3GPP. Each of the test exercises was conducted in accordance with a test plan and results were provided in a test report. Tabl

41、e 1 summarizes these test exercises including a series label, a description of the tests, and the specific Attachments containing the appropriate test plans and test reports. Table 1: Subjective Test Series involved in the Technical Report Standardization exercise Label Test plan Test report 3GPP au

42、dio codec characterization test, phase 1 CT-P1 Attachment 1A Attachment 1B 3GPP audio codec characterization test, phase 2 CT-P2 Attachment 1A Attachment 1C 3GPP audio codec low rate selection test ST-LR Attachment 1D Attachment 1E 3GPP audio codec high rate selection test ST-HR Attachment 1F Attach

43、ment 1G Table 2 lists the subjective tests that provided the results reported in the present document including details such as the test parameters, mode (mono vs. stereo), and number of listening labs. All of the subjective tests described in this report involved some common test parameters. These

44、include the use of 15 subjects and 12 test items where the test items are sub-divided into three classes of Audio Content - four items each for Music-only, Speech-only, and Mixed Music+Speech audio content. All experiments in CT-P1 and CT-P2 used the same 12 items of material. The ST-LR tests used 2

45、4 items in each experiment, but the test items were split into two sets of 12 test items and allocated across to different labs running the same experiment. All experiments in ST-HR used the same 12 test items. ETSI ETSI TR 126 936 V14.0.0 (2017-04)83GPP TR 26.936 version 14.0.0 Release 14Table 2: S

46、ummary of subjective tests involved in the Technical Report Series Test parameter(s) Mode # Labs CT-P1 3 bit-rates mono 2 3 bit-rates stereo 2 CT-P2 4 PLRs, EGPRS mono 1 4 PLRs, EGPRS stereo 1 3 PLRs, UTRAN - Lower rates stereo 1 3 PLRs, UTRAN - Higher rates stereo 1 ST-LR 14 kbps, PSS mono 4 18 kbp

47、s, PSS stereo 4 24 kbps, PSS mono 4 24 kbps, PSS stereo 4 14 kbps, MMS, 16 kHz sampling rate mono 4 18 kbps, MMS stereo 4 14 kbps, PSS, 3 % FER mono 4 24 kbps, MMS, 3 % FER stereo 4 ST-HR 32 kbps, PSS stereo 2 48 kbps, PSS stereo 2 32 kbps, PSS, 1 % and 3 % FER stereo 2 It should be noted that, for

48、both audio codecs, the codecs used in the Characterization Test (i.e. CT-P1 and CT-P2 test series) were different (e.g., bug fixes, optimized configurations) from the candidate codecs used in the earlier Selection Test (i.e. ST-LR and ST-HR test series). The Selection test results also provide infor

49、mation how the selected codecs perform in relation to Release 5 audio codecs. The clauses of the present document are organized as follows: Clause 5: Performance Characterization for Audio Content: - This clause comprises a quality evaluation of the codecs across audio content. It contains results from codec characterization both without and with packet loss. It also contains a similar evaluation from the selection tests. Clause 6: Performance Characterization over Bit-rate: - This clause comprises the test results from codec characterization asses