1、 Access to Additional Content for TIA/EIA-736-A, Dated: August 2000 (Click here to view the publication) This Page is not part of the original publication This page has been added by IHS as a convenience to the user in order to provide access to additional content as authorized by the Copyright hold
2、er of this document Click the link(s) below to access the content and use normal procedures for downloading or opening the files. TIA-EIA-736-A Software Information contained in the above is the property of the Copyright holder and all Notice of Disclaimer furthermore, to prevent any one segment fro
3、m dominating the average, the value of SNRi is limited to a range of -5 to +80 dB. In particular, if xi(n) and yi(n) are identical, the SNRi value is clipped to 80 dB. Two statistics of the SNRi are used. These are the average segmental signal-to-noise ratio (SNRSEG) See 3.3.21: N 1 SNRSEG =MCSNRi i
4、= 1 where M is the number of segments where SNRi # O and the cumulative distribution function f(x) See 3.3.2.2 and 3.3.31: f(x) expresses the probability that an SNRi value takes on a value less than or equal to some particular value x: f(x) = P(SNRi = x). The encodeddecoder objective test is met wh
5、en SNRSEG and f(x) satisfy the specified criteria of the appropriate minimum performance specification. 2.1.2 Method of Measurement For the codec being tested, the source-speech material, supplied in the directory /expt l/ob ject ive of the companion software distribution (see Figure 3.6-l), is proc
6、essed by the master codec and by the codec being tested. The post-filters shall be used in both the master codec and the codec being tested. The master codec is to be initialized with all internal states equal to O. The resulting linear output samples (8 kHz sampling rate) are time-aligned. The time
7、-aligned output of both the master codec and the codec being tested is processed with the snr . c program or the equivalent (see 3.3.2) to produce the SNRi for each 5 ms frame. The program snr . c generates a sample value of SNRSEG and a sample histogram, h 1 : n , for each continuous data stream TI
8、AlEIAl736-A . 1 2 and presented to it. To generate the sample statistics for the entire database, the input files shall be concatenated . 3 4 5 6 7 presented in one pass to the analysis program. The resulting measurements are used to compute the segmental signal-to-noise ratio, SNRSEG, and a sample
9、histogram, h 1 : n , of the SNRi. The histogram is computed using 64 bins, with boundaries being the values two through 64. SNRi values below 2 dB are included in the first bin, while SNRi values above 64 are included in the last bin. Segments corresponding to silence should not be included in the h
10、istogram; thus, given a non-zero SNRi value, x, assign it to a bin according to Table 2.1.2-1. 8 9 Table 2.1.2-1. Computing the Segmental Signal-to-Noise Ratio I SNR Values I Bin Assignment I I x 64.0) then else 1 hl = hl + 1 h64 = h64 + 1 hint(snri) = hint(snri) + 1 SNRSEG = SNRSEG / npoint 34 The
11、histogram array is converted into a sample cumulative distribution function, fl:n, as follows: 2-5 TIA/EIA/736-A 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 hl:64 histogram values npoint number of data points used to compute histogram fl:64 normal
12、ized cumulative distribution function fl = hl/npoint for (i = 2, 64) fi = fi-11 + hi / npoint An ANSI C source language program, cdf . c, that implements this procedure is given in 3.3.3. 2.1.3 Minimum Performance Requirements The codec being tested meets minimum performance requirements if all of t
13、he three following requirements are met over the entire source speech material database of directory /expt Uobjective: fl14= 0.01, which means that the SNRi can have a value lower than 15 dB no more than 1% of the time; and fl24= 0.05, which means that the SNRi can have a value lower than 25 dB no m
14、ore than 5% of the time; and SNRsEG=4OdB. 2.2 Combined ObjectivdSubjective Performance Requirements If the strict objective test has failed or is not attempted, then the codec being tested may be evaluated with the combined objective and subjective tests described in this section. These objective an
15、d subjective tests are divided into three experiments: Experiment I tests the codec under various speaker and input level conditions. Both objective and subjective tests shall be met to satisfy minimum performance requirements. Experiment II tests the decoder under impaired channel conditions. If th
16、e objective performance criteria of this test are met, it is not necessary to perform the subjective component of the test. If the objective performance criteria of this test are not met, the subjective test for channel impairments should be met to satisfy minimum performance requirements. Experimen
17、t III of the test ensures that the rate determination algorithm in the encoder performs correctly under a variety of background noise conditions. If the objective performance criteria of this test are satisfied, it is not necessary to perform the subjective component of the test. If the objective pe
18、rformance criteria of this test are not met, and the average data rate of the codec being tested is less than 1.05 times the master codec average data rate, the subjective test for background noise degradation shall be met to satisfy minimum performance requirements. If the average data rate of the
19、codec being tested is greater than 1 .O5 times the master codec average data rate, then the codec being tested fails the minimum performance requirements. Figure 2.2-1 summarizes the steps necessary to satisfy the combined objective/subjective test requirements. The objective performance requirement
20、 for Experiments I and II (see 2.2.1 and 2.2.2) evaluate the decoder component of the codec being tested. The philosophy of the test is the same as the strict objective test (see 2.1). The objective performance requirement for Experiment III evaluates the encoder component of the codec being tested,
21、 specifically the rate determination mechanism. The method of measurement for comparing the rate output from the master encoder and the test encoder is described in 2.2.3.2. 2-6 TIAlEIAl736-A 1 2 3 The subjective performance requirements for Experiments I, II and III (see 2.2.4) evaluate the codec b
22、eing tested against the master codec performance through the use of a panel of listeners and a Mean Opinion Score criteria. 2-7 TIAlEIAl736-A 1 2 NO 7 NO I M passes Figure 2.2-1. Objective/Subjective Performance Test Flow 2-8 1 2 TIAEIN736-A 2.2.1 Objective Performance Experiment I 2.2.1.1 Definitio
23、n 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 The objective performance Experiment I is intended to validate the implementation of the decoder section of the speech codec being tested under various speaker and input level conditions. The objective perform
24、ance Experiment I that tests the decoder is based upon the same philosophy as the strict objective test (see 2.1.1). Only one combination of encoder and decoder is evaluated against the master encoder/decoder, with the master encoder driving the test decoder. The performance requirements are based u
25、pon sample statistics SNRSEG and f 1 : n , which estimate segmental signal-to-noise ratio, SNRSEG, and the cumulative distribution function,f(x), of the signal-to-noise ratios per segment, SNRi. The codec being tested passes the objective performance requirement of Experiment I if the master encoded
26、test decoder combination satisfies the criteria of 2.2.1.3. 2.2.1.2 Method of Measurement The method of measurement for decoder comparison is the same as for the strict objective test (see 2.1.2). Note: the post-filter shall be used in these objective experiments. 2.2.1.3 Minimum Performance Require
27、ments for Experiment I The codec being tested meets the minimum requirements of the objective performance Experiment I, if all of the four criteria are met over the entire source speech material database in directory /expt l/ob ject ive. The objective test shall be run five times, once for each of t
28、he three levels, -29, -19 and -9 dBmO, once for the background noise file at 15 dB SNR, and once for the -19 dBmO level with the codec running in reduced rate mode 3. These source conditions are provided in the source files /exptl/objective/exptlobj.x29, /exptl/objective/exptlobj.x19, /exptl/objecti
29、ve/exptlobj.x09, /exptl/objective/exptlobjnoise.x19, and /exptl/objective/exptlobj.x19. The numeric bounds given in Sections 2.2.1.3.1 through 2.2.1.3.5 are the performance requirements for Experiment 1. 2.2.1.3.1 Minimum Performance Requirements for Experiment I Level -9 dBmO fll = 0.01, which mean
30、s that the SNR can have a value lower than 2 dB no more than 1% of the time; and fl15= 0.06, which means that the SNR can have a value lower than 16 dB no more than 6% of the time; and SNRSEG=26dB. 2.2.1.3.2 Minimum Performance Requirements for Experiment I Level -19 dBmO fll = 0.01, which means tha
31、t the SNR can have a value lower than 2 dB no more than 1% of the time; and fl15= 0.10, which means that the SNR can have a value lower than 16 dB no more than 10% of the time; 2-9 1 TIAIEIAl736-A 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 and SNRSEG=2
32、2dB. 2.2.1.3.3 Minimum Performance Requirements for Experiment I Level -29 dBmO fll = 0.03, which means that the SNR can have a value lower than 2 dB no more than 3% of the time; and fl15= 0.30, which means that the SNR can have a value lower than 16 dB no more than 30% of the time; and SNRSEG=18dB.
33、 2.2.1.3.4 Minimum Performance Requirements for Experiment I Background Noise fll = 0.01, which means that the SNR can have a value lower than 2 dB no more than 1% of the time; and fl15= 0.10, which means that the SNR can have a value lower than 16 dB no more than 10% of the time; and SNRSEG=22dB. 2
34、.2.1.3.5 Minimum Performance Requirements for Experiment I Level -19 dBmO Mode 3 fll = 0.01, which means that the SNR can have a value lower than 2 dB no more than 1 % of the time; and fl15= 0.14, which means that the SNR can have a value lower than 16 dB no more than 14% of the time; and SNRSEG=22d
35、B. 2.2.2 Objective Performance Experiment II 2.2.2.1 Definition The objective performance Experiment II of the combined test, which validates the implementation of the decoder section of the speech codec under impaired channel conditions, is based upon the same philosophy as the strict objective tes
36、t (see 2.1.1). Only one combination of encoder and decoder is evaluated against the master encoderldecoder: the master encoder driving the test decoder. Impaired channel packets from the master encoder used to drive the master decoder and test decoder for this test are found in the file /expt2/ob je
37、ctive/expt2ob j .pkt of the companion software. The performance requirements are based upon sample statistics SNRSEG and f 1 : n , which estimate segmental signal-to-noise ratio, SNRSEG, and the cumulative distribution function, f(x), of the signal-to-noise ratios per segment, SNRi. The codec under
38、test passes, if the master encoderltest decoder combination satisfies the criteria of 2.2.2.3. If the codec being tested passes the objective test in Experiment II, impaired channel operation has been sufficiently tested and subjective Experiment II is not required for minimum performance validation
39、. However, if the codec fails the objective test in Experiment II (presumably because the codec manufacturer has elected to implement a different error masking algorithm than specified in IS-733), then subjective Experiment II is required. 2.2.2.2 Method of Measurement The method of measurement for
40、decoder comparison is the same as for the strict objective test (see 2.1.2). 2-10 1 TIAlEIAl736-A Note: the post-filter shall be used in these objective experiments. 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 2.2.2.3 Minimum Performance Requirement
41、s for Experiment II The codec being tested meets the minimum requirements of the objective performance part if all of the three following criteria are met over the entire packet speech material database found in the files expt2/objective/expt2obj.pkt and expt2/objective/badrate.pkt. The following nu
42、meric bounds are the performance requirements for Experiment II: fll = 0.01, which means that the SNR can have a value lower than 2 dB no more than 1% of the time; and fl15= 0.05, which means that the SNR can have a value lower than 16 dB no more than 5% of the time; and SNRsEG=22dB. 2.2.3 Objective
43、 Performance Experiment IIi 2.2.3.1 Definition The objective performance Experiment III of the combined test is intended to validate the implementation of the rate determination algorithm in the encoder section of the speech codec under various background noise conditions. The background noise and l
44、evel-impaired source material may be found in the files /expt3/objective/expt3obj.x29, /expt3/objective/expt3obj.x19, and /expt3/objective/expt3obj.x09 of the companion software. These files vary the signal-to-noise ratio of the input speech from 15 dB to 35 dB and back to 23 dB (in 2 dB steps) whil
45、e keeping the signal energy constant at -29, -19, and -9 dBmO, respectively, for each of the three files. Each file is approximately 68 seconds in duration and contains seven sentence pairs. Each sentence pair is separated by at least four seconds of silence. The noise source added digitally to each
46、 speech file varies the noise level in 2 dB steps, every four seconds, to produce the signal-to-noise ratios described above. The codec being tested passes the objective test in Experiment III, if the test encoder rate statistics match the master encoder rate statistics within the criteria of 2.2.3.
47、3 for all three input levels represented by the three input source files. If the codec being tested passes the objective test in Experiment III, the test-codec rate-determination algorithm has been sufficiently tested, and subjective Experiment III is not required for minimum performance validation.
48、 However, if the codec fails the objective test in Experiment III (presumably because the codec manufacturer has elected to implement a different rate determination algorithm than that which is specified in IS-733), then subjective Experiment III shall be performed and shall be passed. 2.2.3.2 Metho
49、d of Measurement For the rate determination objective test, the method of measurement is performed by the program rate-chk . c. The background noise corrupted source files 2-1 1 TIAIEIAl736-A Master Codec 118 rate 114 rate 112 rate full rate 4 5 Test Codec 118 Rate 114 Rate 112 Rate Fun Rate 0.4253 0.0012 0.0 0.0 0.0187 0.1559 0.0062 0.0 0.0 0.0027 0.0676 0.0027 0.0 0.0 0.0003 0.3328 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 are encoded through both the master encoder and the test encoder representing th
