ETSI TS 102 250-6-2004 Speech Processing Transmission and Quality Aspects (STQ) QoS aspects for popular services in GSM and 3G networks Part 6 Post processing and statistical metho.pdf

1、 ETSI TS 102 250-6 V1.2.1 (2004-10)Technical Specification Speech Processing, Transmission and Quality Aspects (STQ);QoS aspects for popular services in GSM and 3G networks;Part 6: Post processing and statistical methodsETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 2 Reference RTS/STQ-00061m Keywords 3G,

2、GSM, network, QoS, service, speech ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice Individual c

3、opies of the present document can be downloaded from: http:/www.etsi.org The present document may be made available in more than one electronic version or in print. In any case of existing or perceived difference in contents between such versions, the reference version is the Portable Document Forma

4、t (PDF). In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current st

5、atus of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: http:/portal.etsi.org/chaircor/ETSI_support.asp Copyright Notification No part may be reproduced exc

6、ept as authorized by written permission. The copyright and the foregoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2004. All rights reserved. DECTTM, PLUGTESTSTM and UMTSTM are Trade Marks of ETSI registered for the benefit of its Members. TIPHO

7、NTMand the TIPHON logo are Trade Marks currently being registered by ETSI for the benefit of its Members. 3GPPTM is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 3 Contents Intellectual Property Rights6

8、 Foreword.6 Introduction 7 1 Scope 8 2 References 8 3 Definitions, symbols and abbreviations .8 3.1 Definitions8 3.2 Symbols9 3.3 Abbreviations .9 4 Important measurement data types in mobile communications .9 4.1 Data with binary values10 4.2 Data out of time-interval measurements.10 4.3 Measuremen

9、t of data throughput10 4.4 Data concerning quality measures10 5 Distributions and moments.11 5.1 Introduction 11 5.2 Continuous and discrete distributions.12 5.3 Definition of density function and distribution function 12 5.3.1 Probability Distribution Function (PDF) 12 5.3.2 Cumulative Distribution

10、 Function (CDF) .13 5.4 Moments and quantiles.13 5.5 Estimation of moments and quantiles.15 5.6 Important distributions .16 5.6.1 Continuous distributions .16 5.6.1.1 Normal distribution .16 5.6.1.1.1 Standard normal distribution .17 5.6.1.1.2 Central limit theorem.18 5.6.1.1.3 Transformation to nor

11、mality18 5.6.1.2 Log-Normal distribution .18 5.6.1.2.1 Use-case: transformations19 5.6.1.3 Exponential distribution19 5.6.1.4 Weibull distribution 20 5.6.1.5 Pareto distribution .21 5.6.1.6 Extreme distribution (Fisher-Tippett distribution) 22 5.6.2 Testing distributions .22 5.6.2.1 Chi-Square distr

12、ibution with n degrees of freedom 23 5.6.2.1.1 Further relations.24 5.6.2.1.2 Relation to empirical variance.24 5.6.2.2 Student t-distribution.24 5.6.2.2.1 Relation to normal distribution25 5.6.2.3 F distribution.26 5.6.2.3.1 Quantiles27 5.6.2.3.2 Approximation of quantiles .27 5.6.2.3.3 Relations t

13、o other distributions28 5.6.3 Discrete distributions 28 5.6.3.1 Bernoulli distribution 28 5.6.3.2 Binomial distribution 29 5.6.3.3 Geometric distribution 30 5.6.3.4 Poisson distribution.31 5.6.4 Transitions between distributions and appropriate approximations32 5.6.4.1 From binomial to Poisson distr

14、ibution 32 5.6.4.2 From binomial to Normal distribution 32 5.6.4.3 From Poisson to Normal distribution 32 ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 4 5.6.5 Truncated distributions .33 5.6.6 Distribution selection and parameter estimation.33 5.6.6.1 Test procedures .33 5.6.6.1.1 Chi-Square test 33 5.6.

15、6.1.2 Kolmogorov-Smirnov test .33 5.6.6.1.3 Shapiro-Wilk test.33 5.6.6.2 Parameter estimation methods 34 5.7 Evaluation of measurement data 34 5.7.1 Statistical tests 34 5.7.1.1 Formulation of statistical tests.34 5.7.1.2 Classes of statistical tests 35 5.7.1.3 Tests for normal and binomial data.35

16、5.7.1.3.1 One-sample tests for normal data 35 5.7.1.3.2 Two-sample tests for normal data36 5.7.1.3.3 Test for binomial data37 5.7.1.4 Distribution-free tests for location 38 5.7.1.4.1 Sign tests38 5.7.1.4.2 Sign rank test .38 5.7.1.4.3 Wilcoxon rank sum test .39 5.7.2 Confidence interval.39 5.7.2.1

17、Binomial distribution 40 5.7.2.2 Normal (Gaussian) distribution.41 5.7.3 Required sample size for certain confidence levels 42 6 Visualization techniques.43 6.1 Visualization of static data .43 6.1.1 Histograms43 6.1.2 Barplots.43 6.1.3 QQ-Plots .44 6.1.4 Boxplots44 6.2 Visualization of dynamic data

18、 45 6.2.1 Line Diagrams 45 6.2.2 Temporal changing Boxplots45 6.2.3 MMQ-Plots.46 7 Time series modelling 46 7.1 Descriptive characterization .47 7.1.1 Empirical moments.48 7.1.2 Decomposition of time series49 7.1.3 Determination of the trend component .50 7.1.3.1 Trend function types .51 7.1.3.1.1 L

19、inear trend function .51 7.1.3.1.2 Polynomial trend function .51 7.1.3.1.3 Non-linear trend models 52 7.1.3.2 Trend estimation .52 7.1.3.3 Transformation of time series by filtering.53 7.1.3.3.1 Linear filters 54 7.1.3.3.2 Exponential filters .56 7.1.4 Seasonal component .57 8 Data aggregation 58 8.

20、1 Basic data aggregation operators58 8.2 Data sources, structures and properties 59 8.2.1 Raw data .59 8.2.1.1 Performance data.59 8.2.1.2 Event data59 8.2.2 Key Performance Indicators / Parameters.60 8.3 Aggregation hierarchies .60 8.3.1 Temporal aggregation.60 8.3.2 Spatial aggregation .61 8.4 Par

21、ameter estimation methods61 8.4.1 Projection method.61 8.4.2 Substitution method 61 ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 5 8.4.3 Application of estimation methods .62 8.4.4 Attributes of aggregation operators 62 8.5 Weighted aggregation.63 8.5.1 Perceived QoS 63 8.5.2 Weighted quantiles .64 8.6 A

22、dditional data aggregation operators65 8.6.1 MAWD and BH65 8.6.2 AVGn65 9 Assessment of performance indices .65 9.1 Estimation of performance parameters based on active service probing systems 65 9.2 Monitoring concepts.65 9.2.1 Control charts66 9.2.1.1 Shewhart control charts.66 9.2.1.2 CUSUM and E

23、WMA charts66 9.2.2 Other alarming rules .66 9.3 Methods for evaluation of objectives .66 9.3.1 Desirability functions67 9.3.2 Loss functions.67 Annex A (informative): Examples of statistical calculations 68 A.1 Confidence intervals for binomial distribution.68 A.1.1 Step by step computation .68 A.1.

24、2 Computation using statistical software.70 A.1.2.1 Computation in R70 A.1.2.2 Computation in Excel .71 A.2 Transition from binomial to normal distribution71 A.3 Definitions of EG 201 769 .72 A.4 Calculation of confidence intervals73 A.4.1 Estimated rate 5 %73 A.4.2 Estimated rate 50 %74 A.4.3 Estim

25、ated rate 95 %74 A.4.4 Lower limit of confidence intervals according to Pearson-Clopper formula75 A.4.5 Upper limit of confidence intervals according to Pearson-Clopper formula 76 A.4.6 Span of confidence intervals according to Pearson-Clopper formula 78 A.5 Different sample sizes 80 A.6 Calculation

26、 methods.82 A.6.1 Calculation of quantiles82 A.7 Reporting of results 84 A.7.1 Methods to use .84 A.7.2 Number of significant decimals85 A.7.3 Rounding of end results85 Annex B (informative): Bibliography.86 History 87 ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 6 Intellectual Property Rights IPRs essen

27、tial or potentially essential to the present document may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: “Intellectual Property Rights (IPRs); Essential, or potential

28、ly Essential, IPRs notified to ETSI in respect of ETSI standards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/webapp.etsi.org/IPR/home.asp). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out b

29、y ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Technical Specification (TS) has been produced by ETSI Technical Committ

30、ee Speech Processing, Transmission and Quality Aspects (STQ). The present document is part 6 of a multi-part deliverable covering the QoS aspects for popular services in GSM and 3G networks, as identified below: Part 1: “Identification of Quality of Service aspects“; Part 2: “Definition of Quality o

31、f Service parameters and their computation“; Part 3: “Typical procedures for Quality of Service measurement equipment“; Part 4: “Requirements for Quality of Service measurement equipment“; Part 5: “Definition of typical measurement profiles“; Part 6: “Post processing and statistical methods“; Part 7

32、: “Sampling methodology“. Part 1 identifies QoS aspects for popular services in GSM and 3G networks. For each service chosen QoS indicators are listed. They are considered to be suitable for the quantitatively characterization of the dominant technical QoS aspects as experienced from the end-custome

33、r perspective. Part 2 defines QoS parameters and their computation for popular services in GSM and 3G networks. The technical QoS indicators, listed in part 1, are the basis for the parameter set chosen. The parameter definition is split into two parts: the abstract definition and the generic descri

34、ption of the measurement method with the respective trigger points. Only measurement methods not dependent on any infrastructure provided are described in the present document. The harmonized definitions given in the present document are considered as the prerequisites for comparison of QoS measurem

35、ents and measurement results. Part 3 describes typical procedures used for QoS measurements over GSM, along with settings and parameters for such measurements. Part 4 defines the minimum requirements of QoS measurement equipment for GSM and 3G networks in the way that the values and trigger-points n

36、eeded to compute the QoS parameter as defined in part 2 can be measured following the procedures defined in part 3. Test-equipment fulfilling the specified minimum requirements, will allow to perform the proposed measurements in a reliable and reproducible way. Part 5 specifies test profiles which a

37、re required to enable benchmarking of different GSM or 3G networks both within and outside national boundaries. It is necessary to have these profiles so that when a specific set of tests are carried out then customers are comparing “like for like“ performance. ETSI ETSI TS 102 250-6 V1.2.1 (2004-10

38、) 7 Part 6 describes procedures to be used for statistical calculations in the field of QoS measurement of GSM and 3G network using probing systems. Part 7 describes the field measurement method procedures used for QoS measurements over GSM where the results are obtained applying inferential statist

39、ics. Introduction All the defined quality of service parameters and their computations are based on field measurements. That indicates that the measurements were made from customers point of view (full end-to-end perspective, taking into account the needs of testing). It is assumed that the end cust

40、omer can handle his mobile and the services he wants to use (operability is not evaluated at this time). For the purpose of measurement it is assumed: that the service is available and not barred for any reason; routing is defined correctly without errors; and the target subscriber equipment is read

41、y to answer the call. Voice quality values measured should only be employed by calls ended successfully for statistical analysis. However, measured values from calls ended unsuccessfully (e.g. dropped) should be available for additional evaluations and therefore, must be stored. Further precondition

42、s may apply when reasonable. ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 8 1 Scope The present document describes definitions and procedures to be used for statistical calculations which are related to Quality of Service (QoS) measurements done by serving probing systems in mobile communications network

43、s, especially GSM and 3G networks. Network performance measurements and their related post-processing are only marginally covered in the present document. 2 References The following documents contain provisions which, through reference in this text, constitute provisions of the present document. Ref

44、erences are either specific (identified by date of publication and/or edition number or version number) or non-specific. For a specific reference, subsequent revisions do not apply. For a non-specific reference, the latest version applies. Referenced documents which are not found to be publicly avai

45、lable in the expected location might be found at http:/docbox.etsi.org/Reference. 1 ETSI EG 201 769: “Speech Processing, Transmission and Quality Aspects (STQ); QoS parameter definitions and measurements; Parameters for voice telephony service required under the ONP Voice Telephony Directive 98/10/E

46、C“. 3 Definitions, symbols and abbreviations 3.1 Definitions For the purposes of the present document, the following terms and definitions apply: rate: measurement result which is related to the portion of time during which it has been executed NOTE: The denominators unit is related to time. ratio:

47、measurement result which represents a subgroup of all single measurements is related to the total number of executed single measurements NOTE: Usually, nominator and denominator share the same unit, namely a counter for measurements (subgroup/all). ETSI ETSI TS 102 250-6 V1.2.1 (2004-10) 9 3.2 Symbo

48、ls For the purposes of the present document, the following symbols apply: E(x)= Expected value of random variable x Var(x)=2Variance of random variable x Standard deviation of random variable x f(x) Probability Density Function (PDF) of random variable x F(x) Cumulative Distribution Function (CDF) o

49、f random variable x S, x S Set of discrete values or interval of values the random variable x may take IR Set of real numbers s, s2Empirical standard deviation / variance, analogous to and 2(theoretical) q-Quantile u-Quantile of standard normal distribution x(i), x(1), x(n)i-th ordered value, minimum and maximum of a given data set xi, i =1,.,n 3.3 Abbreviations For the purposes of the present document, the following abbreviations apply: 3G Third Generation ARMA Auto-Regressive Moving Average AVGn Averaging Operator (regarding n days) BH Busy Hour BSC