UOP 999-2013 Precision Statements in UOP Methods.pdf

1、 COPYRIGHT 1997, 2004, 2005, 2013 UOP LLC. All rights reserved. Nonconfidential UOP Methods are available from ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States. The UOP Methods may be obtained through the ASTM website, www.astm.org, or by contac

2、ting Customer Service at serviceastm.org, 610.832.9555 FAX, or 610.832.9585 PHONE. Precision Statements in UOP Methods UOP Method 999-13 Scope This method is for developing precision statements as reported in UOP methods and relative bias of the same method between different laboratories. The calcul

3、ation of precision, in terms of repeatability and intermediate precision (within a laboratory), reproducibility (between laboratories) and relative bias (between laboratories) is described. Precision statements in methods having a 98 or later suffix were developed by the procedure described; methods

4、 having an 88 through 97 suffix used UOP 888-88, while methods with suffixes earlier than 88 used UOP 666-82. The features of UOP 999 are as follows: Minimum of 16 analyses required with emphasis on multiple concentration levels Better identification of the components of variation and repeatability

5、estimates Alignment with ASTM repeatability, intermediate precision, and reproducibility definitions Relative bias between laboratories with 95% confidence interval UOP 999 is the minimum guideline for estimating the initial precision for new or revised methods. Once a new or revised method is in us

6、e, a reference sample, if available, should be run on a regular basis. When 16 analyses have been completed, individual and range control charts are to be used to estimate the long-term UOP repeatability and are to be maintained on an ongoing basis. The example calculations in UOP 999 are made using

7、 Minitab, a statistical software package. The details of the calculations are in UOP 999-97 Supplement. References ASTM Practice E177, “Practice for the use of Terms Precision and Bias in ASTM Test Methods,” www.astm.org ASTM Practice E178, “Practice for Dealing with Outlying Observations,” www.astm

8、org ASTM Practice E691, “Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method,” www.astm.org “Errors of Routine Analysis,” Student, Biometrika, XIX, 1927, pp. 151-164 “Precision Measurement and Calibration,” NBS Special Publication 300, Vol. 1, Feb. 1969, U.S

9、 Department of Commerce Supplement to UOP 999-97 UOP Method 666-82, “Precision Statements in UOP Methods” UOP Method 888-88, “Precision Statements in UOP Methods” 2 of 19 999-13 Outline of the Method Using the specified UOP test method, at least 16 tests are performed at a given laboratory or labor

10、atories on the same representative sample, or on samples at multiple concentrations, using one of the nested sampling designs shown in the Procedure section. Exceptions to nested sampling design can be made at the discretion of the method coordinator. In each laboratory, the analysis is performed by

11、 at least two different analysts on each of two separate days, each analyst performing two tests per day. The estimated within-laboratory standard deviation (esd) and the estimated between-laboratory standard deviation (esd) are calculated using a stepwise nested analysis of variance (ANOVA) procedu

12、re (see Appendix 2 of the Supplement to this document). The stepwise ANOVA procedure for the nested design is used to estimate the within-day test esd, day-to-day esd, analyst-to-analyst esd and lab-to-lab esd. The estimated components of variability from the stepwise ANOVA are used to calculate rep

13、eatability, intermediate precision, reproducibility, and relative bias as outlined in Table 1. Repeatability is the expected maximum difference between two tests by the same analyst on the same day in one laboratory at the 95% confidence level. Repeatability is calculated using the Within-Day esd. I

14、ntermediate precision is the expected maximum difference between two tests by different analysts on different days in one laboratory at the 95% confidence level and is calculated using the Within-Laboratory esd. Reproducibility is the expected maximum difference between two tests by different analys

15、ts on different days in different laboratories at the 95% confidence level and is calculated using the Within-Laboratory esd plus the Between-Laboratory esd. When all the tests are conducted in only one laboratory, then repeatability and intermediate precision may be calculated, but reproducibility

16、cannot. For this case, applicable nested designs are shown in the Procedure section. When the testing is conducted across two or more laboratories, then the repeatability, the intermediate precision, the reproducibility and the relative bias can be calculated. For two or three laboratories, the appl

17、icable nested designs are found in the Procedure section. Table 2 in the Procedure section summarizes standard nested designs and their attributes with cross-reference to the design layout. If precision data are received from one or two laboratories, the repeatability and intermediate precision is t

18、he only precision information reported. Calculations may also be performed on pairs of data, when such data are available, and the nested sampling designs shown in the Procedure section would be difficult or impossible to perform. See Appendix 3. Definitions 95% Confidence Interval for Mean Differen

19、ce, the interval about the average difference value that is believed to include the “true” population difference for approximately 95 out of 100 such estimates. Analysis of Variance (ANOVA), a statistical procedure that divides the total variability for a set of data into meaningful component parts

20、associated with specific sources of variation. The technique, in conjunction with the F-ratio, is used to provide a test of significance for component sources of variation and to obtain estimates of the standard deviations attributable to those sources. Bias, deviation from a known or accepted value

21、 Day, refers to the span of time over which the analyses are performed on the same day by the same analyst in a laboratory and from which the within-day standard deviation is estimated. 3 of 19 999-13 Table 1 Precision Statement Definitions (The description of the terms is in the CALCULATIONS) Degr

22、ees of Freedom, the number of independent observations available to estimate the standard deviation. In general, when r constants for a model have been estimated from n data values, only (n - r) degrees of freedom remain to estimate the models variability. Duplicate, the false replication of an anal

23、ysis (i.e. one test immediately followed by another) in which all the sources of inherent variability are not operational. Intermediate Precision, the allowable difference between two tests performed by different analysts in one laboratory on different days. Two tests should not differ by more than

24、the stated allowable difference more than five percent of the time (for 95% confidence). This definition is the same as the term “UOP Repeatability” used in past UOP methods describing precision, and should be used when comparing data generated on other than the same day. Intermediate Precision may

25、also be called Site Precision. Quantity Definition Repeatability Allowable difference between two tests performed by same analyst in one lab on same day at 95% confidence level. Repeatability Equation t 2 sDF Within-Day Intermediate Precision Allowable difference between two tests performed by diffe

26、rent analysts in one lab on different days at 95% confidence level. Intermediate Precision Equation t 2 sDF Within-Lab Within-Day Standard Deviation t 2 sDF Within-Day Within-Lab Standard Deviation s s s sWithin-Lab Within - Day2 Day - to - Day2 + Analyst - to - Analyst2 = + Reproducibility Allowabl

27、e difference between two tests performed by different analysts in different labs on different days at 95% confidence. Reproducibility Equation t 2 s sDFWithin - Lab2Between - Lab2+ Relative Bias For specified sample, the average test difference between labs. Within-Lab Mean Standard Deviation sX Wit

28、hin-Lab = sn n n+ sn n+ snWithin-Day2R Days AnalystsDay-to-Day2Days AnalystsAnalyst-to-Analyst2AnalystsDF* = nLabs ( nAnalysts - 1 ) where nDays = No. days testing per analyst nLabs = No. labs nAnalysts = No. analysts nR = No. reps per day Note: See SUPPLEMENT Appendix 2 for other than full model. R

29、elative Bias Confidence Interval x - x t 2sLab 1 Lab 2 DF* X Within-Lab 4 of 19 999-13 Mean Square (MS), an unbiased estimate of the population variance calculated by dividing the sum of squares (SS) by its degrees of freedom (DF). Pooling, two or more variances (standard deviations squared) may be

30、pooled by adding their variances weighted by ratio of the individual variance degrees of freedom divided by the sum of the degrees of freedom for all the variances. Randomization, the operation of assigning a testing sequence in a purely chance manner by using a list of random numbers. Randomization

31、 increases the chance of obtaining a representative sample from the population and thereby assures and ensures a valid estimate of experimental error and associated significance tests. Relative Bias, the average test difference between laboratories for the same sample analyses. Repeatability, the al

32、lowable difference between two tests performed by the same analyst in one laboratory on the same day. The two tests should not differ by more than the stated allowable difference more than five percent of the time, thus giving 95% confidence in the repeatability. This definition more closely mirrors

33、 that of ASTM Repeatability, and should allow for closer comparison to independently reported ASTM repeatability statistics. Replicate, the random repetition of an analysis over a specified period of time, such as a day, under identical conditions subject only to (but to all of) the random inherent

34、variability of that time interval. Reproducibility, the allowable difference between two tests performed by different analysts in different laboratories on different days. Two such tests should not differ by more than the stated allowable difference more than five percent of the time (for 95% confid

35、ence). This definition is the same as the ASTM convention; therefore, independently reported reproducibility statistics can be directly compared. Test, the result of a single analysis performed in a laboratory by a specified UOP method. Determinations performed in duplicate (i.e. one test immediatel

36、y followed by another) are discouraged, since little information is gained. However, when duplicates are routinely performed, a test is defined as the average of the two determinations. Procedure The laboratory supervisor under whose jurisdiction the method is performed is responsible for collecting

37、 the necessary nested data using the proper sampling procedure to assure uniformity between the samples analyzed, following the UOP method exactly as written, and reporting those data together with a record of the analyst, day, time and test number. Care must be taken to accurately record, on the fo

38、rm provided, the origin of each result, noting the analyst, test number, day and time of the test. All the data collected must be reported and no effort should be made to eliminate data points by rejecting individual tests. Nor should the data be truncated by rounding. It is better to record too man

39、y digits than not enough. The resultant data are referred to as a “balanced nested” data set. Choosing a Nested Design Precision studies requiring no less than the minimum 16 separate analyses are characterized in Table 2 and shown in Figures 1A and 1B, Within-Laboratory and Within-plus-Between Labo

40、ratory Nested Sampling Designs. To estimate only the within-laboratory repeatability and intermediate precision for a single sample concentration in one laboratory using four analysts, use Form 1A-1 in 5 of 19 999-13 Appendix 1. When two sample concentrations in a single laboratory are to be used to

41、 estimate only the within-laboratory repeatability and intermediate precision, use Form 1A-2 in Appendix 1. To estimate repeatability, intermediate precision, reproducibility, and relative bias between two laboratories for a single sample concentration using two analysts per laboratory, use Form 1B-

42、1 in Appendix 1. To estimate repeatability, intermediate precision, reproducibility, and relative bias between two laboratories at two sample concentrations using two analysts per laboratory, use Form 1B-1 in Appendix 1, at each laboratory. This requires 16 separate analyses at each laboratory for a

43、 total of 32 analyses. Table 2 Nested Designs for Different Precision Objectives UOP 999 Precision Objective No. Labs No. Sample Concentrations No. Analysts No. Tests See Figure Estimate Only Repeatability Within Lab at 1 Concentration 1 1 4 16 1A-1 Estimate Only Repeatability Within Lab at 2 Concen

44、trations 1 2 2 16 1A-2 Estimate Only Repeatability Within Lab at 3 Concentrations 1 3 2 24 1A-3 Estimate Repeatability, Reproducibility and Relative Bias using 2 Labs 2 1 2 16 1B-1 Estimate Repeatability, Reproducibility and Relative Bias using 2 Labs 2 2 2 32 1B-1 at each Lab Estimate Repeatability

45、 Reproducibility and Relative Bias using 2 Labs 2 1 3 24 1B-2 Estimate Repeatability, Reproducibility and Relative Bias using 3 Labs 3 1 2 24 1B-3 Note: For all the designs given in Table 2, each analyst performs 2 tests on each of 2 days. Figure 1A-3 shows a nested design requiring 24 analyses. Th

46、is plan is for estimating only the within-laboratory repeatability and intermediate precision using three different concentration samples in a single laboratory with two analysts. Another nested design with 24 analyses is shown in Figure 1B-3, and is useful for estimating repeatability, intermediate

47、 precision, reproducibility, and relative bias between three laboratories for a single concentration sample using two analysts per laboratory. No corresponding data form was prepared for the nested designs requiring 24 analyses. 6 of 19 999-13 Within-Laboratory Nested Sampling Designs Figure 1A-1: 1

48、 Sample Concentration Level using 4 Analysts Figure 1A-2: 2 Sample Concentration Levels using 2 Analysts Figure 1A-3: 3 Sample Concentration Levels using 2 Analysts Note: Data forms corresponding to the Figure 1A-1 and Figure 1A-2 designs above are found in Appendix 1. 11 2 3 41 2 1 2 1 2 1 21 2 1 2

49、 1 2 1 2 1 2 1 2 1 2 1 2LaboratoryAnalystsDaysWithinDay 16 Tests per ConcentrationConcentrationsAnalystsDaysWithinDay8 Tests per Concentration 8 Tests per Concentration 8 Tests per Concentration 8 Tests per Concentration 8 Tests per Concentration1 2 31 2 1 2 1 21 2 1 2 1 2 1 2 1 2 1 21 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2ConcentrationAnaly