ASTM D6708-2006 Standard Practice for Statistical Assessment and Improvement of Expected Agreement Between Two Test Methods that Purport to Measure the Same Property of a Material《.pdf

1、Designation: D 6708 06An American National StandardStandard Practice forStatistical Assessment and Improvement of ExpectedAgreement Between Two Test Methods that Purport toMeasure the Same Property of a Material1This standard is issued under the fixed designation D 6708; the number immediately follo

2、wing the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon (e) indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice co

3、vers statistical methodology for assess-ing the expected agreement between two standard test methodsthat purport to measure the same property of a material, anddeciding if a simple linear bias correction can further improvethe expected agreement. It is intended for use with resultscollected from an

4、interlaboratory study meeting the require-ment of Practice D 6300 or equivalent (for example,ISO 4259). The interlaboratory study must be conducted on atleast ten materials that span the intersecting scopes of the testmethods, and results must be obtained from at least sixlaboratories using each met

5、hod.NOTE 1Examples of standard test methods are those developed byvoluntary consensus standards bodies such as ASTM, IP/BSI, DIN,AFNOR, CGSB.1.2 The statistical methodology is based on the premise thata bias correction will not be needed. In the absence of strongstatistical evidence that a bias corr

6、ection would result in betteragreement between the two methods, a bias correction is notmade. If a bias correction is required, then the parsimonyprinciple is followed whereby a simple correction is to befavored over a more complex one.NOTE 2Failure to adhere to the parsimony principle generally res

7、ultsin models that are over-fitted and do not perform well in practice.1.3 The bias corrections of this practice are limited to aconstant correction, proportional correction or a linear (propor-tional + constant) correction.1.4 The bias-correction methods of this practice are methodsymmetric, in the

8、 sense that equivalent corrections are obtainedregardless of which method is bias-corrected to match theother.1.5 A methodology is presented for establishing the 95 %confidence limit (designated by this practice as the cross-method reproducibility) for the difference between two resultswhere each re

9、sult is obtained by a different operator usingdifferent apparatus and each applying one of the two methodsX and Y on identical material, where one of the methods hasbeen appropriately bias-corrected in accordance with thispractice.NOTE 3Users are cautioned against applying the cross-method repro-duc

10、ibility as calculated from this practice to materials that are significantlydifferent in composition from those actually studied, as the ability of thispractice to detect and address sample-specific biases (see 6.8) is dependenton the materials selected for the interlaboratory study. When sample-spe

11、cific biases are present, the types and ranges of samples may need tobe expanded significantly from the minimum of ten as specified in thispractice in order to obtain a more comprehensive and reliable 95 %confidence limits for cross method reproducibility that adequately coverthe range of sample spe

12、cific biases for different types of materials.1.6 This practice is intended for test methods which mea-sure quantitative (numerical) properties of petroleum or petro-leum products.1.7 The statistical methodology outlined in this practice isalso applicable for assessing the expected agreement between

13、any two test methods that purport to measure the same propertyof a material, provided the results are obtained on the samecomparison sample set, the standard error associated with eachtest result is known, the sample set design meets the require-ment of this practice, and the statistical degree of f

14、reedom ofthe data set exceeds 30.1.8 Software program CompTM Version 1.0.21(ADJD6708) performs the necessary computations prescribedby this practice.2. Referenced Documents2.1 ASTM Standards:2D 5580 Test Method for Determination of Benzene, Tolu-ene, Ethylbenzene, p/m-Xylene, o-Xylene, C9and Heavier

15、Aromatics, and Total Aromatics in Finished Gasoline byGas ChromatographyD 5769 Test Method for Determination of Benzene, Tolu-ene, and Total Aromatics in Finished Gasolines by Gas1This practice is under the jurisdiction of ASTM Committee D02 on PetroleumProducts and Lubricants and is the direct resp

16、onsibility of Subcommittee D02.94 onQuality Assurance and Statistics.Current edition approved July 1, 2006. Published July 2006. Originally approvedin 2001. Last previous edition approved in 2005 as D 670805.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Custome

17、r Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.Chromatography/Mass SpectrometryD 6299

18、 Practice for Applying Statistical Quality AssuranceTechniques to Evaluate Analytical Measurement SystemPerformanceD 6300 Practice for Determination of Precision and BiasData for Use in Test Methods for Petroleum Products andLubricants2.2 ISO Standard3ISO 4259 Petroleum ProductsDetermination and app

19、lica-tion of precision data in relation to methods of test.2.3 ASTM Adjuncts:ADJD6708 CompTM Version 1.0.2143. Terminology3.1 Definitions:3.1.1 closeness sum of squares (CSS), na statistic used toquantify the degree of agreement between the results from twotest methods after bias-correction using th

20、e methodology ofthis practice.3.1.2 cross-method reproducibility (RXY), na quantitativeexpression of the random error associated with the differencebetween two results obtained by different operators usingdifferent apparatus and applying the two methods X and Y,respectively, each obtaining a single

21、result on an identical testsample, when the methods have been assessed and an appro-priate bias-correction has been applied in accordance with thispractice; it is defined as the 95 % confidence limit for thedifference between two such single and independent results.3.1.2.1 DiscussionAstatement of cr

22、oss-method reproduc-ibility must include a description of any bias correction used inaccordance with this practice.3.1.2.2 DiscussionCross-method reproducibility is ameaningful concept only if there are no statistically observablesample-specific relative biases between the two methods, or ifsuch bia

23、ses vary from one sample to another in such a way thatthey may be considered random effects. (see 6.7.)3.1.3 total sum of squares (TSS), na statistic used toquantify the information content from the inter-laboratorystudy in terms of total variation of sample means relative to thestandard error of ea

24、ch sample mean.3.2 Symbols:X,Y = single X-method and Y-method results,respectivelyXijk,Yijk= single results from the X-method andY-method round robins, respectivelyXi,Yi= means of results on the ithround robinsampleS = the number of samples in the round robinLXi,LYi= the numbers of laboratories that

25、 returnedresults on the ithround robin sampleRX,RY= the reproducibilities of the X- andY- meth-ods, respectivelysRXi,sRYi= the reproducibility standard deviations,evaluated at the means of the ithroundrobin samplesrXi,srYi= the repeatability standard deviations,evaluated at the means of the ithround

26、robin samplesXi,sYi= standard errors of the means ithround robinsampleX,Y= the weighted means of round robins(across samples)xi,yi= deviations of the means of the ithroundrobin sample results from Xand Y, respec-tively.TSSX, TSSY= total sums of squares, around Xand YF = a ratio for comparing varianc

27、es; notuniquemore than one usevX,vY= the degrees of freedom for reproducibilityvariances from the round robinswi= weight associated with the difference be-tween mean results (or corrected meanresults) from the ithround robin sampleCSS = weighted sum of squared differences be-tween (possibly correcte

28、d) mean resultsfrom the round robina,b = parameters of a linear correction: Y= a +bXt1,t2= ratios for assessing reductions in sums ofsquaresRXY= estimate of cross-method reproducibilityY= Y-method value predicted from X-methodresultYi= ithround robin sample Y-method mean,predicted from corresponding

29、 X-methodmeanei= standardized difference between Yiand Yi.LX,LY= harmonic mean numbers of laboratoriessubmitting results on round robin samples,by X- and Y- methods, respectivelyRXY= estimate of cross-method reproducibility,computed from an X-method result only4. Summary of Practice4.1 Precisions of

30、 the two methods are quantified usinginter-laboratory studies meeting the requirements of PracticeD 6300 or equivalent, using at least ten samples in commonthat span the intersecting scopes of the methods. The arithmeticmeans of the results for each common sample obtained by eachmethod are calculate

31、d. Estimates of the standard errors of thesemeans are computed.NOTE 4For established standard test methods, new precision studiesgenerally will be required in order to meet the common sample require-ment.NOTE 5Both test methods do not need to be run by the samelaboratory. If they are, care should be

32、 taken to ensure the independent testresult requirement of Practice D 6300 is met (for example, by double-blind testing of samples in random order).4.2 Weighted sums of squares are computed for the totalvariation of the mean results across all common samples foreach method. These sums of squares are

33、 assessed against the3Available from American National Standards Institute (ANSI), 25 W. 43rd St.,4th Floor, New York, NY 10036.4Available from ASTM International Headquarters. Order Adjunct No.ADJD6708.D6708062standard errors of the mean results for each method to ensurethat the samples are suffici

34、ently varied before continuing withthe practice.4.3 The closeness of agreement of the mean results by eachmethod is evaluated using appropriate weighted sums ofsquared differences. Such sums of squares are computed fromthe data first with no bias correction, then with a constant biascorrection, then

35、, when appropriate, with a proportional correc-tion, and finally with a linear (proportional + constant) correc-tion.4.4 The weighted sums of squared differences for the linearcorrection is assessed against the total variation in the meanresults for both methods to ensure that there is sufficientcor

36、relation between the two methods.4.5 The most parsimonious bias correction is selected.4.6 The weighted sum of squares of differences, afterapplying the selected bias correction, is assessed to determinewhether additional unexplained sources of variation remain inthe residual (that is, the individua

37、l Yiminus bias-corrected Xi)data. Any remaining, unexplained variation is attributed tosample-specific biases (also known as method-material inter-actions, or matrix effects). In the absence of sample-specificbiases, the cross-method reproducibility is estimated.4.7 If sample-specific biases are pre

38、sent, the residuals (thatis, the individual Yiminus bias-corrected Xi) are tested forrandomness. If they are found to be consistent with a random-effects model, then their contribution to the cross-methodreproducibility is estimated, and accumulated into an all-encompassing cross-method reproducibil

39、ity estimate.4.8 Refer to Fig. 1 for a simplified flow diagram of theprocess described in this practice.5. Significance and Use5.1 This practice can be used to determine if a constant,proportional, or linear bias correction can improve the degreeof agreement between two methods that purport to measu

40、re thesame property of a material.5.2 The bias correction developed in this practice can beapplied to a single result (X) obtained from one test method(method X) to obtain a predicted result ( Y) for the other testmethod (method Y).NOTE 6Users are cautioned to ensure that Yis within the scope ofmeth

41、od Y before its use.5.3 The cross-method reproducibility established by thispractice can be used to construct an interval around Ythatwould contain the result of test method Y, if it were conducted,with about 95 % confidence.5.4 This practice can be used to guide commercial agree-ments and product d

42、isposition decisions involving test methodsthat have been evaluated relative to each other in accordancewith this practice.6. ProcedureNOTE 7For an in-depth statistical discussion of the methodology usedin this section, see Appendix X1. For a worked example, see AppendixX2.6.1 Calculate sample means

43、 and standard errors from Prac-tice D 6300 results.6.1.1 The process of applying Practice D 6300 to the datamay involve elimination of some results as outliers, and it mayalso involve applying a transformation to the data. For thispractice, compute the mean results from data that have notbeen transf

44、ormed, but with outliers removed in accordancewith Practice D 6300. The precision estimates from PracticeD 6300 are used to estimate the standard errors of these means.6.1.2 Compute the means as follows:6.1.2.1 Let Xijkrepresent the kthresult on the ithcommonmaterial by the jthlab in the round robin

45、 for method X.Similarly for Yijk. (The ithmaterial is the same for both roundrobins, but the jthlab in one round robin is not necessarily thesame lab as the jthlab in the other round robin.) Let nXijbe thenumber of results on the ithmaterial from the jthX-method lab,after removing outliers that is,

46、the number of results in cell (i,j).Let LXibe the number of laboratories in the X-method roundrobin that have at least one result on the ithmaterial remainingin the data set, after removal of outliers. Let S be the totalnumber of materials common to both round robins.6.1.2.2 The mean X-method result

47、 for the ithmaterial is:Xi51Lxi(j(kXijknXij(1)where, Xiis the average of the cell averages on the ithmaterial by method X.6.1.2.3 Similarly, the mean Y-method result for the ithmaterial is:Yi51LYi(j(kYijknYij(2)6.1.3 The standard errors (standard deviations of the meansof the results) are computed a

48、s follows:6.1.3.1 If sRXiis the estimated reproducibility standarddeviation from the X-method round robin, and srXiis theestimated repeatibility standard deviation, then an estimate ofthe standard error for Xiis given by:sXi51LXiFsRXi22 srXi2S1 21LXi(j1nXijDG(3)NOTE 8Since repeatability and reproduc

49、ibility may vary with X, evenif the LXiwere the same for all materials and the nXijwere the same for alllaboratories and all materials, the sXi might still differ from one materialto the next.6.1.3.2 sYi, the estimated standard error for Yi, is given by ananalogous formula.6.2 Calculate the total variation sum of squares for eachmethod, and determine whether the samples can be distin-guished from each other by both methods.6.2.1 The total sums of squares (TSS) are given by:TSSx5(iSXi2 XsXiD2and TSSy5(iSYi2 YsYiD2(4)where:X 5(iSXisXi2 D(iS1sXi2 Dand Y 5(i