ASTM D6708-2007 838 Standard Practice for Statistical Assessment and Improvement of Expected Agreement Between Two Test Methods that Purport to Measure the Same Property of a Mater.pdf

1、Designation: D 6708 07An American National StandardStandard Practice forStatistical Assessment and Improvement of ExpectedAgreement Between Two Test Methods that Purport toMeasure the Same Property of a Material1This standard is issued under the fixed designation D 6708; the number immediately follo

2、wing the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon (e) indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice co

3、vers statistical methodology for assess-ing the expected agreement between two standard test methodsthat purport to measure the same property of a material, anddeciding if a simple linear bias correction can further improvethe expected agreement. It is intended for use with resultscollected from an

4、interlaboratory study meeting the require-ment of Practice D 6300 or equivalent (for example,ISO 4259). The interlaboratory study must be conducted on atleast ten materials that span the intersecting scopes of the testmethods, and results must be obtained from at least sixlaboratories using each met

5、hod.NOTE 1Examples of standard test methods are those developed byvoluntary consensus standards bodies such as ASTM, IP/BSI, DIN,AFNOR, CGSB.1.2 The statistical methodology is based on the premise thata bias correction will not be needed. In the absence of strongstatistical evidence that a bias corr

6、ection would result in betteragreement between the two methods, a bias correction is notmade. If a bias correction is required, then the parsimonyprinciple is followed whereby a simple correction is to befavored over a more complex one.NOTE 2Failure to adhere to the parsimony principle generally res

7、ultsin models that are over-fitted and do not perform well in practice.1.3 The bias corrections of this practice are limited to aconstant correction, proportional correction or a linear (propor-tional + constant) correction.1.4 The bias-correction methods of this practice are methodsymmetric, in the

8、 sense that equivalent corrections are obtainedregardless of which method is bias-corrected to match theother.1.5 A methodology is presented for establishing the 95 %confidence limit (designated by this practice as the betweenmethods reproducibility) for the difference between two resultswhere each

9、result is obtained by a different operator usingdifferent apparatus and each applying one of the two methodsX and Y on identical material, where one of the methods hasbeen appropriately bias-corrected in accordance with thispractice.NOTE 3In earlier versions of this standard practice, the term “cros

10、s-method reproducibility” was used in place of the term “between methodsreproducibility.” The change was made because the “between methodsreproducibility” term is more intuitive and less confusing. It is importantto note that these two terms are synonymous and interchangeable with oneanother, especi

11、ally in cases where the “cross-method reproducibility” termwas subsequently referenced by name in methods where a D 6708assessment was performed, before the change in terminology in thisstandard practice was adopted.NOTE 4Users are cautioned against applying the between methodsreproducibility as cal

12、culated from this practice to materials that aresignificantly different in composition from those actually studied, as theability of this practice to detect and address sample-specific biases (see6.8) is dependent on the materials selected for the interlaboratory study.When sample-specific biases ar

13、e present, the types and ranges of samplesmay need to be expanded significantly from the minimum of ten asspecified in this practice in order to obtain a more comprehensive andreliable 95 % confidence limits for between methods reproducibility thatadequately cover the range of sample specific biases

14、 for different types ofmaterials.1.6 This practice is intended for test methods which mea-sure quantitative (numerical) properties of petroleum or petro-leum products.1.7 The statistical methodology outlined in this practice isalso applicable for assessing the expected agreement betweenany two test

15、methods that purport to measure the same propertyof a material, provided the results are obtained on the samecomparison sample set, the standard error associated with eachtest result is known, the sample set design meets the require-ment of this practice, and the statistical degree of freedom ofthe

16、data set exceeds 30.1This practice is under the jurisdiction of ASTM Committee D02 on PetroleumProducts and Lubricants and is the direct responsibility of Subcommittee D02.94 onCoordinating Subcommittee on Quality Assurance and Statistics.Current edition approved Nov. 15, 2007. Published January 200

17、8. Originallyapproved in 2001. Last previous edition approved in 2006 as D 670806.1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.Copyright by ASTM Intl (all rights reserved); Wed Mar 19 02:53:37 EST 2008Downloaded/printed byGuo Dehu

18、a (CNIS) pursuant to License Agreement. No further reproductions authorized.1.8 Software program CompTM Version 1.0.21(ADJD6708) performs the necessary computations prescribedby this practice.2. Referenced Documents2.1 ASTM Standards:2D 5580 Test Method for Determination of Benzene, Tolu-ene, Ethylb

19、enzene, p/m-Xylene, o-Xylene, C9and HeavierAromatics, and Total Aromatics in Finished Gasoline byGas ChromatographyD 5769 Test Method for Determination of Benzene, Tolu-ene, and Total Aromatics in Finished Gasolines by GasChromatography/Mass SpectrometryD 6299 Practice for Applying Statistical Quali

20、ty AssuranceTechniques to Evaluate Analytical Measurement SystemPerformanceD 6300 Practice for Determination of Precision and BiasData for Use in Test Methods for Petroleum Products andLubricants2.2 ISO Standard3ISO 4259 Petroleum ProductsDetermination and applica-tion of precision data in relation

21、to methods of test.2.3 ASTM Adjuncts:ADJD6708 CompTM Version 1.0.2143. Terminology3.1 Definitions:3.1.1 closeness sum of squares (CSS), na statistic used toquantify the degree of agreement between the results from twotest methods after bias-correction using the methodology ofthis practice.3.1.2 betw

22、een methods reproducibility (RXY), na quantita-tive expression of the random error associated with thedifference between two results obtained by different operatorsusing different apparatus and applying the two methods X andY, respectively, each obtaining a single result on an identicaltest sample,

23、when the methods have been assessed and anappropriate bias-correction has been applied in accordancewith this practice; it is defined as the 95 % confidence limit forthe difference between two such single and independentresults.3.1.2.1 DiscussionAstatement of between methods repro-ducibility must in

24、clude a description of any bias correctionused in accordance with this practice.3.1.2.2 DiscussionBetween methods reproducibility is ameaningful concept only if there are no statistically observablesample-specific relative biases between the two methods, or ifsuch biases vary from one sample to anot

25、her in such a way thatthey may be considered random effects. (see 6.7.)3.1.3 total sum of squares (TSS), na statistic used toquantify the information content from the inter-laboratorystudy in terms of total variation of sample means relative to thestandard error of each sample mean.3.2 Symbols:X,Y =

26、 single X-method and Y-method results,respectivelyXijk,Yijk= single results from the X-method andY-method round robins, respectivelyXi,Yi= means of results on the ithround robinsampleS = the number of samples in the round robinLXi,LYi= the numbers of laboratories that returnedresults on the ithround

27、 robin sampleRX,RY= the reproducibilities of the X- andY- meth-ods, respectivelysRXi,sRYi= the reproducibility standard deviations,evaluated at the means of the ithroundrobin samplesrXi,srYi= the repeatability standard deviations,evaluated at the means of the ithroundrobin samplesXi,sYi= standard er

28、rors of the means ithround robinsampleX,Y= the weighted means of round robins(across samples)xi,yi= deviations of the means of the ithroundrobin sample results from Xand Y, respec-tively.TSSX, TSSY= total sums of squares, around Xand YF = a ratio for comparing variances; notuniquemore than one usevX

29、,vY= the degrees of freedom for reproducibilityvariances from the round robinswi= weight associated with the difference be-tween mean results (or corrected meanresults) from the ithround robin sampleCSS = weighted sum of squared differences be-tween (possibly corrected) mean resultsfrom the round ro

30、bina,b = parameters of a linear correction: Y= a +bXt1,t2= ratios for assessing reductions in sums ofsquaresRXY= estimate of between methods reproducibil-ityY= Y-method value predicted from X-methodresultYi= ithround robin sample Y-method mean,predicted from corresponding X-methodmeanei= standardize

31、d difference between Yiand Yi.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.3Available from American Nation

32、al Standards Institute (ANSI), 25 W. 43rd St.,4th Floor, New York, NY 10036.4Available from ASTM International Headquarters. Order Adjunct No.ADJD6708.D6708072Copyright by ASTM Intl (all rights reserved); Wed Mar 19 02:53:37 EST 2008Downloaded/printed byGuo Dehua (CNIS) pursuant to License Agreement

33、. No further reproductions authorized.LX,LY= harmonic mean numbers of laboratoriessubmitting results on round robin samples,by X- and Y- methods, respectivelyRXY= estimate of between methods reproducibil-ity, computed from an X-method resultonly4. Summary of Practice4.1 Precisions of the two methods

34、 are quantified usinginter-laboratory studies meeting the requirements of PracticeD 6300 or equivalent, using at least ten samples in commonthat span the intersecting scopes of the methods. The arithmeticmeans of the results for each common sample obtained by eachmethod are calculated. Estimates of

35、the standard errors of thesemeans are computed.NOTE 5For established standard test methods, new precision studiesgenerally will be required in order to meet the common sample require-ment.NOTE 6Both test methods do not need to be run by the samelaboratory. If they are, care should be taken to ensure

36、 the independent testresult requirement of Practice D 6300 is met (for example, by double-blind testing of samples in random order).4.2 Weighted sums of squares are computed for the totalvariation of the mean results across all common samples foreach method. These sums of squares are assessed agains

37、t thestandard errors of the mean results for each method to ensurethat the samples are sufficiently varied before continuing withthe practice.4.3 The closeness of agreement of the mean results by eachmethod is evaluated using appropriate weighted sums ofsquared differences. Such sums of squares are

38、computed fromthe data first with no bias correction, then with a constant biascorrection, then, when appropriate, with a proportional correc-tion, and finally with a linear (proportional + constant) correc-tion.4.4 The weighted sums of squared differences for the linearcorrection is assessed against

39、 the total variation in the meanresults for both methods to ensure that there is sufficientcorrelation between the two methods.4.5 The most parsimonious bias correction is selected.4.6 The weighted sum of squares of differences, afterapplying the selected bias correction, is assessed to determinewhe

40、ther additional unexplained sources of variation remain inthe residual (that is, the individual Yiminus bias-corrected Xi)data. Any remaining, unexplained variation is attributed tosample-specific biases (also known as method-material inter-actions, or matrix effects). In the absence of sample-speci

41、ficbiases, the between methods reproducibility is estimated.4.7 If sample-specific biases are present, the residuals (thatis, the individual Yiminus bias-corrected Xi) are tested forrandomness. If they are found to be consistent with a random-effects model, then their contribution to the between met

42、hodsreproducibility is estimated, and accumulated into an all-encompassing between methods reproducibility estimate.4.8 Refer to Fig. 1 for a simplified flow diagram of theprocess described in this practice.5. Significance and Use5.1 This practice can be used to determine if a constant,proportional,

43、 or linear bias correction can improve the degreeof agreement between two methods that purport to measure thesame property of a material.5.2 The bias correction developed in this practice can beapplied to a single result (X) obtained from one test method(method X) to obtain a predicted result ( Y) f

44、or the other testmethod (method Y).NOTE 7Users are cautioned to ensure that Yis within the scope ofmethod Y before its use.5.3 The between methods reproducibility established by thispractice can be used to construct an interval around Ythatwould contain the result of test method Y, if it were conduc

45、ted,with about 95 % confidence.5.4 This practice can be used to guide commercial agree-ments and product disposition decisions involving test methodsthat have been evaluated relative to each other in accordancewith this practice.6. ProcedureNOTE 8For an in-depth statistical discussion of the methodo

46、logy usedin this section, see Appendix X1. For a worked example, see AppendixX2.6.1 Calculate sample means and standard errors from Prac-tice D 6300 results.6.1.1 The process of applying Practice D 6300 to the datamay involve elimination of some results as outliers, and it mayalso involve applying a

47、 transformation to the data. For thispractice, compute the mean results from data that have notbeen transformed, but with outliers removed in accordancewith Practice D 6300. The precision estimates from PracticeD 6300 are used to estimate the standard errors of these means.6.1.2 Compute the means as

48、 follows:6.1.2.1 Let Xijkrepresent the kthresult on the ithcommonmaterial by the jthlab in the round robin for method X.Similarly for Yijk. (The ithmaterial is the same for both roundrobins, but the jthlab in one round robin is not necessarily thesame lab as the jthlab in the other round robin.) Let

49、 nXijbe thenumber of results on the ithmaterial from the jthX-method lab,after removing outliers that is, the number of results in cell (i,j).Let LXibe the number of laboratories in the X-method roundrobin that have at least one result on the ithmaterial remainingin the data set, after removal of outliers. Let S be the totalnumber of materials common to both round robins.6.1.2.2 The mean X-method result for the ithmaterial is:Xi51Lxi(j(kXijknXij(1)where, Xiis the average of the cell averages on the ithmaterial by method X.6.1.2.3 Similarly, the mean Y-met