1、Designation: E 691 09An American National StandardStandard Practice forConducting an Interlaboratory Study to Determine thePrecision of a Test Method1This standard is issued under the fixed designation E 691; the number immediately following the designation indicates the year oforiginal adoption or,
2、 in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.This standard has been approved for use by agencies of the Department of Defense.INTRODUCTIO
3、NTests performed on presumably identical materials in presumably identical circumstances do not, ingeneral, yield identical results. This is attributed to unavoidable random errors inherent in every testprocedure; the factors that may influence the outcome of a test cannot all be completely controll
4、ed. Inthe practical interpretation of test data, this inherent variability has to be taken into account. Forinstance, the difference between a test result and some specified value may be within that which canbe expected due to unavoidable random errors, in which case a real deviation from the specif
5、ied valuehas not been demonstrated. Similarly, the difference between test results from two batches of materialwill not indicate a fundamental quality difference if the difference is no more than can be attributedto inherent variability in the test procedure. Many different factors (apart from rando
6、m variationsbetween supposedly identical specimens) may contribute to the variability in application of a testmethod, including: a the operator, b equipment used, c calibration of the equipment, and denvironment (temperature, humidity, air pollution, etc.). It is considered that changing laboratorie
7、schanges each of the above factors. The variability between test results obtained by different operatorsor with different equipment will usually be greater than between test results obtained by a singleoperator using the same equipment. The variability between test results taken over a long period o
8、ftime even by the same operator will usually be greater than that obtained over a short period of timebecause of the greater possibility of changes in each of the above factors, especially the environment.The general term for expressing the closeness of test results to the“ true” value or the accept
9、edreference value is accuracy. To be of practical value, standard procedures are required for determiningthe accuracy of a test method, both in terms of its bias and in terms of its precision. This practiceprovides a standard procedure for determining the precision of a test method. Precision, whene
10、valuating test methods, is expressed in terms of two measurement concepts, repeatability andreproducibility. Under repeatability conditions the factors listed above are kept or remain reasonablyconstant and usually contribute only minimally to the variability. Under reproducibility conditions thefac
11、tors are generally different (that is, they change from laboratory to laboratory) and usuallycontribute appreciably to the variability of test results. Thus, repeatability and reproducibility are twopractical extremes of precision.The repeatability measure, by excluding the factors a through d as co
12、ntributing variables, is notintended as a mechanism for verifying the ability of a laboratory to maintain“ in-control” conditionsfor routine operational factors such as operator-to-operator and equipment differences or any effectsof longer time intervals between test results. Such a control study is
13、 a separate issue for eachlaboratory to consider for itself, and is not a recommended part of an interlaboratory study.The reproducibility measure (including the factors a through d as sources of variability) reflectswhat precision might be expected when random portions of a homogeneous sample are s
14、ent to random“in-control” laboratories.To obtain reasonable estimates of repeatability and reproducibility precision, it is necessary in aninterlaboratory study to guard against excessively sanitized data in the sense that only the uniquelybest operators are involved or that a laboratory takes unusu
15、al steps to get“ good” results. It is alsoimportant to recognize and consider how to treat “poor” results that may have unacceptable assignable1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.causes (for example, departures from the p
16、rescribed procedure). The inclusion of such results in thefinal precision estimates might be questioned.An essential aspect of collecting useful consistent data is careful planning and conduct of the study.Questions concerning the number of laboratories required for a successful study as well as the
17、 numberof test results per laboratory affect the confidence in the precision statements resulting from the study.Other issues involve the number, range, and types of materials to be selected for the study, and theneed for a well-written test method and careful instructions to the participating labor
18、atories.To evaluate the consistency of the data obtained in an interlaboratory study, two statistics may beused: the “k-value”, used to examine the consistency of the within-laboratory precision fromlaboratory to laboratory, and the “h-value”, used to examine the consistency of the test results from
19、laboratory to laboratory. Graphical as well as tabular diagnostic tools help in these examinations.1. Scope1.1 This practice describes the techniques for planning,conducting, analyzing, and treating the results of an interlabo-ratory study (ILS) of a test method. The statistical techniquesdescribed
20、in this practice provide adequate information forformulating the precision statement of a test method.1.2 This practice does not concern itself with the develop-ment of test methods but rather with gathering the informationneeded for a test method precision statement after the devel-opment stage has
21、 been successfully completed. The dataobtained in the interlaboratory study may indicate, however,that further effort is needed to improve the test method.1.3 Since the primary purpose of this practice is the devel-opment of the information needed for a precision statement, theexperimental design in
22、 this practice may not be optimum forevaluating materials, apparatus, or individual laboratories.1.4 Field of ApplicationThis practice is concerned exclu-sively with test methods which yield a single numerical figureas the test result, although the single figure may be the outcomeof a calculation fr
23、om a set of measurements.1.4.1 This practice does not cover methods in which themeasurement is a categorization, such as a go-no-go allocation(two categories) or a sorting scheme into two or morecategories. For practical purposes, the discontinuous nature ofmeasurements of these types may be ignored
24、 when a test resultis defined as an average of several individual measurements.Then, this practice may be applicable, but caution is requiredand a statistician should be consulted.1.5 The information in this practice is arranged as follows:SectionScope 1Referenced Documents 2Terminology 3Summary of
25、Practice 4Significance and Use 5Planning the Interlaboratory Study (ILS)ILS Membership 6Basic Design 7Test Method 8Laboratories 9SectionMaterials 10Number of Test Results per Material 11Protocol 12Conducting the Testing Phase of the ILSPilot Run 13Full Scale Run 14SectionCalculation and Display of S
26、tatisticsCalculation of the Statistics 15Tabular and Graphical Display of Statistics 16Data ConsistencyFlagging Inconsistent Results 17Investigation 18Task Group Actions 19Examples of Interlaboratory Studies 20Precision Statement InformationRepeatability and Reproducibility 21AppendixesTheoretical C
27、onsiderations A1Index to Selected Terms A2ReferencesTables and FiguresTables TableGlucose in Serum Example 14 that the loss or poor performance of a few will notbe fatal to the study, and to provide a reasonably satisfactoryestimate of the reproducibility.9.1.2 Under no circumstances should the fina
28、l statementof precision of a test method be based on acceptable testresults for each material from fewer than 6 laboratories.This would require that the ILS begin with 8 or morelaboratories in order to allow for attrition.9.1.3 The examples given in this practice include only 8 and7 laboratories, re
29、spectively. These numbers are smaller thanordinarily considered acceptable, but they are convenient forillustrating the calculations and treatment of the data.9.2 Any laboratory considered qualified to run the testroutinely (including laboratories that may not be members ofASTM) should be encouraged
30、 to participate in the ILS, if thepreparatory work is not excessive and enough suitably homo-geneous material is available. In order to obtain an adequatenumber of participating laboratories, advertise the proposedILS in where appropriate (for example, trade magazines,meetings, circulars, etc.).9.3
31、“Qualified” implies proper laboratory facilities and test-ing equipment, competent operators, familiarity with the testmethod, a reputation for reliable testing work, and sufficienttime and interest to do a good job. If a laboratory meets all theother requirements, but has had insufficient experienc
32、e with thetest method, the operator in that laboratory should be given anopportunity to familiarize himself with the test method andpractice its application before the ILS starts. For example, thisexperience can be obtained by a pilot run (see Section 13)using one or two trial samples provided by th
33、e task group andreturning the raw data and the test results to the task group.The importance of this familiarization step cannot beoveremphasized. Many interlaboratory studies have turnedout to be essentially worthless due to lack of familiarization.9.4 Obtain written ensurance from each potential p
34、articipat-ing laboratory that it is properly equipped to follow all thedetails of the procedure and is willing to assign the work to askilled operator in a timely manner. The decision of a labora-tory to participate should be recorded on a response form to awritten invitation. The invitation should
35、include informationcovering the required time for calibrating the apparatus and fortesting all of the materials, and other possible costs. Theresponse form should include the name, address, and telephonenumber of the person supervising the ILS work within thelaboratory, the address and other marking
36、s required to ensurethe ILS sample material will be promptly delivered to the ILSsupervisor, answers to brief questions concerning equipment,environment, and personnel, including previous use of the testmethod, upon which the apparent competence of the laboratorymay be judged, and an affirmation tha
37、t the laboratory under-stands what is involved and agrees to carry out its responsi-bilities with diligence.9.5 The ILS should not be restricted to a group of labora-tories judged to be exceptionally qualified and equipped for theILS. Precision estimates for inclusion in a test method shouldbe obtai
38、ned through the efforts of qualified laboratories andpersonnel operating under conditions that will prevail when thetest method is used in practice.10. Materials10.1 Material designates anything with a property that canbe measured. Different materials having the same property maybe expected to have
39、different property levels, meaning higheror lower values of the property. Different dilutions of the samematerial or compound to be assayed are considered“ differentmaterials” for the purpose of this practice. The terminology“different levels of material” may be used, if appropriate.10.2 The number
40、and type of materials to be included in anILS will depend on the range of the levels in the class ofmaterials to be tested and likely relation of precision to levelover that range, the number of different types of materials toTABLE 1 Glucose in Serum ILS Test Result DataLaboratoryMaterialABCDE1 41.0
41、341.4541.3778.2878.1878.49132.66133.83133.10193.71193.59193.65292.78294.09292.892 41.1742.0041.1577.7880.3879.54132.92136.90136.40190.88200.14194.30292.27309.40295.083 41.0140.6842.6679.1879.7280.81132.61135.80135.36192.71193.28190.28295.53290.14292.344 39.3742.3742.6384.0878.6081.92138.50148.30135.
42、69195.85196.36199.43295.19295.44296.835 41.8841.1941.3278.1679.5878.33131.90134.14133.76192.59191.44195.12293.93292.48294.286 43.2840.5042.2878.6679.2781.75137.21135.14137.50195.34198.26198.13297.74296.80290.337 41.0841.2739.0279.7681.4577.35130.97131.59134.92194.66191.99187.13287.29293.76289.368 43
43、.3642.6541.7280.4480.8079.80135.46135.14133.63197.56195.99200.82298.46295.28296.12E691095which the test method is to be applied, the difficulty andexpense involved in obtaining, processing, and distributingsamples, the difficulty of, length of time required for, andexpense of performing the test, th
44、e commercial or legal needfor obtaining a reliable and comprehensive estimate of preci-sion, and the uncertainty of prior information on any of thesepoints.10.2.1 For example, if it is already known that the precisionis either relatively constant or proportional to the average levelover the range of
45、 values of interest, a smaller number ofmaterials will be needed than if it is merely known that theprecision is different at different levels. The ruggedness test(see 8.2) and the preliminary pilot program (see Section 13)help to settle some of these questions, and may often result inthe saving of
46、considerable time and expense in the full ILS.10.2.2 An ILS of a test method should include at least threematerials representing different test levels, and for develop-ment of broadly applicable precision statements, six or morematerials should be included in the study.10.2.3 The materials involved
47、in any one ILS should differprimarily only in the level of the property measured by the testmethod. When it is known, or suspected, that different classesof materials will exhibit different levels of precision whentested by the test method, consideration should be given toconducting separate interla
48、boratory studies for each class ofmaterial.10.3 Each material in an ILS should be made to be orselected to be as homogeneous as possible prior to its subdi-vision into test units or test specimens. If the randomizationand distribution of individual test specimens (rather than testunits) does not con
49、flict with the procedure for preparing thesample for test, as specified in the test method, greaterhomogeneity between test units can be achieved by randomiz-ing test specimens. Then each test unit would be composed ofthe required number of randomized test specimens. (SeeSection 11 and 14.1 for the quantity of each material needed,its preparation and distribution.)NOTE 1It may be convenient to use established reference materials,since their homogeneity has been demonstrated.11. Number of Test Results per Material11.1 In the design of an ILS a sufficient total number of testresults on
