1、Designation: E 2617 08Standard Practice forValidation of Empirically Derived Multivariate Calibrations1This standard is issued under the fixed designation E 2617; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last rev
2、ision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice covers requirements for the validation ofempirically derived calibrations (Note 1) such as calibrationsderived b
3、y Multiple Linear Regression (MLR), Principal Com-ponent Regression (PCR), Partial Least Squares (PLS), Artifi-cial Neural Networks (ANN), or any other empirical calibra-tion technique whereby a relationship is postulated between aset of variables measured for a given sample under test and oneor mor
4、e physical, chemical, quality, or membership propertiesapplicable to that sample.NOTE 1Empirically derived calibrations are sometimes referred to as“models” or “calibrations.” In the following text, for conciseness, the term“calibration” may be used instead of the full name of the procedure.1.2 This
5、 practice does not cover procedures for establishingsaid postulated relationship.1.3 This practice serves as an overview of techniques usedto verify the applicability of an empirically derived multivari-ate calibration to the measurement of a sample under test andto verify equivalence between the pr
6、operties calculated fromthe empirically derived multivariate calibration and the resultsof an accepted reference method of measurement to withincontrol limits established for the prespecified statistical confi-dence level.1.4 This standard does not purport to address all of thesafety concerns, if an
7、y, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2E 131 Terminology Relating to Molecular Spectrosco
8、pyE 1655 Practices for Infrared Multivariate QuantitativeAnalysisE 1790 Practice for Near Infrared Qualitative Analysis3. Terminology3.1 For terminology related to molecular spectroscopicmethods, refer to Terminology E 131. For terminology relatedto multivariate quantitative modeling refer to Practi
9、ces E 1655.While Practices E 1655 is written in the context of multivariatespectroscopic methods, the terminology is also applicable toother multivariate technologies.3.2 Definitions of Terms Specific to This Standard:3.2.1 accuracythe closeness of agreement between a testresult and an accepted refe
10、rence value.3.2.2 biasthe arithmetic average difference between thereference values and the values produced by the analyticalmethod under test, for a set of samples.3.2.3 detection limitthe lowest level of a property in asample that can be detected, but not necessarily quantified, bythe measurement
11、system.3.2.4 estimatethe constituent concentration, identifica-tion, or other property of a sample as determined by theanalytical method being validated.3.2.5 initial validationvalidation that is performed whenan analyzer system is initially installed or after major mainte-nance.3.2.6 Negative Fract
12、ion Identifiedthe fraction of samplesnot having a particular characteristic that is identified as nothaving that characteristic.3.2.6.1 DiscussionNegative Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent. It is not applicable to tests with multiple
13、 possibleoutcomes.3.2.7 ongoing periodic revalidationthe quality assuranceprocess by which, in the case of quantitative calibrations, thebias and precision or, in the case of qualitative calibrations, thePositive Fraction Identified and Negative Fraction Identifiedperformance determined during initi
14、al validation are shown tobe sustained.3.2.8 Positive Fraction Identifiedthe fraction of sampleshaving a particular characteristic that is identified as havingthat characteristic.3.2.8.1 DiscussionPositive Fraction Identified assumesthat the characteristic that the test measures either is or is notp
15、resent. It is not applicable to tests with multiple possibleoutcomes.1This practice is under the jurisdiction of ASTM Committee E13 on MolecularSpectroscopy and Separation Science and is the direct responsibility of Subcom-mittee E13.11 on Multivariate Analysis.Current edition approved May 15, 2008.
16、 Published June 2008.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.1Copyright ASTM International, 100 Barr
17、Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.3.2.9 precisionthe closeness of agreement between inde-pendent test results obtained under stipulated conditions.3.2.9.1 DiscussionPrecision may be a measure of eitherthe degree of reproducibility or degree of repeatability o
18、f theanalytical method under normal operating conditions. In thiscontext, reproducibility refers to the use of the analyticalprocedure in different laboratories, as in a collaborative study.3.2.10 quantification limitthe lowest level of a sampleproperty which can be determined with acceptable precis
19、ionand accuracy under the stated experimental conditions.3.2.11 rangethe interval between the upper and lowerlevels of a property (including these levels) that has beendemonstrated to be determined with a suitable level of preci-sion and accuracy using the method as specified.3.2.12 reference valuet
20、he metric of a property as deter-mined by well-characterized method, the accuracy of whichhas been stated or defined, that is, another, already-validatedmethod.3.2.13 validationthe statistically quantified judgment thatan empirically derived multivariate calibration is applicable tothe measurement o
21、n which the calibration is to be applied andcan perform property estimates with, in the case of quantitativecalibrations, acceptable precision, accuracy and bias or, in thecase of qualitative calibrations, acceptable Positive FractionIdentified and Negative Fraction Identified, as compared withresul
22、ts from an accepted reference method.4. Summary of Practice4.1 Validating an empirically derived multivariate calibra-tion (model) consists of four major procedures: validation atinitial development, revalidation at initial deployment or aftera revision, ongoing periodic revalidation, and qualificat
23、ion ofeach measurement before using the calibration to estimate theproperty(s) of the sample being measured.5. Significance and Use5.1 This practice outlines a universally applicable procedureto validate the performance of a quantitative or qualitative,empirically derived, multivariate calibration r
24、elative to anaccepted reference method.5.2 This practice provides procedures for evaluating thecapability of a calibration to provide reliable estimationsrelative to an accepted reference method.5.3 This practice provides purchasers of a measurementsystem that incorporates an empirically derived mul
25、tivariatecalibration with options for specifying validation requirementsto ensure that the system is capable of providing estimationswith an appropriate degree of agreement with an acceptedreference method.5.4 This practice provides the user of a measurement systemthat incorporates an empirically de
26、rived multivariate calibra-tion with procedures capable of providing information that maybe useful for ongoing quality assurance of the performance ofthe measurement system.5.5 Validation information obtained in the application ofthis practice is applicable only to the material type and propertyrang
27、e of the materials used to perform the validation and onlyfor the individual measurement system on which the practice iscompletely applied. It is the users responsibility to select theproperty levels and the compositional characteristics of thevalidation samples such that they are suitable to the ap
28、plica-tion. This practice allows the user to write a comprehensivevalidation statement for the analyzer system including specificlimits for the validated range of application and specificrestrictions to the permitted uses of the measurement system.Users are cautioned against extrapolation of validat
29、ion resultsbeyond the material type(s) and property range(s) used toobtain these results.5.6 Users are cautioned that a validated empirically derivedmultivariate calibration is applicable only to samples that fallwithin the subset population represented in the validation set.The estimation from an e
30、mpirically derived multivariate cali-bration can only be validated when the applicability of thecalibration is explicitly established for the particular measure-ment for which the estimation is produced. Applicabilitycannot be assumed.6. Methods and Considerations6.1 When validating an empirically d
31、erived multivariatecalibration, it is the responsibility of the user to describe themeasurement system and the required level of agreementbetween the estimations produced by the calibration and theaccepted reference method(s).6.2 When validating a measurement system incorporatingan empirically deriv
32、ed multivariate calibration, it is the respon-sibility of the user to satisfy the requirements of any applicabletests specific to the measurement system including any Instal-lation Qualification (IQ), Operational Qualification (OQ), andPerformance Qualification (PQ) requirements; which may bemandate
33、d by competent regulatory authorities, an applicableQuality Assurance (QA), or Standard Operating Procedure(SOP) or be recommended by the instrument or equipmentmanufacturer.6.3 Reference Values and Quality Controls for the AcceptedReference Method:6.3.1 The reference (or true) value which is compar
34、ed witheach respective estimate produced by the empirically derivedmultivariate calibration is established by applying an acceptedreference method, the characteristics of which are known andstated, to the sample from which the measurement systemderives the measurement.6.3.2 To ensure the reliability
35、 of the reference valuesprovided by an accepted reference method, appropriate qualitycontrols should be applied to the accepted reference method.7. Procedure7.1 The objective of the validation procedure is to quantifythe performance of an empirically derived multivariate calibra-tion in terms of, in
36、 the case of quantitative calibrations,precision, accuracy and bias or, in the case of qualitativecalibrations, Positive Fraction Identified and Negative FractionIdentified relative to an accepted reference method for eachproperty of interest. The user must specify, based on theintended use of the c
37、alibration, acceptable precision and bias orPositive Fraction Identified and Negative Fraction Identifiedperformance criteria before initiating the validation. Thesecriteria will be dependent on the intended use of the analyzerand may be based, all or in part, on risk based criteria.E26170827.1.1 Th
38、e acceptable performance criteria specified by theuser may be constant over the entire range of sample variabil-ity.Alternatively, different acceptable performance criteria maybe specified by the user for different sub-ranges of the fullsample variability.7.2 Validation of calibration is accomplishe
39、d by using thecalibration to estimate the property(s) of a set of validationsamples and statistically comparing the estimates for thesesamples to known reference values. Validation requires thor-ough testing of the model with a sufficient number of repre-sentative validation samples to ensure that i
40、t performs ad-equately over the entire range of possible sample variability.7.3 Initial Validation Sample Set:7.3.1 For the initial validation of a multivariate model, anideal validation sample set will:7.3.1.1 Contain samples that provide sufficient examples ofall combinations of variation in the s
41、ample properties whichare expected to be present in the samples which are to beanalyzed using the calibration;7.3.1.2 Contain samples for which the ranges of variation inthe sample properties is comparable to the ranges of variationexpected for samples that are to be analyzed using the model;7.3.1.3
42、 Contain samples for which the respective variationsof the sample properties are uniformly and mutually indepen-dently distributed over their full respective ranges or, whenapplicable, subranges of variation; and7.3.1.4 Contain a sufficient number of samples to statisti-cally test the relationships
43、between the measured variables andthe properties that are modeled by the calibration.7.3.2 For simple systems, sufficient validation samples cangenerally be obtained to meet the criteria in 7.3.1.1-7.3.1.4. Forcomplex mixtures, obtaining an ideal validation set may bedifficult if not impossible. In
44、such cases, it may be necessary tovalidate discrete subranges of the calibration incrementally,over time as samples become available.7.3.3 The number of samples needed to validate a calibra-tion depends on the complexity of the calibration, the ranges ofproperty variation over which the calibration
45、is to be applied,and the degree of confidence required. It is important tovalidate a calibration with as many samples as possible tomaximize the likelihood of challenging the calibration withrarely occurring, but potentially troublesome samples. Thenumber and range of validation samples should be su
46、fficient tovalidate the calibration to the statistical degree of confidencerequired for the application. In all cases, a minimum of 20validation samples is recommended. In addition, the validationsamples should:7.3.3.1 Multivariately span the ranges of sample propertyvalues over which the calibratio
47、n will be used; that is, the spanand the standard deviation of the ranges of sample propertyvalues for the validation samples should be at least 100 % ofthe spans of the sample property values over which thecalibration will be used, and the sample property values for thevalidation samples should be
48、distributed as uniformly aspossible throughout their respective ranges, and the variationsof the sample property values among the samples should be asmutually independent as possible; and7.3.3.2 Span the ranges of the independent variables overwhich the calibration will be used; that is, if the rang
49、e of anindependent variable is expected to vary from a to b, and thestandard deviation of the independent variable is c, then thevariations of that independent variable in the set of validationsamples should cover at least 100 % of the range from a to b,and should be distributed as uniformly as possible across therange such that the standard deviation in that independentvariable estimated for the validation samples will be at least95 % of c.(1) When validating a calibration for which detection limitor quantification limit is an important consideration, the usershould
