1、Designation: E2617 10Standard Practice forValidation of Empirically Derived Multivariate Calibrations1This standard is issued under the fixed designation E2617; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revis
2、ion. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice covers requirements for the validation ofempirically derived calibrations (Note 1) such as calibrationsderived by
3、Multiple Linear Regression (MLR), Principal Com-ponent Regression (PCR), Partial Least Squares (PLS), Artifi-cial Neural Networks (ANN), or any other empirical calibra-tion technique whereby a relationship is postulated between aset of variables measured for a given sample under test and oneor more
4、physical, chemical, quality, or membership propertiesapplicable to that sample.NOTE 1Empirically derived calibrations are sometimes referred to as“models” or “calibrations.” In the following text, for conciseness, the term“calibration” may be used instead of the full name of the procedure.1.2 This p
5、ractice does not cover procedures for establishingsaid postulated relationship.1.3 This practice serves as an overview of techniques usedto verify the applicability of an empirically derived multivari-ate calibration to the measurement of a sample under test andto verify equivalence between the prop
6、erties calculated fromthe empirically derived multivariate calibration and the resultsof an accepted reference method of measurement to withincontrol limits established for the prespecified statistical confi-dence level.1.4 This standard does not purport to address all of thesafety concerns, if any,
7、 associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2E131 Terminology Relating to Molecular SpectroscopyE
8、1655 Practices for Infrared Multivariate QuantitativeAnalysisE1790 Practice for Near Infrared Qualitative Analysis3. Terminology3.1 For terminology related to molecular spectroscopicmethods, refer to Terminology E131. For terminology relatedto multivariate quantitative modeling refer to Practices E1
9、655.While Practices E1655 is written in the context of multivariatespectroscopic methods, the terminology is also applicable toother multivariate technologies.3.2 Definitions of Terms Specific to This Standard:3.2.1 accuracythe closeness of agreement between a testresult and an accepted reference va
10、lue.3.2.2 biasthe arithmetic average difference between thereference values and the values produced by the analyticalmethod under test, for a set of samples.3.2.3 detection limitthe lowest level of a property in asample that can be detected, but not necessarily quantified, bythe measurement system.3
11、.2.4 estimatethe constituent concentration, identifica-tion, or other property of a sample as determined by theanalytical method being validated.3.2.5 initial validationvalidation that is performed whenan analyzer system is initially installed or after major mainte-nance.3.2.6 Negative Fraction Iden
12、tifiedthe fraction of samplesnot having a particular characteristic that is identified as nothaving that characteristic.3.2.6.1 DiscussionNegative Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent. It is not applicable to tests with multiple possibl
13、eoutcomes.3.2.7 ongoing periodic revalidationthe quality assuranceprocess by which, in the case of quantitative calibrations, thebias and precision or, in the case of qualitative calibrations, thePositive Fraction Identified and Negative Fraction Identifiedperformance determined during initial valid
14、ation are shown tobe sustained.3.2.8 Positive Fraction Identifiedthe fraction of sampleshaving a particular characteristic that is identified as havingthat characteristic.3.2.8.1 DiscussionPositive Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent.
15、It is not applicable to tests with multiple possibleoutcomes.1This practice is under the jurisdiction of ASTM Committee E13 on MolecularSpectroscopy and Separation Science and is the direct responsibility of Subcom-mittee E13.11 on Multivariate Analysis.Current edition approved March 1, 2010. Publis
16、hed April 2010. Originallyapproved in 2008. Last previous edition approved in 2009 as E2617 09a. DOI:10.1520/E2617-10.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer t
17、o the standards Document Summary page onthe ASTM website.1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.3.2.9 precisionthe closeness of agreement between inde-pendent test results obtained under stipulated conditions.3.2.9.1 Discuss
18、ionPrecision may be a measure of eitherthe degree of reproducibility or degree of repeatability of theanalytical method under normal operating conditions. In thiscontext, reproducibility refers to the use of the analyticalprocedure in different laboratories, as in a collaborative study.3.2.10 quanti
19、fication limitthe lowest level of a sampleproperty which can be determined with acceptable precisionand accuracy under the stated experimental conditions.3.2.11 rangethe interval between the upper and lowerlevels of a property (including these levels) that has beendemonstrated to be determined with
20、a suitable level of preci-sion and accuracy using the method as specified.3.2.12 reference valuethe metric of a property as deter-mined by well-characterized method, the accuracy of whichhas been stated or defined, that is, another, already-validatedmethod.3.2.13 validationthe statistically quantifi
21、ed judgment thatan empirically derived multivariate calibration is applicable tothe measurement on which the calibration is to be applied andcan perform property estimates with, in the case of quantitativecalibrations, acceptable precision, accuracy and bias or, in thecase of qualitative calibration
22、s, acceptable Positive FractionIdentified and Negative Fraction Identified, as compared withresults from an accepted reference method.3.2.14 validation spacethe region(s) of a calibrationsmultivariate sample space populated by the independent vali-dation samples which are used to validate the calibr
23、ation.4. Summary of Practice4.1 Validating an empirically derived multivariate calibra-tion (model) consists of four major procedures: validation atinitial development, revalidation at initial deployment or aftera revision, ongoing periodic revalidation, and qualification ofeach measurement before u
24、sing the calibration to estimate theproperty(s) of the sample being measured.5. Significance and Use5.1 This practice outlines a universally applicable procedureto validate the performance of a quantitative or qualitative,empirically derived, multivariate calibration relative to anaccepted reference
25、 method.5.2 This practice provides procedures for evaluating thecapability of a calibration to provide reliable estimationsrelative to an accepted reference method.5.3 This practice provides purchasers of a measurementsystem that incorporates an empirically derived multivariatecalibration with optio
26、ns for specifying validation requirementsto ensure that the system is capable of providing estimationswith an appropriate degree of agreement with an acceptedreference method.5.4 This practice provides the user of a measurement systemthat incorporates an empirically derived multivariate calibra-tion
27、 with procedures capable of providing information that maybe useful for ongoing quality assurance of the performance ofthe measurement system.5.5 Validation information obtained in the application ofthis practice is applicable only to the material type and propertyrange of the materials used to perf
28、orm the validation and onlyfor the individual measurement system on which the practice iscompletely applied. It is the users responsibility to select theproperty levels and the compositional characteristics of thevalidation samples such that they are suitable to the applica-tion. This practice allow
29、s the user to write a comprehensivevalidation statement for the analyzer system including specificlimits for the validated range of application and specificrestrictions to the permitted uses of the measurement system.Users are cautioned against extrapolation of validation resultsbeyond the material
30、type(s) and property range(s) used toobtain these results.5.6 Users are cautioned that a validated empirically derivedmultivariate calibration is applicable only to samples that fallwithin the subset population represented in the validation set.The estimation from an empirically derived multivariate
31、 cali-bration can only be validated when the applicability of thecalibration is explicitly established for the particular measure-ment for which the estimation is produced. Applicabilitycannot be assumed.6. Methods and Considerations6.1 When validating an empirically derived multivariatecalibration,
32、 it is the responsibility of the user to describe themeasurement system and the required level of agreementbetween the estimations produced by the calibration and theaccepted reference method(s).6.2 When validating a measurement system incorporatingan empirically derived multivariate calibration, it
33、 is the respon-sibility of the user to satisfy the requirements of any applicabletests specific to the measurement system including any Instal-lation Qualification (IQ), Operational Qualification (OQ), andPerformance Qualification (PQ) requirements; which may bemandated by competent regulatory autho
34、rities, an applicableQuality Assurance (QA), or Standard Operating Procedure(SOP) or be recommended by the instrument or equipmentmanufacturer.6.3 Reference Values and Quality Controls for the AcceptedReference Method:6.3.1 The reference (or true) value which is compared witheach respective estimate
35、 produced by the empirically derivedmultivariate calibration is established by applying an acceptedreference method, the characteristics of which are known andstated, to the sample from which the measurement systemderives the measurement.6.3.2 To ensure the reliability of the reference valuesprovide
36、d by an accepted reference method, appropriate qualitycontrols should be applied to the accepted reference method.7. Procedure7.1 The objective of the validation procedure is to quantifythe performance of an empirically derived multivariate calibra-tion in terms of, in the case of quantitative calib
37、rations,precision, accuracy and bias or, in the case of qualitativecalibrations, Positive Fraction Identified and Negative FractionIdentified relative to an accepted reference method for eachproperty of interest. The user must specify, based on theintended use of the calibration, acceptable precisio
38、n and bias orPositive Fraction Identified and Negative Fraction IdentifiedE2617 102performance criteria before initiating the validation. Thesecriteria will be dependent on the intended use of the analyzerand may be based, all or in part, on risk based criteria.7.1.1 The acceptable performance crite
39、ria specified by theuser may be constant over the entire range of sample variabil-ity.Alternatively, different acceptable performance criteria maybe specified by the user for different sub-ranges of the fullsample variability.7.2 Validation of calibration is accomplished by using thecalibration to e
40、stimate the property(s) of a set of validationsamples and statistically comparing the estimates for thesesamples to known reference values. Validation requires thor-ough testing of the model with a sufficient number of repre-sentative validation samples to ensure that it performs ad-equately over th
41、e entire range of possible sample variability.7.3 Initial Validation Sample Set:7.3.1 For the initial validation of a multivariate model, anideal validation sample set will:7.3.1.1 Contain samples that provide sufficient examples ofall combinations of variation in the sample properties whichare expe
42、cted to be present in the samples which are to beanalyzed using the calibration;7.3.1.2 Contain samples for which the ranges of variation inthe sample properties is comparable to the ranges of variationexpected for samples that are to be analyzed using the model;7.3.1.3 Contain samples for which the
43、 respective variationsof the sample properties are uniformly and mutually indepen-dently distributed over their full respective ranges or, whenapplicable, subranges of variation; and7.3.1.4 Contain a sufficient number of samples to statisti-cally test the relationships between the measured variables
44、 andthe properties that are modeled by the calibration.7.3.2 For simple systems, sufficient validation samples cangenerally be obtained to meet the criteria in 7.3.1.1-7.3.1.4. Forcomplex mixtures, obtaining an ideal validation set may bedifficult if not impossible. In such cases, it may be necessar
45、y tovalidate discrete subranges of the calibration incrementally,over time as samples become available.7.3.3 The number of samples needed to validate a calibra-tion depends on the complexity of the calibration, the ranges ofproperty variation over which the calibration is to be applied,and the degre
46、e of confidence required. It is important tovalidate a calibration with as many samples as possible tomaximize the likelihood of challenging the calibration withrarely occurring, but potentially troublesome samples. Thenumber and range of validation samples should be sufficient tovalidate the calibr
47、ation to the statistical degree of confidencerequired for the application. In all cases, a minimum of 20validation samples is recommended. In addition, the validationsamples should:7.3.3.1 Multivariately span the ranges of sample propertyvalues over which the calibration will be used; that is, the s
48、panand the standard deviation of the ranges of sample propertyvalues for the validation samples should be at least 100 % ofthe spans of the sample property values over which thecalibration will be used, and the sample property values for thevalidation samples should be distributed as uniformly aspos
49、sible throughout their respective ranges, and the variationsof the sample property values among the samples should be asmutually independent as possible; and7.3.3.2 Span the ranges of the independent variables overwhich the calibration will be used; that is, if the range of anindependent variable is expected to vary from a to b, and thestandard deviation of the independent variable is c, then thevariations of that independent variable in the set of validationsamples should cover at least 100 % of the range from a to b,and should be distributed as uniformly as pos
