ASTM E2617-2008ae1 Standard Practice for Validation of Empirically Derived Multivariate Calibrations.pdf

1、Designation: E 2617 08a1Standard Practice forValidation of Empirically Derived Multivariate Calibrations1This standard is issued under the fixed designation E 2617; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last r

2、evision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1NOTEAnnex A1 was editorially corrected to be classified as Appendix X1 in February 2009.1. Scope1.1 This practice covers requirements f

3、or the validation ofempirically derived calibrations (Note 1) such as calibrationsderived by Multiple Linear Regression (MLR), Principal Com-ponent Regression (PCR), Partial Least Squares (PLS), Artifi-cial Neural Networks (ANN), or any other empirical calibra-tion technique whereby a relationship i

4、s postulated between aset of variables measured for a given sample under test and oneor more physical, chemical, quality, or membership propertiesapplicable to that sample.NOTE 1Empirically derived calibrations are sometimes referred to as“models” or “calibrations.” In the following text, for concis

5、eness, the term“calibration” may be used instead of the full name of the procedure.1.2 This practice does not cover procedures for establishingsaid postulated relationship.1.3 This practice serves as an overview of techniques usedto verify the applicability of an empirically derived multivari-ate ca

6、libration to the measurement of a sample under test andto verify equivalence between the properties calculated fromthe empirically derived multivariate calibration and the resultsof an accepted reference method of measurement to withincontrol limits established for the prespecified statistical confi

7、-dence level.1.4 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2.

8、 Referenced Documents2.1 ASTM Standards:2E 131 Terminology Relating to Molecular SpectroscopyE 1655 Practices for Infrared Multivariate QuantitativeAnalysisE 1790 Practice for Near Infrared Qualitative Analysis3. Terminology3.1 For terminology related to molecular spectroscopicmethods, refer to Term

9、inology E 131. For terminology relatedto multivariate quantitative modeling refer to Practices E 1655.While Practices E 1655 is written in the context of multivariatespectroscopic methods, the terminology is also applicable toother multivariate technologies.3.2 Definitions of Terms Specific to This

10、Standard:3.2.1 accuracythe closeness of agreement between a testresult and an accepted reference value.3.2.2 biasthe arithmetic average difference between thereference values and the values produced by the analyticalmethod under test, for a set of samples.3.2.3 detection limitthe lowest level of a p

11、roperty in asample that can be detected, but not necessarily quantified, bythe measurement system.3.2.4 estimatethe constituent concentration, identifica-tion, or other property of a sample as determined by theanalytical method being validated.3.2.5 initial validationvalidation that is performed whe

12、nan analyzer system is initially installed or after major mainte-nance.3.2.6 Negative Fraction Identifiedthe fraction of samplesnot having a particular characteristic that is identified as nothaving that characteristic.3.2.6.1 DiscussionNegative Fraction Identified assumesthat the characteristic tha

13、t the test measures either is or is notpresent. It is not applicable to tests with multiple possibleoutcomes.3.2.7 ongoing periodic revalidationthe quality assuranceprocess by which, in the case of quantitative calibrations, thebias and precision or, in the case of qualitative calibrations, thePosit

14、ive Fraction Identified and Negative Fraction Identifiedperformance determined during initial validation are shown tobe sustained.1This practice is under the jurisdiction of ASTM Committee E13 on MolecularSpectroscopy and Separation Science and is the direct responsibility of Subcom-mittee E13.11 on

15、 Multivariate Analysis.Current edition approved Oct. 1, 2008. Published October 2008. Originallyapproved in 2008. Last previous edition approved in 2008 as E 2617 08.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual B

16、ook of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.3.2.8 Positive Fraction Identifiedthe fraction of sampleshaving a particular cha

17、racteristic that is identified as havingthat characteristic.3.2.8.1 DiscussionPositive Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent. It is not applicable to tests with multiple possibleoutcomes.3.2.9 precisionthe closeness of agreement between

18、inde-pendent test results obtained under stipulated conditions.3.2.9.1 DiscussionPrecision may be a measure of eitherthe degree of reproducibility or degree of repeatability of theanalytical method under normal operating conditions. In thiscontext, reproducibility refers to the use of the analytical

19、procedure in different laboratories, as in a collaborative study.3.2.10 quantification limitthe lowest level of a sampleproperty which can be determined with acceptable precisionand accuracy under the stated experimental conditions.3.2.11 rangethe interval between the upper and lowerlevels of a prop

20、erty (including these levels) that has beendemonstrated to be determined with a suitable level of preci-sion and accuracy using the method as specified.3.2.12 reference valuethe metric of a property as deter-mined by well-characterized method, the accuracy of whichhas been stated or defined, that is

21、, another, already-validatedmethod.3.2.13 validationthe statistically quantified judgment thatan empirically derived multivariate calibration is applicable tothe measurement on which the calibration is to be applied andcan perform property estimates with, in the case of quantitativecalibrations, acc

22、eptable precision, accuracy and bias or, in thecase of qualitative calibrations, acceptable Positive FractionIdentified and Negative Fraction Identified, as compared withresults from an accepted reference method.4. Summary of Practice4.1 Validating an empirically derived multivariate calibra-tion (m

23、odel) consists of four major procedures: validation atinitial development, revalidation at initial deployment or aftera revision, ongoing periodic revalidation, and qualification ofeach measurement before using the calibration to estimate theproperty(s) of the sample being measured.5. Significance a

24、nd Use5.1 This practice outlines a universally applicable procedureto validate the performance of a quantitative or qualitative,empirically derived, multivariate calibration relative to anaccepted reference method.5.2 This practice provides procedures for evaluating thecapability of a calibration to

25、 provide reliable estimationsrelative to an accepted reference method.5.3 This practice provides purchasers of a measurementsystem that incorporates an empirically derived multivariatecalibration with options for specifying validation requirementsto ensure that the system is capable of providing est

26、imationswith an appropriate degree of agreement with an acceptedreference method.5.4 This practice provides the user of a measurement systemthat incorporates an empirically derived multivariate calibra-tion with procedures capable of providing information that maybe useful for ongoing quality assura

27、nce of the performance ofthe measurement system.5.5 Validation information obtained in the application ofthis practice is applicable only to the material type and propertyrange of the materials used to perform the validation and onlyfor the individual measurement system on which the practice iscompl

28、etely applied. It is the users responsibility to select theproperty levels and the compositional characteristics of thevalidation samples such that they are suitable to the applica-tion. This practice allows the user to write a comprehensivevalidation statement for the analyzer system including spec

29、ificlimits for the validated range of application and specificrestrictions to the permitted uses of the measurement system.Users are cautioned against extrapolation of validation resultsbeyond the material type(s) and property range(s) used toobtain these results.5.6 Users are cautioned that a valid

30、ated empirically derivedmultivariate calibration is applicable only to samples that fallwithin the subset population represented in the validation set.The estimation from an empirically derived multivariate cali-bration can only be validated when the applicability of thecalibration is explicitly est

31、ablished for the particular measure-ment for which the estimation is produced. Applicabilitycannot be assumed.6. Methods and Considerations6.1 When validating an empirically derived multivariatecalibration, it is the responsibility of the user to describe themeasurement system and the required level

32、 of agreementbetween the estimations produced by the calibration and theaccepted reference method(s).6.2 When validating a measurement system incorporatingan empirically derived multivariate calibration, it is the respon-sibility of the user to satisfy the requirements of any applicabletests specifi

33、c to the measurement system including any Instal-lation Qualification (IQ), Operational Qualification (OQ), andPerformance Qualification (PQ) requirements; which may bemandated by competent regulatory authorities, an applicableQuality Assurance (QA), or Standard Operating Procedure(SOP) or be recomm

34、ended by the instrument or equipmentmanufacturer.6.3 Reference Values and Quality Controls for the AcceptedReference Method:6.3.1 The reference (or true) value which is compared witheach respective estimate produced by the empirically derivedmultivariate calibration is established by applying an acc

35、eptedreference method, the characteristics of which are known andstated, to the sample from which the measurement systemderives the measurement.6.3.2 To ensure the reliability of the reference valuesprovided by an accepted reference method, appropriate qualitycontrols should be applied to the accept

36、ed reference method.7. Procedure7.1 The objective of the validation procedure is to quantifythe performance of an empirically derived multivariate calibra-tion in terms of, in the case of quantitative calibrations,precision, accuracy and bias or, in the case of qualitativecalibrations, Positive Frac

37、tion Identified and Negative FractionE 2617 08a12Identified relative to an accepted reference method for eachproperty of interest. The user must specify, based on theintended use of the calibration, acceptable precision and bias orPositive Fraction Identified and Negative Fraction Identifiedperforma

38、nce criteria before initiating the validation. Thesecriteria will be dependent on the intended use of the analyzerand may be based, all or in part, on risk based criteria.7.1.1 The acceptable performance criteria specified by theuser may be constant over the entire range of sample variabil-ity.Alter

39、natively, different acceptable performance criteria maybe specified by the user for different sub-ranges of the fullsample variability.7.2 Validation of calibration is accomplished by using thecalibration to estimate the property(s) of a set of validationsamples and statistically comparing the estim

40、ates for thesesamples to known reference values. Validation requires thor-ough testing of the model with a sufficient number of repre-sentative validation samples to ensure that it performs ad-equately over the entire range of possible sample variability.7.3 Initial Validation Sample Set:7.3.1 For t

41、he initial validation of a multivariate model, anideal validation sample set will:7.3.1.1 Contain samples that provide sufficient examples ofall combinations of variation in the sample properties whichare expected to be present in the samples which are to beanalyzed using the calibration;7.3.1.2 Con

42、tain samples for which the ranges of variation inthe sample properties is comparable to the ranges of variationexpected for samples that are to be analyzed using the model;7.3.1.3 Contain samples for which the respective variationsof the sample properties are uniformly and mutually indepen-dently di

43、stributed over their full respective ranges or, whenapplicable, subranges of variation; and7.3.1.4 Contain a sufficient number of samples to statisti-cally test the relationships between the measured variables andthe properties that are modeled by the calibration.7.3.2 For simple systems, sufficient

44、 validation samples cangenerally be obtained to meet the criteria in 7.3.1.1-7.3.1.4. Forcomplex mixtures, obtaining an ideal validation set may bedifficult if not impossible. In such cases, it may be necessary tovalidate discrete subranges of the calibration incrementally,over time as samples becom

45、e available.7.3.3 The number of samples needed to validate a calibra-tion depends on the complexity of the calibration, the ranges ofproperty variation over which the calibration is to be applied,and the degree of confidence required. It is important tovalidate a calibration with as many samples as

46、possible tomaximize the likelihood of challenging the calibration withrarely occurring, but potentially troublesome samples. Thenumber and range of validation samples should be sufficient tovalidate the calibration to the statistical degree of confidencerequired for the application. In all cases, a

47、minimum of 20validation samples is recommended. In addition, the validationsamples should:7.3.3.1 Multivariately span the ranges of sample propertyvalues over which the calibration will be used; that is, the spanand the standard deviation of the ranges of sample propertyvalues for the validation sam

48、ples should be at least 100 % ofthe spans of the sample property values over which thecalibration will be used, and the sample property values for thevalidation samples should be distributed as uniformly aspossible throughout their respective ranges, and the variationsof the sample property values a

49、mong the samples should be asmutually independent as possible; and7.3.3.2 Span the ranges of the independent variables overwhich the calibration will be used; that is, if the range of anindependent variable is expected to vary from a to b, and thestandard deviation of the independent variable is c, then thevariations of that independent variable in the set of validationsamples should cover at least 100 % of the range from a to b,and should be distributed as uniformly as possible across therange such that the standard deviation in that independentvariable estima