1、Designation: E2617 17Standard Practice forValidation of Empirically Derived Multivariate Calibrations1This standard is issued under the fixed designation E2617; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revis
2、ion. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice covers requirements for the validation ofempirically derived calibrations (Note 1) such as calibrationsderived by
3、Multiple Linear Regression (MLR), Principal Com-ponent Regression (PCR), Partial Least Squares (PLS), Artifi-cial Neural Networks (ANN), or any other empirical calibra-tion technique whereby a relationship is postulated between aset of variables measured for a given sample under test and oneor more
4、physical, chemical, quality, or membership propertiesapplicable to that sample.NOTE 1Empirically derived calibrations are sometimes referred to as“models” or “calibrations.” In the following text, for conciseness, the term“calibration” may be used instead of the full name of the procedure.1.2 This p
5、ractice does not cover procedures for establishingsaid postulated relationship.1.3 This practice serves as an overview of techniques usedto verify the applicability of an empirically derived multivari-ate calibration to the measurement of a sample under test andto verify equivalence between the prop
6、erties calculated fromthe empirically derived multivariate calibration and the resultsof an accepted reference method of measurement to withincontrol limits established for the prespecified statistical confi-dence level.1.4 This standard does not purport to address all of thesafety concerns, if any,
7、 associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety, health, and environmental practices and deter-mine the applicability of regulatory limitations prior to use.1.5 This international standard was developed in accor-dance with international
8、ly recognized principles on standard-ization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recom-mendations issued by the World Trade Organization TechnicalBarriers to Trade (TBT) Committee.2. Referenced Documents2.1 ASTM Standards:2E131 Terminol
9、ogy Relating to Molecular SpectroscopyE1655 Practices for Infrared Multivariate QuantitativeAnalysisE1790 Practice for Near Infrared Qualitative Analysis3. Terminology3.1 For terminology related to molecular spectroscopicmethods, refer to Terminology E131. For terminology relatedto multivariate quan
10、titative modeling refer to Practices E1655.While Practices E1655 is written in the context of multivariatespectroscopic methods, the terminology is also applicable toother multivariate technologies.3.2 Definitions of Terms Specific to This Standard:3.2.1 accuracythe closeness of agreement between a
11、testresult and an accepted reference value.3.2.2 biasthe arithmetic average difference between thereference values and the values produced by the analyticalmethod under test, for a set of samples.3.2.3 detection limitthe lowest level of a property in asample that can be detected, but not necessarily
12、 quantified, bythe measurement system.3.2.4 estimatethe constituent concentration, identification,or other property of a sample as determined by the analyticalmethod being validated.3.2.5 initial validationvalidation that is performed whenan analyzer system is initially installed or after major main
13、te-nance.3.2.6 Negative Fraction Identifiedthe fraction of samplesnot having a particular characteristic that is identified as nothaving that characteristic.1This practice is under the jurisdiction of ASTM Committee E13 on MolecularSpectroscopy and Separation Science and is the direct responsibility
14、 of Subcom-mittee E13.11 on Multivariate Analysis.Current edition approved Dec. 15, 2017. Published February 2018. Originallyapproved in 2008. Last previous edition approved in 2010 as E2617 10. DOI:10.1520/E2617-17.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM
15、 Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United StatesThis international standard was
16、developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.13.2.6.1 Discus
17、sionNegative Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent. It is not applicable to tests with multiple possibleoutcomes.3.2.7 ongoing periodic revalidationthe quality assuranceprocess by which, in the case of quantitative calibrations, thebias
18、and precision or, in the case of qualitative calibrations, thePositive Fraction Identified and Negative Fraction Identifiedperformance determined during initial validation are shown tobe sustained.3.2.8 Positive Fraction Identifiedthe fraction of sampleshaving a particular characteristic that is ide
19、ntified as havingthat characteristic.3.2.8.1 DiscussionPositive Fraction Identified assumesthat the characteristic that the test measures either is or is notpresent. It is not applicable to tests with multiple possibleoutcomes.3.2.9 precisionthe closeness of agreement between inde-pendent test resul
20、ts obtained under stipulated conditions.3.2.9.1 DiscussionPrecision may be a measure of eitherthe degree of reproducibility or degree of repeatability of theanalytical method under normal operating conditions. In thiscontext, reproducibility refers to the use of the analyticalprocedure in different
21、laboratories, as in a collaborative study.3.2.10 quantification limitthe lowest level of a sampleproperty which can be determined with acceptable precisionand accuracy under the stated experimental conditions.3.2.11 rangethe interval between the upper and lowerlevels of a property (including these l
22、evels) that has beendemonstrated to be determined with a suitable level of preci-sion and accuracy using the method as specified.3.2.12 reference valuethe metric of a property as deter-mined by well-characterized method, the accuracy of whichhas been stated or defined, that is, another, already-vali
23、datedmethod.3.2.13 validationthe statistically quantified judgment thatan empirically derived multivariate calibration is applicable tothe measurement on which the calibration is to be applied andcan perform property estimates with, in the case of quantitativecalibrations, acceptable precision, accu
24、racy and bias or, in thecase of qualitative calibrations, acceptable Positive FractionIdentified and Negative Fraction Identified, as compared withresults from an accepted reference method.3.2.14 validation spacethe region(s) of a calibrationsmultivariate sample space populated by the independent va
25、li-dation samples which are used to validate the calibration.4. Summary of Practice4.1 Validating an empirically derived multivariate calibra-tion (model) consists of four major procedures: validation atinitial development, revalidation at initial deployment or aftera revision, ongoing periodic reva
26、lidation, and qualification ofeach measurement before using the calibration to estimate theproperty(s) of the sample being measured.5. Significance and Use5.1 This practice outlines a universally applicable procedureto validate the performance of a quantitative or qualitative,empirically derived, mu
27、ltivariate calibration relative to anaccepted reference method.5.2 This practice provides procedures for evaluating thecapability of a calibration to provide reliable estimationsrelative to an accepted reference method.5.3 This practice provides purchasers of a measurementsystem that incorporates an
28、 empirically derived multivariatecalibration with options for specifying validation requirementsto ensure that the system is capable of providing estimationswith an appropriate degree of agreement with an acceptedreference method.5.4 This practice provides the user of a measurement systemthat incorp
29、orates an empirically derived multivariate calibra-tion with procedures capable of providing information that maybe useful for ongoing quality assurance of the performance ofthe measurement system.5.5 Validation information obtained in the application ofthis practice is applicable only to the materi
30、al type and propertyrange of the materials used to perform the validation and onlyfor the individual measurement system on which the practice iscompletely applied. It is the users responsibility to select theproperty levels and the compositional characteristics of thevalidation samples such that the
31、y are suitable to the applica-tion. This practice allows the user to write a comprehensivevalidation statement for the analyzer system including specificlimits for the validated range of application and specificrestrictions to the permitted uses of the measurement system.Users are cautioned against
32、extrapolation of validation resultsbeyond the material type(s) and property range(s) used toobtain these results.5.6 Users are cautioned that a validated empirically derivedmultivariate calibration is applicable only to samples that fallwithin the subset population represented in the validation set.
33、The estimation from an empirically derived multivariate cali-bration can only be validated when the applicability of thecalibration is explicitly established for the particular measure-ment for which the estimation is produced. Applicabilitycannot be assumed.6. Methods and Considerations6.1 When val
34、idating an empirically derived multivariatecalibration, it is the responsibility of the user to describe themeasurement system and the required level of agreementbetween the estimations produced by the calibration and theaccepted reference method(s).6.2 When validating a measurement system incorpora
35、tingan empirically derived multivariate calibration, it is the respon-sibility of the user to satisfy the requirements of any applicabletests specific to the measurement system including any Instal-lation Qualification (IQ), Operational Qualification (OQ), andPerformance Qualification (PQ) requireme
36、nts; which may bemandated by competent regulatory authorities, an applicableE2617 172Quality Assurance (QA), or Standard Operating Procedure(SOP) or be recommended by the instrument or equipmentmanufacturer.6.3 Reference Values and Quality Controls for the AcceptedReference Method:6.3.1 The referenc
37、e (or true) value which is compared witheach respective estimate produced by the empirically derivedmultivariate calibration is established by applying an acceptedreference method, the characteristics of which are known andstated, to the sample from which the measurement systemderives the measuremen
38、t.6.3.2 To ensure the reliability of the reference valuesprovided by an accepted reference method, appropriate qualitycontrols should be applied to the accepted reference method.7. Procedure7.1 The objective of the validation procedure is to quantifythe performance of an empirically derived multivar
39、iate calibra-tion in terms of, in the case of quantitative calibrations,precision, accuracy and bias or, in the case of qualitativecalibrations, Positive Fraction Identified and Negative FractionIdentified relative to an accepted reference method for eachproperty of interest. The user must specify,
40、based on theintended use of the calibration, acceptable precision and bias orPositive Fraction Identified and Negative Fraction Identifiedperformance criteria before initiating the validation. Thesecriteria will be dependent on the intended use of the analyzerand may be based, all or in part, on ris
41、k based criteria.7.1.1 The acceptable performance criteria specified by theuser may be constant over the entire range of sample variabil-ity.Alternatively, different acceptable performance criteria maybe specified by the user for different sub-ranges of the fullsample variability.7.2 Validation of c
42、alibration is accomplished by using thecalibration to estimate the property(s) of a set of validationsamples and statistically comparing the estimates for thesesamples to known reference values. Validation requires thor-ough testing of the model with a sufficient number of repre-sentative validation
43、 samples to ensure that it performs ad-equately over the entire range of possible sample variability.7.3 Initial Validation Sample Set:7.3.1 For the initial validation of a multivariate model, anideal validation sample set will:7.3.1.1 Contain samples that provide sufficient examples ofall combinati
44、ons of variation in the sample properties whichare expected to be present in the samples which are to beanalyzed using the calibration;7.3.1.2 Contain samples for which the ranges of variation inthe sample properties is comparable to the ranges of variationexpected for samples that are to be analyze
45、d using the model;7.3.1.3 Contain samples for which the respective variationsof the sample properties are uniformly and mutually indepen-dently distributed over their full respective ranges or, whenapplicable, subranges of variation; and7.3.1.4 Contain a sufficient number of samples to statisti-call
46、y test the relationships between the measured variables andthe properties that are modeled by the calibration.7.3.2 For simple systems, sufficient validation samples cangenerally be obtained to meet the criteria in 7.3.1.1 7.3.1.4.For complex mixtures, obtaining an ideal validation set may bedifficu
47、lt if not impossible. In such cases, it may be necessary tovalidate discrete subranges of the calibration incrementally,over time as samples become available.7.3.3 The number of samples needed to validate a calibra-tion depends on the complexity of the calibration, the ranges ofproperty variation ov
48、er which the calibration is to be applied,and the degree of confidence required. It is important tovalidate a calibration with as many samples as possible tomaximize the likelihood of challenging the calibration withrarely occurring, but potentially troublesome samples. Thenumber and range of valida
49、tion samples should be sufficient tovalidate the calibration to the statistical degree of confidencerequired for the application. In all cases, a minimum of 20validation samples is recommended. In addition, the validationsamples should:7.3.3.1 Multivariately span the ranges of sample propertyvalues over which the calibration will be used; that is, the spanand the standard deviation of the ranges of sample propertyvalues for the validation samples should be at least 100 % ofthe spans of the sample property values over which thecalibration will be used, and the s
