1、Designation: D6122 15D6122 18Standard Practice forValidation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer BasedAnalyzer Systems1This standard is issued under the fixed designation D6122; the number immediately following the designation indicates the y
2、ear oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1 This practice is under the jurisdiction of ASTM Committee D02 o
3、n Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of SubcommitteeD02.25 on Performance Assessment and Validation of Process Stream Analyzer Systems.Current edition approved June 1, 2015July 1, 2018. Published February 2016January 2019. Originally approved in 1997. L
4、ast previous edition approved in 20102015 asD6122 13.D6122 15. DOI: 10.1520/D6122-15.10.1520/D6122-18.This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Becauseit may not be technica
5、lly possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current versionof the standard as published by ASTM is to be considered the official document.Copyright ASTM International, 100 Barr Harbor Drive, PO Box
6、C700, West Conshohocken, PA 19428-2959. United States1INTRODUCTIONOperation of a laboratory or process stream analyzer system typically involves fourfive sequentialactivities. (1) CorrelationPrior to the initiation of the procedures described in this practice, amultivariate model is derived which re
7、lates the spectrum produced by the analyzer to the Primary TestMethod Result (PTMR). (1a) If the analyzer and Primary Test Method (PTM) measure the samematerial, then the multivariate model directly relates the spectra to PTMR collected on the samesamples. Alternatively (1b) if the analyzer measures
8、 the spectra of a material that is subjected totreatment prior to being measured by the PTM, then the multivariate model relates the spectra ofthe untreated sample to the PTMR for the same sample after treatment. (2) AnalyzerCalibrationQualificationWhen an analyzer is initially installed, or after m
9、ajor maintenance hasbeen performed, or after the multivariate model has been changed, diagnostic testing is performed todemonstrate that the analyzer meets the manufacturers specifications and historical performancestandards. These diagnostic tests may require that the analyzer be adjusted so as to
10、providepredetermined output levels for certain reference materials.materials (2a)Correlation, where analyzerand Primary Test Method (PTM) measure the same materialOnce the diagnostic testing iscompleted, process stream samples are analyzed using both the analyzer system and the correspondingPTM. A m
11、athematical function is derived that relates the analyzer output to the PTM. The applicationof this mathematical function to an analyzer output produces a Predicted Primary Test Method Result(PPTMR) for the same material. (2b)Correlation, where analyzer measures a material which issubjected to treat
12、ment before being measured by the PTMOnce the diagnostic testing is completed,the process stream samples are analyzed by the analyzer system. The same samples are subjected toa consistent treatment, and the treated samples are analyzed by the PTM. A mathematical function isderived that related the a
13、nalyzer output for the untreated sample to the Primary Test Method Result(PTMR) for the treated material. The application of the mathematical function to the analyzer outputfor the untreated material produces a PPTMR for the treated material. (3) ProbationaryLocalValidationOnce the relationship betw
14、een the analyzer output and PTMRs has been established, aprobationary A local validation is performed using an independent but limited set of materials thatwere not part of the correlation activity. This probationarylocal validation is intended to demonstratethat the PPTMRs agree with the PTMRs to w
15、ithin user-specified requirements for the analyzer systemapplication. agreement between the Predicted Primary Method Test Results (PPTMRs) and thePTMRs are consistent with expectations based on the multivariate model. (4) General and ContinualValidationAfter an adequate number of PPTMRs and PTMRs ha
16、ve been accrued on materials thatwere not part of the correlation activity, activity and which adequately span the multivariate modelcompositional space, a comprehensive statistical assessment is can be performed to demonstrate thatthe PPTMRs agree with the PTMRs to within user-specified requirement
17、s. Subsequent(5) ContinualValidationSubsequent to a successful local or general validation, quality assurance control chartmonitoring of the differences between PPTMR and PTMR is conducted during normal operation ofthe process analyzer system to demonstrate that the agreement between the PPTMRs and
18、the PTMRsestablished during the General Validation is maintained. This practice deals with the third third, fourth,and fourthfifth of these activities.“Correlation where analyzer measures a material which is subjected to treatment before beingmeasured by the PTM” as outlined in this practice is inte
19、nded primarily to can be applied to biofuelswhere the biofuel material is added at a terminal or other facility and not included in the processstream material sampled by the analyzer at the basestock manufacturing facility. The “treatment” shallbe a constant percentage addition of the biofuels mater
20、ial to the basestock material. The correlationis deemed valid only for the specific percentage addition and type of biofuel material used in itsdevelopment.1. Scope*1.1 This practice covers requirements for the validation of measurements made by laboratory or process (online or at-line) near-or mid-
21、infrared analyzers, or both, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquidpetroleum products and fuels. The properties are calculated from spectroscopic data using multivariate modeling methods. Therequirements include verification of adequate i
22、nstrument performance, verification of the applicability of the calibration model tothe spectrum of the sample under test, and verification that the degree of agreement between the results calculated from the infraredmeasurements and the results produced by the PTM used for the development of the ca
23、libration model meets user-specifiedrequirements. Initially, a limited number of validation samples representative of current production are used to do a localvalidation. When there is adequate variation in property level, the an adequate number of validation samples with sufficientvariation in both
24、 property level and sample composition to span the model calibration space, the statistical methodology of PracticeD6122 182D6708 is can be used to provide general validation of this equivalence over the complete operating range of the analyzer. For caseswhere there is inadequate property variation,
25、 methodology for level specific validation is adequate property and compositionvariation is not achieved, local validation continues to be used.1.1.1 For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate modelto the analyzer output (spectru
26、m) directly produces a PPTMR for the same material for which the spectrum was measured. ThePPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.1.1.2 For other applications, the material measured by the analyzer system is subjected to a consistent trea
27、tment prior to beinganalyzed by the PTM. The application of the multivariate model to the analyzer output (spectrum) produces a PPTMR for thetreated material. The PPTMRs based on the analyzer outputs are compared to the PTMRs measured on the treated materials todetermine the degree of agreement.1.2
28、Multiple physical, chemical, or quality properties of the sample under test are typically predicted from a single spectralmeasurement. In applying this practice, each property prediction is validated separately. The separate validation procedures foreach property may share common features, and be af
29、fected by common effects, but the performance of each property predictionis evaluated independently. The user will typically have multiple validation procedures running simultaneously in parallel.1.3 Results used in analyzer validation are for samples that were not used in the development of the mul
30、tivariate model, andfor spectra which are not outliers or nearest neighbor inliers relative to the multivariate model.1.4 Performance Validation is conducted by calculating the precision and bias of the differences between results from theanalyzer system (or subsystem) produced by application of the
31、 multivariate model, (such results are herein referred to as PPTMRs),versus the PTMRs for the same sample set. Results used in the calculation are for samples that are not used in the developmentof the multivariate model. The calculated precision and bias are statistically compared to user-specified
32、 requirements for theanalyzer system application.When the number, composition range or property range of available validation samples do not spanthe model calibration range, a local validation is done using available samples representative of current production. When thenumber, composition range and
33、 property range of available validation samples becomes comparable to those of the modelcalibration set, a general validation can be done.1.4.1 Local Validation:1.4.1.1 The calibration samples used in developing the multivariate model must show adequate compositional and propertyvariation to enable
34、the development of a meaningful correlation, and must span the compositional range of samples to be analyzedusing the model to ensure that such analyses are done via interpolation rather than extrapolation. The Standard Error of Calibration(SEC) is a measure of how well the PTMRs and PPTMRs agree fo
35、r this set of calibration samples. SEC includes contributionsfrom spectrum measurement error, PTM measurement error, and model error. Sample (type) specific biases are a part of the modelerror. Typically, spectroscopic analyzers are very precise, so that spectral measurement error is small relative
36、to the other types oferror.1.4.1.2 During initial analyzer validation, the compositional range of available samples may be small relative to the range ofthe calibration set. Because of the high precision of the spectroscopic measurement, the average difference between the PTMRsand PPTMRs may reflect
37、 a sample (type) specific bias which is statistically observable, but which are less than the 95 %uncertainty of PPTMR, U(PPTMR). Therefore, the bias and precision of the PTMR/PPTMR differences are not used as the basisfor local validation.1.4.1.3 Based on SEC, and the leverage statistic, a 95 % unc
38、ertainty for each PPTMR, U(PPTMR) is calculated. Duringvalidation, for each non-outlier sample, a determination is made as to whether the absolute difference between PPTMR and PTMR,|, is less than or equal to U(PPTMR). Counts are maintained as to the total number of non-outlier validation samples, a
39、nd thenumber of samples for which | is less than or equal to U(PPTMR). Given the total number of non-outlier validation samples,an inverse binomial distribution is used to calculate the minimum number of results for which | must be less than U(PPTMR).If the number of results for which | is less than
40、 U(PPTMR) is greater than or equal to this minimum, then the results are consistentwith the expectations of the multivariate model, and the analyzer passes local validation. The calculations involved are describedin detail in Section 11 and Annex A4.1.4.1.4 The user must establish that results that
41、are consistent with the expectations based on the multivariate model will beadequate for the intended application. A 95 % probability is recommended for the inverse binomial distribution calculation. Theuser may adjust this based on the criticality of the application. See Annex A4 for details.1.4.2
42、General Validation:1.4.2.1 When the validation samples are of sufficient number, and their compositional and property ranges are comparable tothat of the model calibration set, then a General Validation can be done.1.4.2.2 General Validation is conducted by doing a D6708 based assessment of the diff
43、erences between results from the analyzersystem (or subsystem) produced by application of the multivariate model, (such results are herein referred to as PPTMRs), versusthe PTMRs for the same sample set. The calculated precision and bias are statistically compared to user- specified requirementsfor
44、the analyzer system application.1.4.3 For analyzers used in product release or product quality certification applications, the precision and bias requirement forthe degree of agreement are typically based on the site or published precision of the PTM.D6122 183NOTE 1In most applications of this type,
45、 the PTM is the specification-cited test method.1.4.4 This practice does not describe procedures for establishing precision and bias requirements for analyzer systemapplications. Such requirements must be based on the criticality of the results to the intended business application and oncontractual
46、and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validationprocedures described herein.NOTE 1In most applications of this type, the PTM is the specification-cited test method.1.5 This practice does not cover procedures for establishing the
47、calibration model (correlation) used by the analyzer. Calibrationprocedures are covered in Practices E1655 and references therein.1.6 This practice is intended as a review for experienced persons. For novices, this practice will serve as an overview oftechniques used to verify instrument performance
48、, to verify model applicability to the spectrum of the sample under test, and toverify equivalence between the parameters calculated from the infrared measurement and the results of the primary test methodmeasurement.1.7 This practice teaches and recommends appropriate statistical tools, outlier det
49、ection methods, for determining whether thespectrum of the sample under test is a member of the population of spectra used for the analyzer calibration. The statistical toolsare used to determine if the infrared measurement results in a valid property or parameter estimate.1.8 The outlier detection methods do not define criteria to determine whether the sample or the instrument is the cause of anoutlier measurement. Thus, the operator who is measuring samples on a routine basis will find criteria to determine that a spectralmeasurement lies outside the calibra
