1、Designation: E3023 15Standard Practice forProbability of Detection Analysis for Versus a Data1This standard is issued under the fixed designation E3023; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A n
2、umber in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice defines the procedure for performing astatistical analysis on Nondestructive Testing (NDT) versus adata to determine the d
3、emonstrated probability of detection(POD) for a specific set of examination parameters. Topicscovered include the standard versus a regressionmethodology, POD curve formulation, validation techniques,and correct interpretation of results.1.2 The values stated in inch-pound units are to be regardedas
4、 standard. The values given in parentheses are mathematicalconversions to SI units that are provided for information onlyand are not considered standard.1.3 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of th
5、is standard to establish appro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2E178 Practice for Dealing With Outlying ObservationsE456 Terminology Relating to Quality and StatisticsE1316 Terminolog
6、y for Nondestructive ExaminationsE1325 Terminology Relating to Design of ExperimentsE2586 Practice for Calculating and Using Basic StatisticsE2782 Guide for Measurement Systems Analysis (MSA)E2862 Practice for Probability of Detection Analysis forHit/Miss Data2.2 Department of Defense Document:3MIL-
7、HDBK-1823A Nondestructive Evaluation System Re-liability Assessment3. Terminology3.1 Definitions of Terms Specific to This Standard:3.1.1 analyst, nthe person responsible for performing aPOD analysis on versus a data resulting from a PODexamination.3.1.2 decision threshold, dec,nthe value of abovewh
8、ich the signal is interpreted as a find and below which thesignal is interpreted as a miss.3.1.2.1 DiscussionA decision threshold is required tocreate a POD curve. The decision threshold is always greaterthan or equal to the noise threshold and is the value of thatcorresponds with the flaw size that
9、 can be detected with 50%POD.3.1.3 demonstrated probability of detection, nthe calcu-lated POD value resulting from the statistical analysis of the versus a data.3.1.4 false call, n the perceived detection of a disconti-nuity that is identified as a find during a POD examinationwhen no discontinuity
10、 actually exists at the inspection site.3.1.5 noise, nsignal response containing no useful targetcharacterization information.3.1.6 noise threshold, noise,nthe value of below whichthe signal is indistinguishable from noise.3.1.6.1 DiscussionThe noise threshold is always less thanor equal to the deci
11、sion threshold. The noise threshold is usedto determine left censored data.3.1.7 probability of detection, nthe fraction of nominaldiscontinuity sizes expected to be found given their existence.3.1.8 saturation threshold, sat,nthe value of associatedwith the maximum output of the system or the large
12、st value of that the system can record.3.1.8.1 DiscussionThe saturation threshold is used todetermine right censored data.3.2 Symbols:3.2.1 adiscontinuity size.3.2.2 the measured signal response for a given disconti-nuity size, a.3.2.2.1 DiscussionThe measured signal response is as-sumed to be conti
13、nuous in nature. Units depend on the NDT1This test method is under the jurisdiction of ASTM Committee E07 onNondestructive Testing and is the direct responsibility of Subcommittee E07.10 onSpecialized NDT Methods.Current edition approved June 15, 2015. Published August 2015. DOI: 10.1520/E302315.2Fo
14、r referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.3Available from Standardization Documents Order Desk, DODSSP, Bl
15、dg. 4,Section D, 700 Robbins Ave., Philadelphia, PA 19111-5098, http:/dodssp.daps.dla.mil.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States1inspection system and can be, for example, scale divisions,number of contiguous illuminated pixe
16、ls, or millivolts.3.2.3 apthe discontinuity size that can be detected withprobability p.3.2.3.1 DiscussionEach discontinuity size has an indepen-dent probability of being detected and corresponding probabil-ity of being missed. For example, being able to detect a specificdiscontinuity size with prob
17、ability p does not guarantee that alarger size discontinuity will be found.3.2.4 ap/cthe discontinuity size that can be detected withprobability p with a statistical confidence level of c.3.2.4.1 DiscussionAccording to the formula in MIL-HDBK-1823A, ap/cis a one-sided upper confidence bound onap. ap
18、/crepresents how large the true apcould be given thestatistical uncertainty associated with limited sample data.Hence ap/c ap. Note that POD is equal to p for both ap/candap. apis based solely on the observed relationship between the and a data and represents a snapshot in time, whereas ap/caccounts
19、 for the uncertainty associated with limited sampledata.4. Summary of Practice4.1 This practice describes, step-by-step, the process foranalyzing nondestructive testing versus a data resulting froma POD examination, including minimum requirements forvalidating the resulting POD curve.4.2 This practi
20、ce also includes definitions and discussionsfor results of interest (e.g., a90/95) to provide for correctinterpretation of results.4.3 Definitions of statistical terminology used in the body ofthis practice can be found in Annex A1.5. Significance and Use5.1 The POD analysis method described herein
21、is based onwell-known and well-established statistical methods. It shall beused to quantify the demonstrated POD for a specific set ofexamination parameters and known range of discontinuitysizes under the following conditions.5.1.1 The initial response from a nondestructive evaluationinspection syst
22、em is measurable and can be classified as acontinuous variable.5.1.2 The relationship between discontinuity size (a) andmeasured signal response () exists and is best described by alinear regression model with an error structure that is normallydistributed with mean zero and constant variance, 2. (N
23、otethat “linear” refers to linear with respect to the model coeffi-cients. For example, a quadratic model y 5011x12x2is alinear model.)5.2 This practice does not limit the use of other statisticalmodels if justified as more appropriate for the versus a data.5.3 This practice is not appropriate for d
24、ata resulting from aPOD examination on nondestructive evaluation systems thatgenerate an initial response that is binary in nature (forexample, hit/miss). Practice E2862 is appropriate for systemsthat generate a hit/miss-type response (for example, fluorescentpenetrant).5.4 Prior to performing the a
25、nalysis it is assumed that thediscontinuity of interest is clearly defined; the number anddistribution of induced discontinuity sizes in the POD speci-men set is known and well documented; the POD examinationadministration procedure (including data collection method) iswell designed, well defined, u
26、nder control, and unbiased; theinitial inspection system response is measurable and continu-ous in nature; the inspection system is calibrated; and themeasurement error has been evaluated and deemed acceptable.The analysis results are only valid if the versus a data areaccurate and precise and the l
27、inear model adequately representsthe versus a data.5.5 The POD analysis method described herein is consistentwith the analysis method for versus a data described inMIL-HDBK-1823A and is included in several widely utilizedPOD software packages to perform a POD analysis on versusa data. It is also fou
28、nd in statistical software packages that havelinear regression capability. This practice requires that theanalyst has access to either POD software or other softwarewith linear regression capability.6. Procedure6.1 The POD analysis objective shall be clearly defined bythe responsible engineer or by
29、the customer.6.2 The analyst shall obtain the versus a data resultingfrom the POD examination, which shall include at a minimumthe documented known induced discontinuity sizes, the asso-ciated measured signal response, and any false calls.6.3 The analyst shall also obtain specific information aboutt
30、he POD examination, which shall include at a minimum thespecimen standard geometry (e.g., flat panels), specimen stan-dard material (e.g., Nickel), examination date, number ofinspectors, type of inspection method (e.g., Eddy CurrentInspection), pertinent information about the instrument andinstructi
31、ons for use (e.g., settings, probe type, scan path), andpertinent comments from the inspector(s) and test administra-tor.6.3.1 In general, the results of an experiment apply to theconditions under which the experiment was conducted. Hence,the POD analysis results apply to the conditions under whicht
32、he POD examination was conducted.6.4 Prior to performing the analysis, the analyst shallconduct a preliminary review of the POD examination proce-dure to identify any issues with the administration of theexamination. The analyst shall identify and attempt to resolveany issues prior to conducting the
33、 POD analysis. Identifiedissues and their resolution shall be documented in the report.Examples of examination administration issues and possibleresolutions are outlined in the following subsections.6.4.1 If problems or interruptions occurred during the PODexamination that may bias the results, the
34、POD examinationshould be re-administered.6.4.2 If the examination procedure was poorly designedand/or executed, the validity of the resulting data is question-able. In this case, the examination procedure design andexecution should be reevaluated. For design guidelines seeMIL-HDBK-1823A.E3023 1526.5
35、 Prior to performing the analysis, the analyst shallconduct a preliminary review of the versus a data to identifyany data issues. The analyst shall identify and attempt toresolve any issues prior to conducting the POD analysis.Identified issues and their resolution shall be documented in thereport.
36、Examples of data issues and possible resolutions areoutlined in the following subsections.6.5.1 Any apparent outlying observations shall be reviewedfor correctness. If a typo is identified, the typo shall becorrected prior to performing the analysis. If the value iscorrect, it shall be retained in t
37、he analysis and its influence onthe versus a model shall be evaluated during the modeldiagnostic assessment. The analyst should also reference Prac-tice E178.6.5.2 POD cannot be modeled as a continuous function ofdiscontinuity size if all the measured signal responses arebelow the noise threshold or
38、 above the saturation threshold. Ifthis occurs, the adequacy of the nondestructive testing systemshould be evaluated.6.6 Only versus a data for induced discontinuities shall beused in the development of the linear regression model. Falsecall data shall not be included in the development of the linea
39、rmodel when using standard linear regression methods.6.7 The analyst in conjunction with the responsible engineershall determine the value of the noise threshold, noise, andsaturation threshold, sat, prior to performing the analysis.6.7.1 The value of noiseis determined by performing anoise analysis
40、. A noise analysis is typically accomplished byassessing the distribution of measured signal responses fromsites with no known discontinuity (false calls) and/or measuredsignal responses that are not influenced by the size of thediscontinuities (noise). Details on performing a noise analysiscan be f
41、ound in MIL-HDBK-1823A.6.8 The analyst shall select an appropriate linear regressionmodel to establish the relationship between and a. Selectionof a linear model may be an iterative process as the significanceof the predictor variable(s) and the appropriateness of theselected model are typically ass
42、essed after the model has beendeveloped.6.8.1 “Linear” refers to linear with respect to the modelcoefficients. For example, yi5b01b1x2! and yi5b01b1x11b2lnx2! are linear regression models.6.8.2 In general, only significant and uncorrelated predictorvariables are included in a regression model. If mo
43、re than onepredictor variable is being considered for inclusion in themodel, a preliminary graphical analysis of the response vari-able against each predictor variable may help identify whichpredictor variables appear to influence the response and thetype of relationship (for example, direct, invers
44、e, quadratic). Inaddition, a preliminary graphical analysis of all possiblepairings of predictor variables shall be performed to verifyindependence of the predictor variables. When plotted againsteach other, there should be no apparent relationship betweenany two predictor variables.6.8.3 The approp
45、riateness of a selected model is determinedby how well the model fits the observed data and how well theunderlying regression assumptions are met.6.9 The analyst shall use software that has the appropriatelinear regression capabilities to perform a linear regressionanalysis on the versus a data.6.9.
46、1 If censored data are present, the analyst shall do thefollowing:6.9.1.1 Include and identify the censored data in the analysis(according to the notation required by the software).6.9.1.2 Use the method of maximum likelihood to estimatethe model coefficients.6.9.1.3 Verify that convergence was achi
47、eved. If conver-gence is not achieved, the resulting versus a model shall notbe used to develop a POD curve.6.9.1.4 Check the number of iterations it took to convergeprovided that information on convergence and the number ofiterations it took to converge is included in the analysissoftware output. I
48、f more than twenty iterations were needed toreach convergence, the model may not be reliable.6.9.1.5 Include a statement in the report indicating thatconvergence was achieved and the number of iterations neededto achieve convergence.6.9.2 If no censored data are present, the method of maxi-mum likel
49、ihood or the method of least squares shall be used.6.10 If included in the analysis software output, the analystshall assess the significance of the predictor variables in themodel. Only significant predictor variables should be includedin the model.6.11 Once the versus a model is estimated, the analystshall use, at a minimum, the model diagnostic methods listedbelow to assess the underlying linear regression assumptions.The methods listed below shall be performed using onlynon-censored data. If
