1、Designation: E2862 12Standard Practice forProbability of Detection Analysis for Hit/Miss Data1This standard is issued under the fixed designation E2862; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A n
2、umber in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This practice defines the procedure for performing astatistical analysis on nondestructive testing hit/miss data todetermine the demonst
3、rated probability of detection (POD) fora specific set of examination parameters. Topics coveredinclude the standard hit/miss POD curve formulation, valida-tion techniques, and correct interpretation of results.1.2 The values stated in inch-pound units are to be regardedas standard. The values given
4、 in parentheses are mathematicalconversions to SI units that are provided for information onlyand are not considered standard.1.3 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish ap
5、pro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2E1316 Terminology for Nondestructive Examinations2.2 Department of Defense Handbook:MIL-HDBK-1823A Nondestructive Evaluation System Re-liability A
6、ssessment33. Terminology3.1 Definitions of Terms Specific to This Standard:3.1.1 analyst, nthe person responsible for performing aPOD analysis on hit/miss data resulting from a POD examina-tion.3.1.2 demonstrated probability of detection, nthe calcu-lated POD value resulting from the statistical ana
7、lysis on thehit miss data.3.1.3 false call, nthe perceived detection of a discontinu-ity that is identified as a find during a POD examination whenno discontinuity actually exists at the inspection site.3.1.4 hit, nan existing discontinuity that is identified as afind during a POD demonstration exam
8、ination.3.1.5 miss, nan existing discontinuity that is missedduring a POD examination.3.1.6 probability of detection, nthe fraction of nominaldiscontinuity sizes expected to be found given their existence.3.2 Symbols:3.2.1 adiscontinuity size.3.2.2 apthe discontinuity size that can be detected withp
9、robability p.3.2.2.1 DiscussionEach discontinuity size has an inde-pendent probability of being detected and corresponding prob-ability of being missed. For example, being able to detect aspecific discontinuity size with probability p does not guaran-tee that a larger size discontinuity will be foun
10、d.3.2.3 ap/cthe discontinuity size that can be detected withprobability p with a statistical confidence level of c.3.2.3.1 Discussionap/cis calculated by applying a statis-tical uncertainty bound to ap. The uncertainty bound is afunction the amount of data, the scatter in the data, and thespecified
11、level of statistical confidence. The resulting valuerepresents how large the discontinuity with POD equal to pcould be when uncertainty associated with estimating apisaccounted for. Hence ap/c ap. Note that POD is equal to p forboth ap/cand ap. apis based solely on the hit/miss data resultingfrom th
12、e examination and represents a snapshot in time,whereas ap/caccounts for the uncertainty associated withlimited sample data.1This practice is under the jurisdiction of ASTM Committee E07 on Nonde-structive Testing and is the direct responsibility of Subcommittee E07.10 onSpecialized NDT Methods.Curr
13、ent edition approved Jan. 15, 2012. Published February 2012. DOI:10.1520/E2862-12.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary pag
14、e onthe ASTM website.3Available from Standardization Documents Order Desk, DODSSP, Bldg. 4,Section D, 700 Robbins Ave., Philadelphia, PA 19111-5098, http:/dodssp.daps.dla.mil.1Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.4. Summary
15、 of Practice4.1 This practice describes step-by-step the process foranalyzing nondestructive testing hit/miss data resulting from aPOD examination, including minimum requirements for vali-dating the resulting POD curve.4.2 This practice also includes definitions and discussionsfor results of interes
16、t (for example, a90/95) to provide forcorrect interpretation of results.5. Significance and Use5.1 The POD analysis method described herein is based ona well-known and well established statistical method. It shallbe used to quantify the demonstrated POD for a specific set ofexamination parameters an
17、d known range of discontinuitysizes when the initial response from a nondestructive evalua-tion inspection system is ultimately binary in nature (that is, hitor miss). This method requires that a relationship betweendiscontinuity size and POD exists and is best described by ageneralized linear model
18、 with the appropriate link function forbinary outcomes.5.2 Prior to performing the analysis it is assumed that thediscontinuity of interest is clearly defined; the number anddistribution of induced discontinuity sizes in the POD speci-men set is known and well-documented; discontinuities in thePOD s
19、pecimen set are unobstructed; the POD examinationadministration procedure (including data collection method) iswell-defined, under control, and unbiased; and the initialresponse is ultimately binary in nature (that is, hit or miss). Theanalysis results are only valid if convergence is achieved andth
20、e model adequately represents the data.5.3 The POD analysis method described herein is consistentwith the analysis method for binary data described in MIL-HDBK-1823A, which is included in several widely utilizedPOD software packages to perform a POD analysis on hit/missdata. It is also found in stat
21、istical software packages that havegeneralized linear modeling capability. This practice requiresthat the analyst has access to either POD software or othersoftware with generalized linear modeling capability.6. Procedure6.1 The POD analysis objective shall be clearly defined bythe responsible engin
22、eer or by the customer.6.1.1 The analyst shall obtain the hit/miss data resultingfrom the POD examination, which shall include at a minimumthe documented known induced discontinuity sizes, whether ornot the discontinuity was found, and any false calls.6.2 The analyst shall also obtain specific infor
23、mation aboutthe POD examination, which shall include at a minimum thespecimen standard geometry (for example, flat panels), speci-men standard material (for example, Nickel), examination date,number of inspectors, type of inspection method (for example,line-of-site Level 3 Fluorescent Penetrant Insp
24、ection), andpertinent comments from the inspector(s) and test administra-tor.6.3 Prior to performing the analysis, the analyst shallconduct a preliminary review of the POD examination proce-dure and resulting hit/miss data to identify any examinationadministration or data issues. The analyst shall r
25、esolve anyissues prior to conducting the POD analysis. Examples ofexamination administration or data issues and possible resolu-tions are:6.3.1 If problems or interruptions occurred during the PODexamination that may bias the results, the POD examinationshould be re-administered. If this occurs, it
26、shall be docu-mented in the report.6.3.2 If a discontinuity was missed because it was ob-structed (such as a clogged discontinuity), the discontinuityshall be removed from the POD analysis since there was not anopportunity for the discontinuity to be found. If a discontinuityis removed from the anal
27、ysis, the specific discontinuity andrationale for removal shall be documented in the final report.6.3.3 POD cannot be modeled as a continuous function ofdiscontinuity size if there is a complete separation of missesand hits as crack size increases. If a complete separation ofmisses and hits is prese
28、nt in the data, the POD examinationmay be re-administered. If this occurs, it shall be documentedin the report. If a complete separation of misses and hits occurson a regular basis, the specimen set should be examined forsuitability as a POD examination specimen set.6.3.4 POD cannot be modeled as a
29、continuous function ofdiscontinuity size if all the discontinuities are found or if all thediscontinuities are missed. If this occurs, the specimen set isinadequate for the POD examination.6.4 The analyst shall use a generalized linear model with theappropriate link function to establish the relatio
30、nship betweenPOD and discontinuity size. For application to POD, thegeneralized linear model with discontinuity size as the singlepredictor variable is typically expressed as g(y) = b0+b1a org(y) = b0+b1ln(a), where a or ln(a) is the continuous predictorvariable, b0is the intercept, b1is the slope,
31、y is the binaryresponse variable, and g() is the function that “links” thebinary response with the predictor variable. If predictor vari-ables other than discontinuity size are quantifiable factors, ageneralized linear model with more than one predictor may beused.6.5 The analyst shall choose the ap
32、propriate link functionbased on how well the model fits the observed data. MIL-HDBK-1823A discusses four different link functions (Logit,Probit, Log-Log, Complementary-LogLog) and describesmethods for selecting the appropriate one. In general, the logitand probit link functions have worked well in p
33、ractice formodeling hit/miss data.6.6 Only hit/miss data for induced discontinuities shall beused in the development of the generalized linear model. Falsecall data shall not be included in the development of thegeneralized linear model.6.7 The analyst shall conduct the analysis using softwarethat h
34、as generalized linear modeling capabilities.6.8 After running the analysis, the analyst shall verify thatconvergence has been achieved. The resulting POD curve shallnot be used if convergence has not been achieved.6.9 After verifying convergence, the analyst shall use at aminimum the informal model
35、diagnostic methods listed belowto assess the reliability of the model and verify that the modeladequately fits the data.E2862 1226.9.1 If included in the analysis output, the analyst shallcheck the number of iterations it took to meet the convergencecriterion. If more than twenty iterations were nee
36、ded to reachconvergence, the model may not be reliable. A statementindicating that convergence was achieved and the number ofiterations needed to achieve convergence shall be included inthe report.6.9.2 The analyst shall visually assess the shape of the PODcurve. (POD curves tend to be s-shaped.)6.9
37、.3 The analyst shall visually assess how well the PODcurve fits the data by comparing how well the range over whichthe POD curve is rising matches the range over which missesbegin to overlap with and transition to hits as discontinuity sizeincreases.6.9.4 The analyst should also compare an empirical
38、 PODcurve to the POD curve based on the generalized linear model.The empirical POD curve shall be used for validation purposesonly. It shall not be used as a substitute for a POD curveresulting from a hit/miss analysis.6.9.4.1 To create an empirical POD curve, divide thediscontinuity sizes into bins
39、. For example, (0.010 in., 0.020in.), (0.020 in., 0.030 in.), , (0.100 in., 0.110 in.), etc.(0.0254 cm, 0.0508 cm), (0.0508 cm, 0.0762 cm), , (0.2540cm, 0.2794 cm), etc.). For each bin, calculate the total numberof discontinuities contained in the bin and how many weredetected. Calculate the empiric
40、al POD in each bin by dividingthe number detected in the bin over the total number ofdiscontinuities in the bin. Plot the empirical POD versus themidpoint of the bin to obtain the empirical POD curve. Overlaythe POD curve based on the generalized linear model on theempirical POD curve to assess how
41、well the generalized linearmodel fits the data by how well it matches the empirical PODcurve.6.9.5 If applicable, the analyst shall visually assess theshape of the confidence bound on the POD curve. Theconfidence bound should roughly follow the same shape as thePOD curve. If the confidence bound fla
42、res out significantly oneither or both ends or intersects the x-axis, the confidencebound should be viewed as suspect and may not be reliable.6.9.6 The analyst should assess the impact of data thatappears to be outlying (for example, an early hit in the smallsize range or a late miss in the large si
43、ze range) by removingthe outlying value from the data and re-running the analysis toassess its influence on the shape of the POD curve andconfidence bound (if applicable). Both analysis results (withand without the outlying data) shall be included in the reportalong with a discussion of the impact t
44、o the POD curve andconfidence bound (if applicable).6.9.7 The analyst shall analyze any false call data and shallreport the false call rate at the 50 %, 90 %, and 95 % level ofstatistical confidence. Acceptable false call rates shall bedetermined by the responsible engineer or by the customer.6.9.7.
45、1 The false call rate shall be defined as the number offalse calls divided by the number of opportunities in thespecimen set that do not contain discontinuity.6.9.7.2 What constitutes a false call shall be clearly definedby the responsible engineer or by the customer.6.9.7.3 What constitutes an oppo
46、rtunity in the specimen setthat does not contain a discontinuity shall be clearly defined bythe responsible engineer or by the customer.6.9.7.4 The Clopper-Pearson binomial method for con-structing confidence intervals for proportions should be used tocalculate the false call rate at the 50 %, 90 %
47、and 95 % level ofstatistical confidence. The Clopper-Pearson upper 100(1-a)%confidence bound for p is:PU5H1 1n xx 1 1! F1a,2x12, 2n2x!J1where F(1a, 2x+2, 2n2x)is the F-statistics with degrees offreedom (2x+2, 2n2x) and PF F(1a, 2x+2, 2n2x)=1a. Thismethod is consistent with that used in MIL-HDBK-1823
48、A.7. Report7.1 At a minimum the following information about the PODanalysis shall be included in the report.7.1.1 The specimen standard geometry (for example, flatpanels).7.1.2 The specimen standard material (for example, Nickel).7.1.3 Examination date.7.1.4 Number of inspectors.7.1.5 Type of inspec
49、tion method (for example, line-of-siteLevel 3 Fluorescent Penetrant Inspection).7.1.6 Any comments from the inspector(s) or test adminis-trator.7.1.7 The documented known induced discontinuity sizes.7.1.8 Which discontinuities were found and which weremissed.7.1.9 Any false calls.7.1.10 The selected link function.7.1.11 The generalized linear model coefficients.7.1.12 The variance-covariance matrix (if included in thesoftware output).7.1.13 A statement indicating that convergence wasachieved.7.1.14 The number of iterations needed to achieve conver-g
