1、Designation: F3263 17Standard Guide forPackaging Test Method Validation1This standard is issued under the fixed designation F3263; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses i
2、ndicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.INTRODUCTIONThe tests often used by engineers in regulated industries such as medical device or pharmaceuticalsare well known and referenced in both ASTM and ISO literatu
3、re. However, questions around thevalidation of these tests are not nearly as well understood. Questions that often arise are; how shouldone validate these test methods? Should they be validated at all? To what degree should they bevalidated?One answer to this is the guidance provided by ISO 11607-1
4、and ISO 11607-2 where it is stated that“all test methods used to show compliance with this part of ISO 11607 shall be validated anddocumented.”Unfortunately, this does not answer all questions as little is provided in how to demonstrateconformance to these requirements. This is due to the fact that
5、there needs to be a great deal offlexibility in how these test methods are used. Not all circumstances and test methods require the samedegree of scrutiny. Therefore, when assessing when, why, and how a test method should be validated,it is critical to keep this flexibility in mind and use the best
6、tools available to answer the abovequestions appropriately for a given situation.Arobust risk assessment process is arguably the best toolfor determining the risk associated with a particular design element being tested. For example, thereare clear differences in the risk associated with testing the
7、 adhesion of a label versus testing theintegrity of a sterile barrier when viewed from the perspective of patient safety. If a label is missing,the product would be discarded, and a new one that is properly labeled chosen. However, if the sterilebarrier has been compromised due to a seal breach or p
8、inhole in the web of the material, this may goundetected, a contaminated device may be used, and the patient may become infected.The typical process for determining the level of risk associated with medical device packagingcomponents is the failure mode effects analysis tool, commonly referred to as
9、 an FMEA. The FMEAprocess is intended to identify potential failure modes for a product or process, to assess the riskassociated with those failure modes, to rank the issues in terms of importance, and to identify anddocument mitigation strategies that address the most serious concerns. There are ma
10、ny guides andstandards available that describe this process, such as SAE J1739, AIAG FMEA-3 and MIL-STD-1629A. The present guide will be helpful in proposing ways to go about defining what approaches totest method validation that will work best in a given application based on the associated risk, an
11、d willalso provide guidance on the execution of the validation.1. Scope1.1 This guide provides information to clarify the process ofvalidating packaging test methods specific for an organizationutilizing them as well as through inter-laboratory studies (ILS),addressing consensus standards with inter
12、-laboratory studies(ILS) and methods specific to an organization.1.1.1 ILS discussion will focus on writing and interpretationof test method precision statements and on alternative ap-proaches to analyzing and stating the results.1.2 This document provides guidance for defining anddeveloping validat
13、ions for both variable and attribute dataapplications.1This test method is under the jurisdiction ofASTM Committee F02 on PrimaryBarrier Packaging and is the direct responsibility of Subcommittee F02.50 onPackage Design and Development.Current edition approved Dec. 15, 2017. Published March 2018. DO
14、I: 10.1520/F326317.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United StatesThis international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theD
15、evelopment of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.11.3 This guide provides limited statistical guidance;however, this document does not purport to give concretesample sizes for all packaging types and
16、test methods. Empha-sis is on statistical techniques effectively contained in referencedocuments already developed by ASTM and other organiza-tions.1.4 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this st
17、andard to establish appro-priate safety, health, and environmental practices and deter-mine the applicability of regulatory limitations prior to use.1.5 This international standard was developed in accor-dance with internationally recognized principles on standard-ization established in the Decision
18、 on Principles for theDevelopment of International Standards, Guides and Recom-mendations issued by the World Trade Organization TechnicalBarriers to Trade (TBT) Committee.2. Referenced Documents2.1 ASTM Standards:2E177 Practice for Use of the Terms Precision and Bias inASTM Test MethodsE456 Termino
19、logy Relating to Quality and StatisticsE691 Practice for Conducting an Interlaboratory Study toDetermine the Precision of a Test MethodE2282 Guide for Defining the Test Result of a Test MethodE2782 Guide for Measurement Systems Analysis (MSA)F17 Terminology Relating to Primary Barrier PackagingF2097
20、 Guide for Design and Evaluation of Primary FlexiblePackaging for Medical Products2.2 ISO Standards:3ISO 11607-1: 2006/A1: 2014 Packaging for terminallysterilized medical devicesPart 1: Requirements formaterials, sterile barrier systems, and packaging, Amend-ment 1ISO/TS 16775 Packaging for terminal
21、ly sterilized medicaldevicesGuidance on the application of ISO 11607-1 andISO 11607-23. Terminology3.1 Definitions of Terms Specific to This Standard:3.1.1 accuracy, nsee E177.3.1.2 alpha risk error (), nthe probability that an inspec-tor will reject a conforming unit. Also referred to as producersr
22、isk or type I error. For the purposes of this document this errortype will be referred to as Alpha risk error.3.1.3 appraiser, nterm used to identify individual(s) thatwill execute test method validation activities. May commonlyalso be referred to as appraisers or technicians.3.1.4 as defined by tea
23、m with rationale, nthe validationteam determines a performance level or sample size withacceptance criteria. When a test method falls under thiscategory another option may be no testing required.3.1.5 attribute test method, ntests that return a pass/failoutput measurement on a characteristic that is
24、 either conform-ing or nonconforming. Variable measurement data treated asattribute also qualifies.3.1.6 acceptable quality level (AQL), nrepresents a levelof quality that a sampling plan routinely accepts. Lots at orbelow theAQLare accepted at least 95% of the time. TheAQLmay be determined from the
25、 sampling plans Operating Char-acteristic (OC) Curve.3.1.7 beta risk error (), nthe probability that an inspectorwill accept a nonconforming unit.Also referred to as beta error(escape rate) or type II error. For the purposes of this documentthis error type will be referred to as Beta risk error or (
26、).3.1.8 borderline samples, nmarginally passing or failingsamples.3.1.9 comparative test method, na test method that is usedfor comparing the means of two or more populations using astatistical test (e.g. 2-sample t test, ANOVA test). A compara-tive test method is NOT used for accepting or rejecting
27、individual units, and the output usually does NOT havespecification limits.3.1.10 failure modes effects analysis, nFailure modes andeffects analysis (FMEA) is a step-by-step approach for identi-fying all possible failures in a design, a manufacturing orassembly process, or a product or service.43.1.
28、11 highly instrumental method, na test method wherethe result is not dependent on the operator.3.1.12 lot tolerance percent defective (LTPD), nin asampling plan, represents a level of quality that a samplingplan routinely rejects. Lots at or above the LTPD are rejectedat a probability level determin
29、ed by the confidence level. TheLTPD may be determined from the sampling plans OperatingCharacteristic (OC) Curve. Also known as the RejectableQuality Level (RQL), Limiting Quality Level (LQ), andUnacceptable Quality Level (UQL).3.1.13 measurement resolution, nthe smallest detectableincrement that ca
30、n be measured by the test method.3.1.14 precision, nsee E177.3.1.15 %P/T (precision to tolerance ratio), n%P/T is atest method performance metric of a Gage R theprotection offered by a sampling plan shown graphically.3.1.17 repeatability, nsee E177.3.1.18 reproducibility, nsee E177.2For referenced A
31、STM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.3Available from American National Standards Institute (ANSI), 25 W. 43rd St.,4
32、th Floor, New York, NY 10036, http:/www.ansi.org.4http:/asq.org/learn-about-quality/process-analysis-tools/overview/fmea.htmlF3263 1723.1.19 %R createa task group, identify an ILS coordinator, create the experi-mental design, execute the testing, analyze the results, anddocument the resulting precis
33、ion statement in the test method.For more detail on how to conduct an ILS refer to E691-14.4.1.4 Writing Precision and Bias StatementsWhen writingPrecision and Bias Statements for an ASTM standard, theminimum expectation is that the Standard Practice outlined inE177-14 will be followed. However, in
34、some cases it may alsobe useful to present the information in a form that is moreeasily understood by the user of the standard. Examples can befound in 4.1.5 below.4.1.5 Alternative Approaches to Analyzing and StatingResultsVariable Data:4.1.5.1 Capability Study:(1) Aprocess capability greater than
35、2.00 indicates the totalvariability (part-to-part plus test method) of the test outputshould be very small relative to the tolerance. Mathematically,Pp 5Specifiction Tolerance6Total$2.00$Total#112Specification Tolerance(1)(2) Notice, Totalin the above equation includes PartandTM. Therefore, two conc
36、lusions can be made:F3263 173(a) The test method can discriminate at least 1/12 of thetolerance and hence the test method resolution is adequateTherefore, no additional analysis such as a Gage R or(2) Refer to the chart example in Appendix X1.4.2.6.5 Step5Create the validation plan:(1) Determine the
37、 proportion of each defect in the sample.(a) While some sort of rationale should be provided forhow the defect proportions are distributed in the ATMV, thereis some flexibility in choosing the proportions. Therefore,different strategies may be employed for different products andprocesses, for exampl
38、e 10 defective parts in 30 or 20 defects in30. The cost of the samples along with the risk associated withincorrect outcomes affects decision making.(b) Scrap production data will often not be available fornew products. In these instances, use historical scrap from asimilar product or estimate the e
39、xpected scrap proportionsbased on process challenges that were observed during devel-opment.Another option is to represent all of the defects evenly.4.2.6.6 Step 6 Determine the number of inspectors anddevices needed:(1) When the number of trials is large, consider employingmore than three inspector
40、s to reduce the number of uniqueparts required for the test. More inspectors can inspect thesame parts without adding more parts to achieve additionaltrials and greater statistical power.(2) Inspectors are not required to all look at the samesamples, although this is probably the simplest approach.(
41、3) For semi-automated inspection systems that are sensi-tive to fixture placement or setup by the inspector, multipleinspectors should still be employed for the test.(4) For automated inspection systems that are completelyinspector independent, only one inspector is needed. However,in order to reduc
42、e the number of unique parts needed, considercontrolling other sources of variation such as various lightingconditions, temperature, humidity, inspection time, day/nightshift, and part orientations.4.2.6.7 Step7Prepare the Inspectors:(1) Train the inspectors prior to testing:(a) Explain the purpose
43、and importance of ATMV to theinspectors.(b) Inspector training should be a two-way process. Thevalidation team should seek feedback from the inspectors onthe quality and clarity of visual standards, pictures and writtendescriptions in the inspection documentation.(1) Are there any gray areas that ne
44、ed clarification?(2) Would a diagram be more effective than an actualpicture of the defect?(c) Review borderline samples. Consider adding pictures/diagrams of borderline samples to the visual standards. In somecases there may be a difference between functional andcosmetic defects. This may vary by m
45、ethod/package type.(d) Some validation teams have performed dry run testingto characterize the current effectiveness of the inspection. Notethat the same samples should not be used for dry run testingand final testing if the same inspectors are involved in bothtests.4.2.6.8 Step 8 Select a represent
46、ative group of inspectorsas the test group:(1) There will be situations, such as site transfer, where allof the inspectors have about the same level of familiarity withthe product. If this is the case, select the test group ofinspectors based on other sources of variability within theinspectors, suc
47、h as their production shift, skill level or years ofexperience with similar product inspection.(2) The inspectors selected for testing should at least havefamiliarity with the product, or this becomes an overly conser-vative test. For example, a lack of experience with the productmay result in an in
48、crease in false positives.(3) Document that a varied group of inspectors wereselected for testing.4.2.6.9 Step9Prepare the Test Samples:(1) Collect representative units.(a) Be prepared forATMV testing by collecting represen-tative defect devices early and often in the developmentprocess. Borderline
49、samples are particularly valuable to collectat this time. However, be aware that a sample that cannot evenbe agreed upon as good or bad by the subject matter experts isonly going to cause problems in the testing. Instead, choosesamples that are representative of “just passing” and “justfailing” relative to the acceptance criteria.(2) Use the best judgment as to whether the man-madedefect samples adequately represent defects that naturallyoccur during the sealing process, distribution simulation, orother manufacturing processes, for example. I
