1、AN AMERICAN NATIONAL STANDARDMETHODS FORPERFORMANCE EVALUATIONOF COORDINATE MEASURINGSYSTEM SOFTWARE ASME B89.4.10-2000Date of Issuance: July 15, 2002The next edition of this Standard is scheduled for publication in 2005. Therewill be no addenda issued to this Edition.ASME issues written replies to
2、inquiries concerning interpretations oftechnical aspects of this Standard.ASME is the registered trademark of The American Society of Mechanical Engineers.This code or standard was developed under procedures accredited as meeting the criteria forAmerican National Standards. The Standards Committee t
3、hat approved the code or standardwas balanced to assure that individuals from competent and concerned interests have had anopportunity to participate. The proposed code or standard was made available for public reviewand comment that provides an opportunity for additional public input from industry,
4、 academia,regulatory agencies, and the public-at-large.ASME does not “approve,” “rate,” or “endorse” any item, construction, proprietary device,or activity.ASME does not take any position with respect to the validity of any patent rights asserted inconnection with any items mentioned in this documen
5、t, and does not undertake to insure anyoneutilizing a standard against liability for infringement of any applicable letters patent, nor assumeany such liability. Users of a code or standard are expressly advised that determination of thevalidity of any such patent rights, and the risk of infringemen
6、t of such rights, is entirely theirown responsibility.Participation by federal agency representative(s) or person(s) affiliated with industry is not tobe interpreted as government or industry endorsement of this code or standard.ASME accepts responsibility for only those interpretations of this docu
7、ment issued inaccordance with the established ASME procedures and policies, which precludes the issuanceof interpretations by individuals.No part of this document may be reproduced in any form,in an electronic retrieval system or otherwise,without the prior written permission of the publisher.The Am
8、erican Society of Mechanical EngineersThree Park Avenue, New York, NY 10016-5990Copyright 2002 byTHE AMERICAN SOCIETY OF MECHANICAL ENGINEERSAll Rights ReservedPrinted in U.S.A.CONTENTSForeword vStandards Committee Roster . viCorrespondence With the B89 Committee . viii1 Scope 11.1 1Assumptions .1.2
9、 1Application 1.3 1References 2 Terms and Definitions 13 Software Functions . 23.1 2Input Data 3.2 2Data Analysis .4 Performance Characterization . 34.1 3Evaluation of Quality 4.2 4Characteristics of Robustness .4.3 4Characteristics of Reliability 4.4 5Characteristics of Ease-of-Use 4.5 5Related Iss
10、ues .5 Test Methodologies . 65.1 6Test Principles .5.2 6Apparatus .5.3 7Test Procedure 5.4 7Input Parameters .5.5 8Generation of Test Data .5.6 10Test Set .5.7 10Process Data With Test Software .5.8 10Calculation and Interpretation of Results 5.9 10Reporting of Test Results 5.10 11Periodic Reverific
11、ation .6 Software Documentation . 116.1 11Purpose .6.2 11Compliance 6.3 11Required Information Figures1 Example of Fit Bounding 22 Line Evaluation 23 Circle Evaluation 3iii4 Plane Evaluation . 35 Sphere Evaluation 36 Cylinder Evaluation 47 Cone Bounding 58 Cone Evaluation . 69 Major Components of a
12、Software Testing System . 7Tables1 Circle Fit Types . 42 Evaluation Parameters 53 Number of Required Form Errors . 9Mandatory AppendixI Mathematical Descriptions of Form Errors . 13Nonmandatory AppendicesA Factors That Influence the Results 14B NIST Algorithm Testing System (ATS) . 15C Example Docum
13、entation 17D Substitute Features . 19E Datum Reference Frames (DRF) 22F Functional Gage Simulation 23G References . 25ivFOREWORDCoordinate measuring systems (CMSs) rely upon software that processes coordinate data;often this software computes fits of geometric elements to such data. The performance
14、ofthese fits can vary among software packages, and in some cases can be a significantcontributor to the overall uncertainty of measurement.The purpose of this document is to provide guidelines for evaluating the quality ofsolutions generated by CMS software and to define minimal documentation requir
15、ementsfor software providers. This Standard is concerned with testing the behavior of algorithmimplementation, not the testing of algorithms themselves. It is not the intent of this documentto endorse or rate any computational method or system. A mechanism for generatingcollections of test data sets
16、 is specified. While a specific, static collection of standardizedtest data sets is not defined, the generating mechanism can produce several collections ofsimilar character.This Standard was approved by the American National Standards Institute on December1, 2000.vASME STANDARDS COMMITTEE B89Dimens
17、ional Metrology(The following is the roster of the Committee at the time of approval of this Standard.)OFFICERSR. B. Hook, ChairB. Parry, Vice ChairP. Esteban, SecretaryCOMMITTEE PERSONNELK. L. Blaedel, University of CaliforniaJ. B. Bryan, Bryan AssociatesT. Carpenter, U.S. Air ForceT. Charlton, Bro
18、wn (b) reduces the possibility of error in software appli-cation;(c) defines a method of comparing CMS software.This document covers the following areas: inputdata, feature construction, software documentation, per-formance characterization, and test methodologies.1.1 AssumptionsThe assumptions inhe
19、rent in this document are asfollows.(a) Measurement uncertainty in coordinate samplesis not addressed. For information on measurement uncer-tainty, refer to Appendix A and ASME B89.3.2, Dimen-sional Measurement Methods (when released).(b) Methods to input predetermined samples to thecomputational sy
20、stem are available.(c) Personnel have adequate experience and trainingto implement the evaluation and understand the implica-tions of the results.11.2 ApplicationThis document is one component required for theevaluation of CMSs. Other relevant documents can befound in Appendix G.1.3 ReferencesWhen t
21、he following American National Standardsreferred herein are superseded by a revision approvedby the American National Standards Institute, Inc., therevision shall apply.ASME B89.1.12M-1990 Methods for Performance Eval-uation of Coordinate Measuring MachinesASME B89.4.1-1997, Methods for Performance
22、Evalua-tion of Coordinate Measuring MachinesASME Y14.5M-1994, Dimensioning and TolerancingASME Y14.5.1M-1994, Mathematical Definition of Di-mensioning and Tolerancing PrinciplesPublisher: The American Society of Mechanical Engi-neers (ASME International), Three Park Avenue, NewYork, NY 10016-5990; A
23、SME Order Department, 22Law Drive, Box 2900, Fairfield, NJ 07007-2900References to other documents are for informationalpurposes only.2 TERMS AND DEFINITIONSalgorithm: a well defined procedure for solving aparticular problem, e.g., sorting algorithms.Coordinate Measuring System (CMS): any piece ofeq
24、uipment which collects coordinates (points), calculatesand displays additional information using the measuredpoints.datum: a theoretically exact point, line, or plane de-rived from a feature on a part. See ASME Y14.5M-1994.datum reference frame (DRF): a part coordinate sys-tem constructed from datum
25、s. See ASME Y14.5M-1994.METHODS FOR PERFORMANCE EVALUATIONOF COORDINATE MEASURING SYSTEM SOFTWAREASME B89.4.10-2000Test fitReference lineBounding pointsFIG. 1 EXAMPLE OF FIT BOUNDINGleast squares fit feature: a feature of perfect orthogonalform, corresponding to a set of data points, whichminimizes
26、the sum of the squared deviations betweenthe feature and the individual data points. (ReferenceAppendix D for additional information.)objective function: a function which is to be optimizedby searching for a minimum (or maximum) as itsparameters are varied. A different objective functionis used for
27、each type of fit, e.g., least squares versusminimum circumscribed circle.reference evaluation: the evaluation of the substitutefeature using a known implementation of an algorithm.reference feature: a substitute feature used as thebasis for evaluating a test feature.substitute feature: a feature of
28、perfect geometric formwhich corresponds to a set of data points and is intendedto minimize an objective function.test: a basic unit of evaluation, based on one or morerelated data sets, which are applied to one or moresoftware implementations of an algorithm.test feature: a substitute feature comput
29、ed by thesoftware under test.3 SOFTWARE FUNCTIONSCMS hardware is used to collect data points (rawdata) on the surfaces of parts being inspected. CMSsoftware processes these raw data to construct datumreference frames (part coordinate systems) and substitutefeatures that represent the surfaces being
30、inspected.From these constructions, the CMS software can evalu-ate such characteristics as size, location, orientation,and form.2Bounded test fitReference lineAngleDFIG. 2 LINE EVALUATION3.1 Input DataRaw data to be used to test and analyze CMSsoftware may be obtained by physically inspecting atest
31、workpiece or by mathematical computation. Theformer represents a test of the entire measuring system,while the latter approach avoids operator, workpiece,environment, and machine influences. The latter ap-proach also makes possible closer control of the rawdata sets, including limits on their spatia
32、l distributionas well as inclusion of artificially induced form errors.For software analysis, the latter approach is the mostuniversally accepted and the most reliable. This is theapproach addressed herein.3.2 Data AnalysisThe raw data points are processed by mathematicalalgorithms whose purposes ar
33、e to calculate perfect-form substitute features. First, substitute features arecalculated to represent the original data. Then thesubstitute features are used to evaluate conformance totolerances or to determine other geometric characteris-tics of the work piece. An alternative to the useof substitu
34、te features is the use of Functional GageSimulation, described in Appendix F.Different methods can be used for obtaining substitutefeatures. These methods may have different objectivefunctions, i.e., different criteria for deciding that aparticular substitute feature is better or worse than otherpos
35、sible substitute features. Different criteria can, ingeneral, lead to different results. The proper selectionof fitting criterion and data analysis method is outsidethe scope of this document.Fit criteria are usually based on LP-norm estimation,or minimum circumscribed, or maximum inscribedmethods.
36、Refer to Appendix D for explanations of thesemethods.A Datum Reference Frame (DRF) is used to establishthe proper relationships of features to their specifica-tions. A discussion of DRFs is contained in Appendix E.METHODS FOR PERFORMANCE EVALUATIONOF COORDINATE MEASURING SYSTEM SOFTWARE ASME B89.4.1
37、0-2000The objective of this Standard is not to decree thatany one method is better than any other. Guidance isprovided to the user for checking whether particularCMS software produces proper results within the contextof the design requirement.4 PERFORMANCE CHARACTERIZATIONThis section establishes th
38、e characteristics by whichCMS software performance is evaluated. These charac-teristics are discussed in terms of four categories:quality, robustness, reliability, and ease of use.Characteristics which are not used for performanceevaluation in this Standard are discussed at the end ofthis section.4.
39、1 Evaluation of QualityIn this Standard, the quality of the algorithm isevaluated on the basis of the geometric deviation ofthe test feature from a reference feature.4.1.1 Evaluation Concept. Some features haveunbounded geometry, e.g., lines have infinite length.For the purposes of evaluation, unbou
40、nded features arebounded by their sample point sets. The resultantbounded test feature is then compared to the referencefeature. Evaluation parameters are defined for each typeof feature. See Fig. 1.4.1.2 Evaluation Parameters. Each feature typehas a unique set of evaluation parameters. Test results
41、are reported as outlined below. The figures in thissection have the following annotation conventions:Test fitReference circleDrrRtFIG. 3 CIRCLE EVALUATION3Reference planeDTest planeAFIG. 4 PLANE EVALUATIONR p reference fit parameter subscriptt p test fit parameter subscriptD p separation distancer p
42、 radiusA p anglea p cone half-angle4.1.2.1 Line. The test line is bounded by theperpendicular projection of the sample points onto thetest line. The evaluation parameters are (see Fig. 2):(a) the largest separation distance between the axesof test and reference features;(b) the angle between the tes
43、t and reference features.4.1.2.2 Circle. The test circle is a closed objectand naturally bounded. The evaluation parameters are(see Fig. 3):(a) the absolute value of the difference between theradii of the test and reference circles (|rR rt|);(b) the distance between the centers of the test andrefere
44、nce circles. This may be a three-dimensionaldistance.(c) the angle between the planes of the test andreference circles if applicable (see Table 1).Test fitReference sphereDrrRtFIG. 5 SPHERE EVALUATIONMETHODS FOR PERFORMANCE EVALUATIONOF COORDINATE MEASURING SYSTEM SOFTWAREASME B89.4.10-2000TABLE 1 C
45、IRCLE FIT TYPESCircle Fit Type Reported AngleTwo-dimensional Not applicable (p 0)Three-dimensional, both use same ref. plane Not applicable (p 0)Three-dimensional, fit plane then two-dimensional circle Angle between fit planesThree-dimensional circle fit Angle between planes4.1.2.3 Plane. The test p
46、lane is unbounded. Sam-ple points are projected onto the test plane for theevaluation. The evaluation parameters are:(a) the largest perpendicular distance from the refer-ence plane to any projected sample point in the testplane (see D in Fig. 4);(b) the angle between the test and reference planes.4
47、.1.2.4 Sphere. The test sphere is a closed objectand naturally bounded. The evaluation parameters are(see Fig. 5):(a) the absolute value of the difference between theradii of the test and reference spheres (|rR rt|);(b) the distance between the centers of the test andreference spheres.4.1.2.5 Cylind
48、er. The test cylinder is boundedalong its axis by projecting the sample points perpendic-ularly onto its axis. It is naturally bounded in circumfer-ence. The evaluation parameters are:(a) the absolute value of the difference between theradii of the test and reference cylinders (|rR rt|);(b) the maxi
49、mum perpendicular distance from thebounded test cylinder axis to the axis of the referencecylinder (see D in Fig. 6);(c) the angle between the axes of the test andreference cylinders (see A in Fig. 6).4.1.2.6 Cone(a) The test cone is bounded along its axis by:(1) projecting the sample data perpendicularly ontothe test cone surface; and(2) projecting these surface points perpendicularlyonto the test fit axis (see Fig. 7).It is naturally bounded in circumference. The referencecone axis is similarly bounded.(b) The cone evaluation parameters are:(1) for each cone, the perpendicular distance
