1、Designation: E3064 16Standard Test Method forEvaluating the Performance of Optical Tracking Systemsthat Measure Six Degrees of Freedom (6DOF) Pose1This standard is issued under the fixed designation E3064; the number immediately following the designation indicates the year oforiginal adoption or, in
2、 the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 PurposeThis test method presents metrics and proce-dures for measuring, analyzing,
3、and reporting the relative poseerror of optical tracking systems that compute the pose (that is,position and orientation) of a rigid object while the object ismoving.1.2 UsageSystem vendors may use this test method todetermine the performance of their Six Degrees of Freedom (6DOF) optical tracking s
4、ystem which measures pose. This testmethod also provides a uniform way to report the measurementerrors and measurement capability of the system. System usersmay use this test method to verify that the systems perfor-mance is within the users specific requirements and within thesystems rated performa
5、nce.1.3 Test LocationThe procedures defined in this standardshall be performed in a facility in which the environmentalconditions are within the optical tracking systems ratedconditions.1.4 Test VolumeThis standard shall be used for testing anoptical tracking system working volumes of 3000 mm long b
6、y2000 mm wide by 2000 mm high, 6000 mm long by 4000 mmwide by 2000 mm high, or 12 000 mm long by 8000 mm wideby 2000 mm high.1.5 UnitsThe values stated in SI units are to be regardedas standard. No other units of measurement are included in thisstandard.1.6 This standard does not purport to address
7、all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2E2919 Test Method
8、for Evaluating the Performance ofSystems that Measure Static, Six Degrees of Freedom(6DOF), PoseE177 Practice for Use of the Terms Precision and Bias inASTM Test Methods2.2 ASME Standard:3B89.4.19 Performance Evaluation of Laser-Based SphericalCoordinate Measurement Systems2.3 ISO/IEC Standards:4ISO
9、/IEC Guide 99:2007 International Vocabulary ofMetrologyBasic and General Concepts and AssociatedTerms (VIM: 2007)ISO/IEC Guide 983:2008 Uncertainty of measurementPart 3: Guide to the expression of uncertainty in measure-ment (GUM: 1995)IEC 60050-300:2001 International Electro technicalVocabularyElec
10、trical and electronic measurements andmeasuring instrumentsJCGM 200:2012 International Vocabulary of Metrology Ba-sic and General Concepts and Associated Terms (VIM),3rd edition3. Terminology3.1 Definitions:3.1.1 degrees of freedom, DOF, nany of the minimumnumber of translation or rotation component
11、s required tospecify completely the pose of a rigid object. E29193.1.1.1 Discussion1This test method is under the jurisdiction of ASTM Committee E57 on 3DImaging Systems and is the direct responsibility of Subcommittee E57.02 on TestMethods.Current edition approved June 1, 2016. Published June 2016.
12、 DOI: 10.1520/E3064-162For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.3Available from American Society of Mec
13、hanical Engineers (ASME), ASMEInternational Headquarters, Two Park Ave., New York, NY 10016-5990, http:/www.asme.org.4Available from American National Standards Institute (ANSI), 25 W. 43rd St.,4th Floor, New York, NY 10036, http:/www.ansi.org.Copyright ASTM International, 100 Barr Harbor Drive, PO
14、Box C700, West Conshohocken, PA 19428-2959. United States1(1) In a 3D space, a rigid object can have at most 6DOF,three translations and three rotations.(2) The term “degree of freedom” is also used with regardto statistical testing. It will be clear from the context in whichit is used whether the t
15、erm relates to a statistical test or therotation/translation aspect of the object.3.1.2 measurement error, error of measurement, and error,nmeasured quantity value minus a reference quantity value.(JCGM 200:2012)3.1.3 metrology bar, na rod of a known length havingmarkers (active or passive) attached
16、 to both ends and used toestimate the errors of an optical tracking system.3.1.4 optical tracking system, na tracking system that usesmeasurements obtained from camera images.3.1.5 pose, na 6DOF vector whose components representthe position and orientation of a rigid object with respect to acoordina
17、te frame. E29193.1.6 precision, nthe closeness of agreement betweenindependent test results obtained under stipulated conditions.E1773.1.7 rated conditions, nmanufacturer-specified limits onenvironmental, utility, and other conditions within which themanufacturers performance specifications are guar
18、anteed atthe time of installation of the instrument. ASME B89.4.193.1.8 reference system, na measurement instrument orsystem used to generate a reference value or quantity. E29193.1.9 relative pose, nchange of an objects pose betweentwo poses measured in the same coordinate frame. E29193.1.10 repeat
19、ability, nprecision under repeatabilityconditions. E1773.1.11 repeatability conditions, nconditions where inde-pendent test results are obtained with the same method onidentical test items in the same laboratory by the same operatorusing the same equipment within short intervals of time. E1773.1.12
20、tracking system, na system that is used for mea-suring the pose of moving objects and supplies the data as atimely ordered sequence.3.1.13 work volume, na physical space, or region within aphysical space, that defines the bounds within which a rigidobject tracking system is acquiring data. E29194. S
21、ummary of Test Method4.1 This test method provides a set of statistically basedperformance metrics and a test procedure to quantitativelyevaluate the performance of an optical tracking system.4.2 The measurement errors include the positional andorientation error components. Specifically, the test pr
22、oceduremeasures the relative pose between two marker sets rigidlyattached to the opposing ends of a fixed-length metrology baras shown in Fig. 1. The relative pose is then decomposed intopositional and angular components. Measurement errors arecalculated from the positional and angular components as
23、 theartifact is moved about the work volume.5. Significance and Use5.1 Optical tracking systems are used in a wide range offields including: video gaming, filming, neuroscience,biomechanics, flight/medical/industrial training, simulation,robotics, and automotive applications.5.2 This standard provid
24、es a common set of metrics and atest procedure for evaluating the performance of opticaltracking systems and may help to drive improvements andinnovations of optical tracking systems.5.3 Potential users often have difficulty comparing opticaltracking systems because of the lack of standard performan
25、cemetrics and test methods, and therefore must rely on the claimsof a vendor regarding the systems performance, capabilities,and suitability for a particular application. This standard makesit possible for a user to assess and compare the performance ofcandidate optical tracking systems, and allows
26、the user todetermine if the measured performance results are within thespecifications with regard to the application requirements.6. Apparatus6.1 Artifact:6.1.1 A 300 mm long bar with markers rigidly attached toeach end of the metrology bar shall be used as the 6DOFartifact. The bar shall have stiff
27、ness and thermal expansioncharacteristics such that the deflection is less than or equal to0.01 mm. For example, the metrology bar shown in Fig. 2satisfies these requirements.6.1.2 A constant relative 6DOF pose is formed between thetwo clusters of markers located at the ends of the metrologybar. All
28、 markers shall be contained within hemisphericalvolumes with a maximum radius of 100 mm from the ends ofthe bar (see Fig. 1). Examples of metrology bars that can beused to evaluate optical tracking systems are shown in Fig. 2(Ref (1)5.5The boldface numbers in parentheses refer to a list of reference
29、s at the end ofthis standard.FIG. 1 Drawing of the artifact showing critical dimensions of thebar length and the maximum hemispherical volume inside whichthe markers can be placed.E3064 1627. Measurement Procedure7.1 Introduction:7.1.1 This section describes the basic procedure for deter-mining the
30、pose measurement error of an optical trackingsystem.7.2 Pose Measurement:7.2.1 The X and Y axes are aligned with the work volume inthe horizontal plane as shown in Fig. 3, and Z is aligned withthe vertical axis. Move the metrology bar throughout the workvolume along two regular patterns: (X pattern)
31、 parallel, straightline segments back-and-forth along the X axis with the pathsseparated by at most, the metrology bar length as shown in Fig.3 (a) and (Y pattern) parallel, straight line segments back-and-forth along the Y axis with the paths separated by at most, themetrology bar length as shown i
32、n Fig. 3 (b). The distancebetween the boundary lines and the limits of the work volumeshall be at most, one-half of the metrology bar length. Thecenter of the metrology bar shall traverse the X patternfollowed by the Y pattern with artifact orientation #1 as shownin Fig. 3 (c) in a continuous smooth
33、 motion.This traversal shallbe repeated two more times, once with artifact orientation #2and once with artifact orientation #3. The data from all threetraversals shall be combined into a single data set.7.2.2 The metrology bar paths and orientations shall bechosen as described in 7.2.1. The height o
34、f the centroid of themetrology bar shall remain approximately 1000 mm above thebottom of the test volume.FIG. 2 Examples of artifacts for evaluating optical tracking systems having a 300 mm long metrology bar (a) with six passive, reflectivemarkers, within 100 mm radius from the bar end, on each end
35、, and (b) with a reduced pose ambiguity cuboctahedron, Ref (1), within100 mm radius from the bar end, on each end.FIG. 3 The (a) X pattern and (b) Y pattern are combined to make a single path along which the metrology bar is moved throughout thework volume. (c) Artifact (shown with axes on bar cente
36、r) orientations with respect to the path: 1) perpendicular to the path segmentsin the plane of motion, 2) perpendicular to the path segments and normal to the plane of motion, and 3) in-line with the path segmentsin the plane of motion.NOTE 1Example artifact shown in (a) and (b) is oriented with res
37、pect to the path as in 1) perpendicular to the path segments in the plane of motion.E3064 1637.2.3 The centroid of the metrology bar shall be moved at arelatively constant walking speed of 1200 6 700 mm/s.8. Pose Measurement Error8.1 This section describes methods for computing posemeasurement error
38、s of an optical tracking system (OTS) usingthe artifact. For each instance of time t, the optical trackingsystem measures the pose of an object as:OTSHObjectt! 5FOTSRObjectt!0OTSTObjectt!1G(1)Here,OTSRObjectt! isa33matrix describing the orientationof the object andOTSTObjectt! is a 3-dimensional vec
39、tor de-scribing the position of the object in the optical tracking sys-tem coordinate frame. In our artifact, two objects are consid-ered corresponding to the left and right ends of themetrology bar. The optical tracking system measures theposes of the left and right ends, and the corresponding44mat
40、rices are defined respectively as:OTSHLeftt! 5FRLeftt!0TLeftt!1Gand(2)OTSHRightt! 5FRRightt!0TRightt!1GBecause the bar is rigid, the relative pose between the leftobject and the right object is constant in time (see Fig. 4)and can be defined as:LeftHRightt! 5OTSHLeft21OTSHRight5FRLeftt!0TLeftt!1G21F
41、RRightt!0TRightt!1G5FRt!0Tt!1G(3)The angle of rotation is calculated as:t! 5 2* asin =qx2t!1qy2t!1qz2t! (4)Here, (qw(t),qx(t),qy(t),qz(t)Tis the unit quaternion represen-tation of Rt!, Ref (2), where qwt! is the scalar componentof the quaternion.8.1.1 A reference system measurement is used to measur
42、ethe relative pose between two groups of markers.The referencesystem measurement shall have an uncertainty that is at leastten times smaller than the uncertainty of the optical trackingsystem under test. The relative pose between the left object andright object at the ends of the metrology bar is me
43、asured by thereference system measurement as:LeftHRight5FR0T1G5FI0T1G(5)The coordinate frames associated with the right and left endsof the bar are rotationally aligned using the reference sys-tem. Where R=I, and I is the identity matrix.8.1.2 The following sections describe two methods forevaluatin
44、g the optical tracking system. In 8.2, measurementsare taken relative to the measured relative pose of a testartifact, which is measured by a more accurate system, and in8.3 the measurements are taken relative to the mean value ofthe collected data.8.2 Error Statistics using a Reference System:8.2.1
45、 This section describes the computation of system errorstatistics relative to a precisely characterized test artifact, suchas the one described in Section 6.8.2.2 The relative measured poseLeftHRightt! (see Eq 3)attime t can be compared to the reference system measured poseLeftHRight(see Eq 5). Spec
46、ifically, the positional error at time tcan be calculated as:ept!5 Tt!22 T2(6)and the orientation error at time t can be calculated as:eot!5 t! 2 0 5 t! (7)where t! is calculated using Eq 4 and 2denotes the2-norm of the vector.8.2.3 The statistics on these errors include: the root meansquare error,
47、the maximum error, and the percentile error. Theroot mean square error is calculated as:Root Mean Square Error 5 1N(t51Net2(8)The maximum of the errors is defined as:emax5 max?e1?,?e2?,.?eN?! (9)Here, etis the (positional or orientation) error at time t, andN is the number of data samples collected.
48、8.2.4 A method for estimating the percentile error of a dataset is described in Ref (3). For a series of errors |e1|,|e2|,.,|eN|, a new ordered set E1, E2, ., EN is constructed wherethe errors are ordered by increasing value. The percentile errorof the data can be estimated from N measurements as fo
49、llows:for the pth percentile, setp100N 1 1! equal to k+d for k,aninteger, and d, a fraction greater than or equal to 0 and less than1. The estimated error percentile E(p) is defined as follows:Ep! 5HEk1dEk112 Ek! 0,k,NE1, k 5 0EN, k $ N(10)In this standard, E(99.7), E(95) and E(50) shall be reported.8.3 Repeatability without Reference System Measurement:8.3.1 This section describes the computation of systemrepeatability statistics using a test artifact not meas
