1、Designation: E3124 17Standard Test Method forMeasuring System Latency Performance of Optical TrackingSystems that Measure Six Degrees of Freedom (6DOF)Pose1This standard is issued under the fixed designation E3124; the number immediately following the designation indicates the year oforiginal adopti
2、on or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 PurposeThis test method presents metrics and a pro-cedure for measuring, a
3、nalyzing, and reporting the systemlatency of an optical tracking system (OTS) that computes thepose of a rigid object.1.2 UsageSystem vendors may use this test method todetermine or validate the system latency in their trackingsystems. This test method provides a uniform way to measureand report the
4、 system latency along with the uncertainty in thesystem latency. System users may use this test method to verifythat the system latency performance is within the usersspecific requirements and within the systems rated perfor-mance.1.3 This standard does not measure the display latency ofgraphical re
5、presentations of the tracked objects. Display la-tency is external to the optical tracking system.1.4 Test LocationThe procedures defined in this testmethod shall be performed in an environment conforming tothe manufacturers rated conditions.1.5 The values stated in SI units are to be regarded assta
6、ndard. No other units of measurement are included in thisstandard.1.6 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety, health, and environmental practices and d
7、eter-mine the applicability of regulatory limitations prior to use.1.7 This international standard was developed in accor-dance with internationally recognized principles on standard-ization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recom-men
8、dations issued by the World Trade Organization TechnicalBarriers to Trade (TBT) Committee.2. Referenced Documents2.1 ASTM Standards:2E177 Practice for Use of the Terms Precision and Bias inASTM Test MethodsE2655 Guide for Reporting Uncertainty of Test Results andUse of the Term Measurement Uncertain
9、ty in ASTM TestMethodsE2919 Test Method for Evaluating the Performance ofSystems that Measure Static, Six Degrees of Freedom(6DOF), PoseE3064 Test Method for Evaluating the Performance ofOptical Tracking Systems that Measure Six Degrees ofFreedom (6DOF) Pose2.2 ASME Standard:3ASME B89.4.19 Performan
10、ce Evaluation of Laser-BasedSpherical Coordinate Measurement Systems3. Terminology3.1 Definitions:3.1.1 degrees of freedom, DOF, nany of the minimumnumber of translation or rotation components required tospecify completely the pose of a rigid object. E29193.1.1.1 Discussion(1) In a 3D space, a rigid
11、 objects pose can be minimallyrepresented by 6DOF, three translations and three rotations.(2) The term “degrees of freedom” is also used with regardto statistical testing. It will be clear from the context in whichit is used whether the term relates to a statistical test or therotation/translation a
12、spect of the object.3.1.2 frame rate, nfrequency at which a camera acquiresconsecutive images.3.1.3 integration time, nthe length of time when thedigital sensor inside a camera collects light.1This test method is under the jurisdiction of ASTM Committee E57 on 3DImaging Systems and is the direct res
13、ponsibility of Subcommittee E57.02 on TestMethods.Current edition approved Oct. 1, 2017. Published December 2017. DOI:10.1520/E3124-17.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume inf
14、ormation, refer to the standards Document Summary page onthe ASTM website.3Available from American Society of Mechanical Engineers (ASME), ASMEInternational Headquarters, Two Park Ave., New York, NY 10016-5990, http:/www.asme.org.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West
15、 Conshohocken, PA 19428-2959. United StatesThis international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recommendations issued by the World Tra
16、de Organization Technical Barriers to Trade (TBT) Committee.13.1.3.1 DiscussionIn some systems integration time isalso called exposure time.3.1.4 optical image event, nthe instant in time when theOTS registers an event.3.1.4.1 DiscussionIn most systems, it is defined either bythe beginning or the ce
17、nter of the image integration time.3.1.5 optical tracking system, na tracking system that usesmeasurements obtained from camera images. E30643.1.6 physical event, na point in time corresponding to thephysical motion of a target object tracked by the OTS in thetest space (see Fig. 1).3.1.7 physical e
18、vent latency, ntime between an actualoccurrence and OTS report of the corresponding occurrence.3.1.8 pose, na 6DOF vector whose components representthe position and orientation of a rigid object with respect to acoordinate frame. E29193.1.9 precision, nthe closeness of agreement betweenindependent t
19、est results obtained under stipulated conditions.E1773.1.10 rated conditions, nmanufacturer-specified limitson environmental, utility, and other conditions within whichthe manufacturers performance specifications are guaranteedat the time of installation of the instrument. ASME B89.4.193.1.11 standa
20、rd uncertainty, nuncertainty reported as thestandard deviation of the estimated value of the quantitysubject to measurement. E26553.1.12 system latency, nthe elapsed time between theoptical image event and the instant in time when the clientreceives the 6DOF pose information corresponding to thateve
21、nt from the optical tracking system (OTS).3.1.12.1 DiscussionIn Fig. 1, the optical image event timeis marked by the letter “B” and the time of receipt of the posedata by a client system is marked as “D”. OTS computationtime is the time for the completion of the computation of thepose data by the OT
22、S. OTS communication latency is the timebetween the completion of the computation of the pose dataand the receipt of the data by the client.3.1.12.2 DiscussionCertain optical tracking systems willobserve the optical image event some amount of time after thephysical event “A” happens. In the diagram
23、of Fig. 1, OTScomputation time incorporates the vendor-specific image ac-quisition times. As indicated in Section 9, vendors shouldreport the operating parameters of the system includingsystem-wide integration periods and camera frame rates.3.1.13 temporal pose errorthe pose error in time domain.3.1
24、.14 tracking system, na system that is used for mea-suring the pose of moving objects and supplies the data as atimely ordered sequence. E30643.1.15 uncertainty, nan indication of the magnitude oferror associated with a value that takes into account bothsystematic errors and random errors associated
25、 with the mea-surement or test process. E26554. Summary of Test Method4.1 This test method provides a set of statistically-basedperformance metrics and a test procedure to quantitativelymeasure the system latency of an optical tracking system.4.2 Specifically, the test procedure measures the differe
26、ncebetween the time instant when the motion of a test apparatuscauses an electric circuit to be closed and the instant when theoptical tracking system has detected the event. The time instantwhen the electric circuit is closed is measured by a low-latencydata acquisition system which is synchronized
27、 with the opticaltracking system. The performance metrics include the mean,the standard deviation, the maximum, the minimum, andcertain percentiles of latency measurements.5. Significance and Use5.1 Optical tracking systems are used in a wide range offields including: video games, film, neuroscience
28、,biomechanics, flight/medical/industrial training, simulation,robotics, and automotive applications.5.2 This standard provides a common set of metrics and atest procedure for evaluating the performance of opticaltracking systems and may help to drive improvements andinnovations in optical tracking s
29、ystems.45.3 Potential users often have difficulty comparing opticaltracking systems due to the lack of standard performancemetrics and test methods, and must therefore rely on the vendorclaims regarding the systems performance, capabilities, and4“Motion Capture Software Developers in the US: Market
30、Research Report,”IBIS World, 2014.FIG. 1 OTS LatencyE3124 172suitability for a particular application. This standard makes itpossible for a user to assess and compare the performance ofoptical tracking systems, and allows the user to determine if themeasured performance results are within the specif
31、icationswith regard to the application requirements.6. Apparatus6.1 Data Acquisition DeviceFor best results, a low latencyData Acquisition Device (DAQ) should be used. Althoughthere are no requirements on the DAQ, a device which isintegrated into the computer via a PCI Express port has shownacceptab
32、le performance. Other DAQ devices may be used, butthey may incur additional latency and uncertainty to the tests,which may be incorrectly construed as OTS system latency.While PCI Express devices tend to show sub-microsecondlatencies, USB 3.0 and 2.0 devices, for example, have latenciesbetween tens
33、of microseconds to approximately one hundredmicroseconds nad Gigabit Ethernet devices can incur addi-tional latencies on the order of one millisecond.5The usershould also make sure that the operating system does notintroduce significant delays into the measurements and thatthere are no processes run
34、ning in the background during thetest procedure. In general, the user should minimize instrumen-tation related latencies. Standalone measurement devices mayalso be used to avoid these issues as long as they can producetimestamps which are synchronized with those generated bythe OTS.6.2 Hammer Device
35、A device such as a hammer or pen-dulum shall be designed in such a way that it can be releasedand free fall onto a conductive plate (for example, a copperplate or other low-latency devices that can form electricalcontact such as a contact switch). Although there are no strictrequirements on its desi
36、gn, it has been found that a metal barthat rotates around a hinge point is an acceptable design. Abearing to constrain the rotation to a single axis is important toavoid adding noise to the measurements of the device motion.Fig. 2 shows a diagram of a potential apparatus.Apicture of anillustrative d
37、esign is given in Appendix X2. Trackable ele-ments such as markers should be attached to the deviceaccording to the vendor requirements (for example, minimumnumber of markers and marker locations) so that it can beaccurately tracked as a rigid body.6.3 Other MaterialsThis test method also requires a
38、computer (that is, the client system), a solid surface, two wires,and a conductive plate (Fig. 2). Additionally, depending on thespecifications provided by the DAQ manufacturer, a pull-downresistor (that is, resistor to ground connection) or a pull-upresistor (that is, resistor to power supply conne
39、ction) may benecessary. The resistance of the pull-down/pull-up resistorshould be defined according to the DAQ manufacturer recom-mendation (10,000 Ohm is a typical value). To ensure repeat-able contact conditions, the conductive plate should not moverelative to the hard surface on which it is set e
40、ven under theaction of the hammer striking the conductive plate.7. Measurement and Test Procedure7.1 IntroductionThis section describes the basic proce-dure for measuring the latency of an optical tracking system.Latency measurements shall be carried out with a singletrackable object without other o
41、bjects in the background.Optinally, vendors may choose to carry out additional testswith multiple trackable objects, in which case object countshall be included in the report.7.1.1 SetupAttach a wire to the head of the hammerdevice and another wire to a conductive plate. Connect thewires from the ha
42、mmer and the conductive plate to a DAQdevice connected directly into the computer. In the opticaltracking system software make the hammer device a trackableobject and start streaming its positional data.7.1.2 Client ProgramCreate a client program that recordsa “DAQ time” value on the computer when t
43、he hammer devicemakes contact with the conductive plate. This time correspondsto time instant “A” in Fig. 1. Since the hammer might reboundafter initial contact with the conductive plate, the DAQsampling rate should be sufficiently high to capture briefcontact times.Although there are no requirement
44、s for the DAQsampling rate, it has been found that 10,000 samples per5Instrument Bus Performance Making Sense of Competing Bus Technologiesfor Instrument Control, Nov. 2016, http:/ 2 Schematic Diagram of the Suggested ApparatusE3124 173second produces acceptable results. The client program shouldals
45、o record the positional data of the tracked hammer. Define aparameter that will cause the client to measure an “OTS time,”that is, the time instant when the movement of the hammerdevice matches this parameter. As Fig. 3 indicates, the param-eter that triggers the “OTS time” event corresponds to the
46、firstsignificant change of slope of the motion of the hammer (notethat the magnitude of the change of slope will be dependentupon the settings of the OTS and the design of the hammerdevice and that in high frame rate systems, multiple changes ofslope might be observed when the hammer rebounds from i
47、tsinitial impact). The “OTS time” represents time instant “D” inFig. 1. Appendix X3 includes the description of a suggestedalgorithm to detect the change of slope. Appendix X4 showsone example of experimental data.7.1.3 MethodDrop the hammer device on the conductiveplate to complete the circuit. The
48、 hammer should be droppedfrom a height that allows the tracking system to accuratelymeasure its trajectory but that minimizes bouncing effects orpermanent deformations of the conductive plate. Althoughthere are no requirements on the height, it has been found thata height of approximately 0.2 m prod
49、uces acceptable results inseveral tracking systems. Take the difference between the pointin time when the circuit was completed (“DAQ time”) andwhen data was received showing that the optical trackingsystem fulfilled its trigger parameter (“OTS time”). This timedifference will be used to approximate the system latency of anoptical tracking system. The latency corresponds to the timeintervalDBinFig. 1, but since that interval cannot bedirectly measured (other than by the optical tracking systemvendor), the mi
