1、 ISO 2013 Road vehicles Objective rating metrics for dynamic systems Vhicules routiers Mesures pour lvaluation objective des systmes dynamiques TECHNICAL REPORT ISO/TR 16250 First edition 2013-07-15 Reference number ISO/TR 16250:2013(E) ISO/TR 16250:2013(E)ii ISO 2013 All rights reserved COPYRIGHT P
2、ROTECTED DOCUMENT ISO 2013 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior written permission.
3、Permission can be requested from either ISO at the address below or ISOs member body in the country of the requester. ISO copyright office Case postale 56 CH-1211 Geneva 20 Tel. + 41 22 749 01 11 Fax + 41 22 749 09 47 E-mail copyrightiso.org Web www.iso.org Published in Switzerland ISO/TR 16250:2013
4、(E) ISO 2013 All rights reserved iii Contents Page Foreword iv Introduction v 1 Scope . 1 2 T erms and definitions . 1 3 Symbols and abbreviated terms . 1 3.1 General abbreviated terms . 1 3.2 General symbols and subscripts 2 3.3 CORA 2 3.4 EARTH and EEARTH . 3 3.5 Model reliability metric . 4 3.6 B
5、ayesian confidence metric . 5 3.7 Overall ISO rating 5 4 General requirements to the data 5 5 CORA metric 6 5.1 Corridor rating 6 5.2 Cross-correlation rating 8 5.3 Step-by-step procedure 10 6 EARTH metric 11 6.1 EARTH phase score 12 6.2 EARTH magnitude score 13 6.3 EARTH slope score 14 6.4 Overall
6、EARTH score .15 6.5 Step-by-step procedure 15 7 Model reliability metric .16 8 Ba y esian c onfidenc e metric .16 9 ISO metric .18 9.1 CORA corridor method .18 9.2 EEARTH method .18 9.3 Calculation of the overall ISO rating 23 9.4 Meaning of the objective rating score 24 10 Pre-processing of the dat
7、a .24 10.1 Sampling rate.25 10.2 Filtering 25 10.3 Interval of evaluation .25 11 Limitations .26 11.1 Type of signals 26 11.2 Metrics validation 26 11.3 Meaning of the results .26 11.4 Multiple responses .27 Annex A (informative) Child restraint example .28 Annex B (informative) Sled test example 46
8、 Annex C (informative) Case studies 51 Bibliography .65 ISO/TR 16250:2013(E) Foreword ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO
9、 technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates cl
10、osely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization. The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular the different approval criteri
11、a needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2. www.iso.org/directives Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
12、rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any patent rights identified during the development of the document will be in the Introduction and/or on the ISO list of patent declarations received. www.iso.org/patents Any trade name used in this
13、document is information given for the convenience of users and does not constitute an endorsement. The committee responsible for this document is ISO/TC 22, Road vehicles, Subcommittee SC 10, Impact test procedures, and SC 12, Passive safety crash protection systems.iv ISO 2013 All rights reserved I
14、SO/TR 16250:2013(E) Introduction Computer-Aided Engineering (CAE) has become a vital tool for product development in the automobile industry. Various computer programs and models are developed to simulate dynamic systems. To maximize the use of these models, their validity and predictive capabilitie
15、s need to be assessed quantitatively. Model validation is the process of comparing CAE model outputs with test measurements in order to assess the validity or predictive capabilities of the CAE model for its intended usage. The fundamental concepts and terminology of model validation have been estab
16、lished mainly by standard committees including the United States Department of Energy (DOE), 6the American Institute of Aeronautics and Astronautics (AIAA), 1the Defense Modeling and Simulation Office (DMSO) of the US Department of Defense (DOD), 5the American Society of Mechanical Engineers Standar
17、ds Committee (ASME) on verification and validation of Computational Solid Mechanics, 2Computational Fluid Dynamics and Heat Transfer, 3and various other professional societies. 42223 One of the critical tasks to achieve quantitative assessment of models is to develop a validation metric that has the
18、 desirable metric properties to quantify the discrepancy between functional or time history responses from both physical test and simulation result of a dynamic system. 71920Developing quantitative model validation methods has attracted considerable researchers interest in recent years. 121314182021
19、26282932However, the primary consideration in the selection of an effective metric should be based on the application requirements. In general, the validation metric is a quantitative measurement of the degree of agreement between the physical test and simulation result. In this Technical Report, fo
20、ur state-of-the-art objective rating metrics are investigated and they are: CORrelation and Analysis (CORA) metric, 103031Error Assessment of Response Time Histories (EARTH) metric, 2834model reliability metric, 182735and Bayesian confidence metric. 141636Multiple dynamic system examples for both te
21、sts and CAE models are used to show their advantages and limitations. Further enhancements of the CORA corridor rating and the development of an Enhanced Error Assessment of Response Time Histories (EEARTH) metric are proposed to improve the robustness of these metrics. A new combined objective rati
22、ng metric is developed to standardize the calculation of the correlation between two time history signals of dynamic systems. Multiple vehicle safety case studies are used to demonstrate the effectiveness and usefulness of the proposed metric for an ISO Technical Report. ISO 2013 All rights reserved
23、 v Road vehicles Objective rating metrics for dynamic systems 1 Scope This Technical Report specifies a method to calculate the level of correlation between two non-ambiguous signals. The focus of the methods described in this Technical Report is on the comparison of time-history signals or function
24、al responses obtained in all kinds of tests of the passive safety of vehicles and the corresponding numerical simulations. It is validated with signals of various kinds of physical loads such as forces, moments, accelerations, velocities, and displacements. However, other applications might be possi
25、ble too, but are not in the scope of this Technical Report. 2 T erms a nd definiti ons For the purposes of this document, the following terms and definitions apply. 2.1 f i lt er i n g smoothing of signals by using standardized algorithms 2.2 goodness or level of correlation similarity of two signal
26、s 2.3 interval of evaluation time domain that is used to calculate the correlation between two signals 2.4 rating rating score calculated value that represents a certain level of correlation (objective rating) 2.5 sampling rate recording frequency of a signal 2.6 time sample pair values (e.g. time a
27、nd amplitude) of a recorded signal 2.7 time-history signal physical value recorded in a time domain; those signals are non-ambiguous 3 Symbols and abbreviated terms 3.1 General abbreviated terms CAE Computer-Aided Engineering CORA CORrelation and Analysis DTW Dynamic Time Warping TECHNICAL REPORT IS
28、O/TR 16250:2013(E) ISO 2013 All rights reserved 1 ISO/TR 16250:2013(E) EARTH Error Assessment of Response Time Histories EEARTH Enhanced Error Assessment of Response Time Histories SME Subject Matter Expert 3.2 General symbols and subscripts C, C(t) analysed signal (CAE signal) T, T(t) reference sig
29、nal (test signal) t time signal (axis of abscissa) t interval between two time samples t 0time zero of an event (e.g. test, crash, impact, etc.) starting time of the interval of evaluation ending time of the interval of evaluation N total number of sample points (e.g. time steps) between the startin
30、g time, , and ending time, 0all natural numbers without zero 3.3 CORA CORA rating Z 1corridor rating corridor rating at time t (curve) Z 2cross-correlation rating 2phase-shift rating 2size rating 2shape (progression) rating 1weighting factor of the corridor rating, Z 1 2weighting factor of the cross
31、-correlation rating, Z 2 2weighting factor of the phase-shift rating, 2 2weighting factor of the size rating, 2 2weighting factor of the shape rating, 2 1exponent factor for calculating the corridor rating between the inner and outer corridors 2exponent factor for calculating phase-shift rating, 2 2
32、exponent factor for calculating size rating, 2 2exponent factor for calculating shape rating, 2 absolute maximum amplitude of the reference signal, T a 0relative half width of the inner corridor2 ISO 2013 All rights reserved ISO/TR 16250:2013(E) b 0relative half width of the outer corridor ihalf wid
33、th of the inner corridor ohalf width of the outer corridor lower/upper inner corridor at time t (curve) lower/upper outer corridor at time t (curve) D coefficient of the allowable lower limit of the phase shift D coefficient of the allowable upper limit of the phase shift sum of the square of the ar
34、ea for the time-shifted evaluated curve, C sum of the square of the area for the reference curve, T percentage of the minimum remaining overlapping time of the reference and evaluation curves after time shift m shift of a signal along the axis of abscissa m minimum m shift of a signal m maximum m sh
35、ift of a signal n number of samples n time step shifted to get the maximum cross correlation cross correlation m cross correlation at shift, m phase-shift time at the maximum cross correlation, lower limit of CORA phase shift upper limit of CORA phase shift 3.4 EARTH and EEARTH E Eoverall EARTH scor
36、e EARTH magnitude score EARTH phase score EARTH slope (topology) score weighting factor of the magnitude score, weighting factor of the phase score, weighting factor of the slope score, exponent factor for calculating the magnitude score, exponent factor for calculating the phase score, exponent fac
37、tor for calculating the slope score, EARTH magnitude error ISO 2013 All rights reserved 3 ISO/TR 16250:2013(E) EARTH slope error M *maximum allowable magnitude error P *maximum allowable percentage of time shift S *maximum allowable slope error mean value of CAE curve , truncated and shifted CAE cur
38、ve derivative CAE curve, warped CAE curve, derivative warped CAE curve, mean value of test curve , truncated and shifted test curve derivative test curve, warped test curve, derivative warped test curve, Emaximum cross correlation of all and cross correlation signal is moved to the left cross correl
39、ation signal is moved to the right local cost function to perform the dynamic time warping m time steps moved to evaluate the EARTH phase error n number of time shifts to get, E 3.5 Model reliability metric absolute maximum amplitude of the reference signal, T a reliability target b threshold factor
40、 of the reliability assessment Llower bound of the threshold interval Uupper bound of the threshold interval Llower bound of the Bayesian interval hypothesis in probabilistic principal component analysis space (PPCA) Ulower bound of the Bayesian interval hypothesis in PPCA space r model reliability
41、P cumulative probability4 ISO 2013 All rights reserved ISO/TR 16250:2013(E) , t reduced data matrix 3.6 Ba y esian c onfidenc e metric A constant vector Bayes factor for multivariate case likelihood function predefined threshold vector H 0null hypothesis H 1alternative hypothesis K confidence of acc
42、epting the model measure of confidence within an interval variance of error variable, i * p mean values obtained from * N normal distribution N normal distribution of with mean vector, p , and variance matrix, 0prior probability of hypothesis prior mean, variance matrix of * , t difference curve bet
43、ween test curve, T , and CAE curve, C f prior density function of 3.7 Overall ISO rating R combined rating of EEARTH and the CORA corridor method E EEARTH rating score Z CORA corridor rating ( ZZ 1 ) weighting factor of the EEARTH rating, E weighting factor of the CORA corridor rating, Z r rank of t
44、he sliding scale of the ISO metric lower threshold of rank, r upper threshold of rank, r 4 General requirements to the data The metrics described in this Technical Report require non-ambiguous curves (e.g. time-history curves). Furthermore, it is required that the reference curve, T(t), and the eval
45、uated curve, C(t), are both defined between starting time, , and ending time, . Both curves shall have the same number of sample points, N, with a constant time interval, t, within the evaluation interval. ISO 2013 All rights reserved 5 ISO/TR 16250:2013(E) 5 CORA metric The objective evaluation met
46、ric called CORA correlation and analysis 103031 uses two independent sub-ratings, a corridor rating, and a cross-correlation rating to assess the correlation of two signals. The rating structure of CORA is shown in Figure 1. Figure 1 CORA rating structure The corridor and cross-correlation ratings a
47、re used to compensate each others disadvantages, and the CORA rating tool is trying to separate an engineers knowledge from the objective rating metric by using external parameters. However, it is possible to fine-tune the evaluation to the specific needs of the applications by adjusting those metric parameters to reflect the SMEs knowledge of the applications. The corridor rating, Z 1 , calculates the deviation between both curves with the help of user-defined or automatically generated corrid
