SAE J 2450-2005 Translation Quality Metric《翻译质量衡量标准》.pdf

1、 SURFACE VEHICLE STANDARD Translation Quality Metric SAE Technical Standards Board Rules provide that: This report is published by SAE to advance the state of technical and engineering sciences. The use of this report is entirely voluntary, and its applicability and suitability for any particular us

2、e, including any patent infringement arising therefrom, is the sole responsibility of the user. SAE reviews each technical report at least every five years at which time it may be reaffirmed, revised, or cancelled. SAE invites your written comments and suggestions. Copyright 2005 SAE International A

3、ll rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE. TO PLACE A DOCUMENT ORDER: Tel: 877-606-7323 (insid

4、e USA and Canada) Tel: 724-776-4970 (outside USA) Fax: 724-776-0790 Email: custsvcsae.org SAE WEB ADDRESS: http:/www.sae.org Issued 2001-12 Revised 2005-08 Superseding J2450 DEC2001 J2450 REV. AUG2005Foreword In the past, quality measurement of language translation in the automotive industry was som

5、ewhat subjective, if such measurement was undertaken at all. Cost and effort go into obtaining translations from suppliers, the same as for any manufactured product. It is necessary to have a tangible method for measuring the quality of translation deliverables as precisely as for any manufactured p

6、roduct. The objective of the proposed quality metric is to establish a consistent standard against which the quality of translation of automotive service information can be objectively measured regardless of the source language, regardless of the target language, and regardless of how the translatio

7、n is performed (i.e., human translation or machine translation). The metric allows an evaluator to tag errors in a translation and compute a weighted, numeric score that represents the quality of the translation. The metric consists of four parts: Seven error categories Two sub-categories Two meta-r

8、ules to help decide ambiguities on the assignment of an error to the categories and sub-categories Numeric weights Consistent use of a metric across the industry will allow tighter control of translation quality. The risks of low-quality translations of service information include erosion of custome

9、r confidence, higher warranty costs, and (at an extreme) damage to vehicles or injury to people. Copyright SAE International Provided by IHS under license with SAENot for ResaleNo reproduction or networking permitted without license from IHS-,-,-SAE J2450 Revised AUG2005 - 2 - TABLE OF CONTENTS 1. S

10、cope . 2 1.1 Rationale 2 2. References. 3 3. How To Use This Document 3 4. Technical Requirements (Content Of The Metric) . 3 4.1 General Comments 3 4.2 Guidelines for Evaluators. 5 4.3 Error Categories. 7 4.3.1 Wrong Term . 7 4.3.2 Wrong Meaning 8 4.3.3 Omission 8 4.3.4 Structural Error. 9 4.3.5 Mi

11、sspelling . 10 4.3.6 Punctuation Error . 10 4.3.7 Miscellaneous Error . 11 5. Notes 12 5.1 Marginal Indicia 12 Appendix A Translation Metric Score Sheet 13 Appendix B Quick Reference Guide 14 1. Scope This SAE Standard is applicable to translations of automotive service information into any target l

12、anguage. The metric may be applied regardless of the source language or the method of translation (i.e., human translation, computer assisted translation or machine translation). Note that the current version of the metric does not measure errors in style, making it unsuitable for evaluations of mat

13、erial in which style is important (e.g., owners manuals or marketing literature). The metric can be expanded to accommodate style and other requirements of particular new media. 1.1 Rationale Up to now, quality measurement on language translation in the automotive industry has largely been subjectiv

14、e, if such measurement was undertaken at all. If an automotive company did set up a quality process with its translation suppliers, the quality of translated service information would generally be reviewed by in-country validators designated by the automotive company. Markups of the translated docum

15、ents were provided back to the translation supplier for correction and editing. There would likely not be any standardized measurement metrics for determining or rating quality in a manner similar to methods used in the manufacturing side of the automotive business. Copyright SAE International Provi

16、ded by IHS under license with SAENot for ResaleNo reproduction or networking permitted without license from IHS-,-,-SAE J2450 Revised AUG2005 - 3 - The objective of the metric is to establish a consistent standard against which the quality of translation of automotive service information can be obje

17、ctively measured a. Regardless of the source language, b. regardless of the target language, c. and regardless of how the translation is performed-i.e., human translation, computer assisted translation or machine translation 2. References 2.1 Applicable Publication The following publication forms a

18、part of this specification to the extent specified herein. Unless otherwise indicated, the latest version of SAE publications shall apply. 2.1.1 SAE PUBLICATION Available from SAE, 400 Commonwealth Drive, Warrendale, PA 15096-0001. SAE J2450 Training Document 3. How to Use this Document The General

19、Comments section introduces the correct use of the metric. Guidelines for Evaluators indicates scoring practices. The error categories are defined in the next section. 4. Technical Requirements (content of the metric) 4.1 General Comments The SAE J2450 metric should be regarded as only one element i

20、n a total Quality Assurance process, albeit an important one. This document defines SAE J2450 and should be regarded as a reference document, not a training document. The task force was chartered to create a metric, and it is up to users of that metric to determine how to train quality evaluators in

21、 the use of the metric. We recommend that assessors be trained in the use of this document before the implementation of any quality assessment based on SAE J2450. The SAE J2450 Training Document is a supplement to this definition document; it is meant to provide both OEMs and translation suppliers w

22、ith some methods for integrating SAE J2450 into their business practices as well as basic training in the actual scoring of documents. In using SAE J2450 each error found by the evaluator should be marked in two ways. First, it should be classified into one of the seven categories described below, e

23、.g., wrong term. After its primary category has been identified, the evaluator should indicate if it is a serious or a minor error, according to how severe the evaluator considers the error. Both the primary (category) and the secondary (serious/minor) classification are judgment calls by the evalua

24、tor. Copyright SAE International Provided by IHS under license with SAENot for ResaleNo reproduction or networking permitted without license from IHS-,-,-SAE J2450 Revised AUG2005 - 4 - While reasonably complete definitions are given below to help the evaluator identify the major categories with som

25、e consistency, there may yet be room for ambiguity and the final category selection may, in fact, be arbitrary. For example, the English word replace may be translated into French as either replacer (to put back into place) or remplacer (to replace with another item), according to the context of use

26、. That is, the English verb replace is ambiguous with respect to French. Suppose that an English text describes a situation where a seat belt must be replaced in the event of an accident because the belt may be weakened by the first accident and fail in a subsequent accident. If a French translation

27、 then uses the word replacer, an evaluator should mark this verb as an error. However, is it a wrong meaning error, a misspelling error or a miscellaneous error? Without talking to the original translator, it is fundamentally impossible for an evaluator to know if the translator mistakenly thought t

28、hat replacer and not remplacer as the correct verb, or whether the translator simply mistyped and accidentally omitted the m. In such cases where the primary category classification is ambiguous and there is no evidence as to which category an error belongs, then an evaluator should place it into th

29、e category that appears the earliest in the following list. This is completely arbitrary, but if all evaluators follow this simple rule, then greater consistency of classification across evaluators will result. The secondary classification of an error as either serious or minor is always a judgment

30、call by the evaluator, and necessarily so. While it is virtually impossible to define the notion of a serious or minor error, as a general guideline if an error is clearly serious in its consequences for a technician or its effect on the meaning of the translation, then it should be marked as seriou

31、s. If not, then it should be sub-classified as minor. It is a natural and expected consequence that there will be cases when an evaluator hesitates in this choice. Whenever an evaluator hesitates, let us take it again as a general guideline to sub-classify the error as serious. This again is complet

32、ely arbitrary, but it seems the safer alternative since safety may be an issue in a service text. Once the primary and secondary classifications have been assigned, then a numeric weight between 1 and 5 is looked up in the definition and assigned to the error. 5 corresponds to the most severe error,

33、 while a 1 indicates an error with relatively small consequences. This assignment of weights is a mechanical process and there is no room for judgment. While sometimes this assignment of numeric weights will over-value the severity of an error, it will under-value it at other times. The underlying a

34、ssumption of SAE J2450 is that these deviations will tend to cancel each other, i.e., regression to the mean will normalize these deviations on a large evaluation text or with a large number of evaluation texts. The SAE J2450 categories are meant to be used to tag errors that are linguistic in natur

35、e. Thus, a translation deliverable that is free of any SAE J2450 errors may still be unacceptable to the client due to other problems, such as formatting errors. Also, SAE J2450 is designed only for the evaluation of translations of service information where the style, tone, register etc., of the ta

36、rget language deliverable is not as important to the client as it may be for owners manuals or marketing literature. The target customer of the translation is the service technician. Therefore, SAE J2450 deliberately avoids tagging errors of style. The definitions as follows are generally syntactic

37、in nature. That is, they depend upon the surface form of the translation deliverable and are generally divorced from the meaning. However, meaning is accommodated in the notion of a serious versus a minor occurrence of an error type. Copyright SAE International Provided by IHS under license with SAE

38、Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-SAE J2450 Revised AUG2005 - 5 - The general concept is that once an error has been identified by category, the evaluator decides whether the effect on the translation of that error is severe or not. If severe, then th

39、e error is classified as a serious occurrence of that error type, and otherwise it is regarded as a minor occurrence. This is necessarily a judgment call by the evaluator. A simple example will clarify. If a value is represented in a target language as 1.6 liters rather than the correct 1,6 liters,

40、the technician reading the translation is not likely to be confused. This error would be marked punctuation error, minor. However, if the value is represented as 16 liters, the translation should clearly be marked as punctuation error, serious. Care should be taken by any evaluator in the applicatio

41、n of these SAE J2450 categories NOT to confuse an error with the cause of that error. That is, a translation deliverable may contain a token of a wrong term error (or an error of any other type) because the translator made a mistake or because there is an error or ambiguity in the source language te

42、xt. Errors in the target text that are caused by source language errors or ambiguities are still errors in the target text. Any contractual or working relationships between clients and suppliers should recognize this distinction, and not penalize suppliers for errors that are caused by the clients s

43、ource documents, even though there is an SAE J2450 error in the target text. Again: Do not confuse the existence of an error with its cause. SAE J2450 is only concerned with the existence of an error, regardless of the reason for that error. Causal analysis should be performed on all errors, accordi

44、ng to the needs of the client. Finally, although some translators and evaluators may be accustomed to working with a category called mistranslation, the SAE J2450 metric has no such category. Any error in a translation is regarded as a mistranslation, potentially affecting the meaning of the target

45、text. The first six categories below attempt to identify which kind of a mistranslation an error is, and the seventh (miscellaneous) is a catch-all for when the evaluator knows something is an error, yet cannot categorize it easily as any of the first six types. The remainder of this document will d

46、efine SAE J2450, but the reader should keep in mind that the metric itself consists of four elements. They are: a. The seven primary error categories, b. The two secondary subcategories (i.e., serious and minor), c. The two meta-rules: 1. When an error is ambiguous, always choose the earliest primar

47、y category. 2. When in doubt, always choose serious over minor. d. The numeric weights. 4.2 Guidelines for Evaluators An evaluator should be thoroughly familiar with the definitions of the seven categories that follow, as well as the general comments in the preceding section. SAE J2450 offers no gui

48、delines as to how to select texts for evaluation. Texts may be evaluated in part or in whole, according to the desires of the client organization. If in part, texts may be selected at random or in some predetermined manner. Similarly, SAE J2450 does not suggest how to use the results of an evaluatio

49、n. This includes the acceptance threshold for a numeric score associated with an evaluation. Scores will be normalized by dividing by the number of words in a sample, but SAE J2450 does not attempt to define what any client finds acceptable. There will be as many methods to employ SAE J2450 as there are client-supplier contracts for producing translations. Copyright SAE International Provided by IHS under license with SAEN