1、Designation: E1499 16Standard Guide forSelection, Evaluation, and Training of Observers1This standard is issued under the fixed designation E1499; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number
2、in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This guide describes criteria and tests for selecting,evaluating, and training human visual-sensory observers fortasks involving the perceptio
3、n and scaling of properties andphenomena relating to appearance.1.2 Examples of tests requiring the use of trained observersinclude but are not limited to those described in the followingASTM standards: on color, Practice D1535 and PracticeE1360; on color difference, Practice D1729 and Test MethodD2
4、616; on gloss, Test Method D4449; on metamerism, Prac-tice D4086; and on setting tolerances, Practice D3134.1.3 This standard does not purport to address all of thesafety concerns, if any, associated with its use. It is theresponsibility of the user of this standard to establish appro-priate safety
5、and health practices and determine the applica-bility of regulatory limitations prior to use.2. Referenced Documents2.1 ASTM Standards:2D1535 Practice for Specifying Color by the Munsell SystemD1729 Practice for Visual Appraisal of Colors and ColorDifferences of Diffusely-Illuminated Opaque Material
6、sD2616 Test Method for Evaluation of Visual Color Differ-ence With a Gray ScaleD3134 Practice for Establishing Color and Gloss TolerancesD4086 Practice for Visual Evaluation of MetamerismD4449 Test Method for Visual Evaluation of Gloss Differ-ences Between Surfaces of Similar AppearanceE284 Terminol
7、ogy of AppearanceE1360 Practice for Specifying Color by Using the OpticalSociety of America Uniform Color Scales System3. Terminology3.1 DefinitionsDefinitions of appearance terms in Termi-nology E284 are applicable to this guide.3.2 Definitions of Terms Specific to This Standard:3.2.1 appearance, n
8、in psychophysical studies, perceptionin which the spectral and geometric aspects of a visual stimulusare integrated with its illuminating and viewing environment.3.2.2 observer, none who judges visually, qualitatively orquantitatively, the content of one or more appearance attributesin each member o
9、f a set of objects or stimuli.3.2.3 scale, vto assess the content of one or more appear-ance attributes in the members of a set of stimuli.3.2.3.1 DiscussionAlternatively, scales may be deter-mined by assessing the difference in content of an attributewith respect to the differences in that attribut
10、e among themembers of the set.4. Summary of Guide4.1 This guide provides descriptions of techniques and testsfor the selection of candidates for observers for use in visualtesting, for the evaluation of their capabilities in this field, andfor their training to enhance these capabilities.4.2 Discuss
11、ion is provided of precautions required for theefficient use of observers in visual tests, including avoidance ofovertaxing the observers and the control of test variables.4.3 Other considerations of test design, including the num-bers of observers and observations required and the precisionof the v
12、isual results, are to be covered elsewhere.5. Significance and Use5.1 The term appearance (see 3.2.1) implies the essentialpresence of human visual observations. The results of visualobservation involve not only the step of observing, accom-plished by the eye, but also the inseparable step of interp
13、reta-tion in the brain. Instrumental test methods currently cannotduplicate this second step, and therefore can now onlyapproximate, but not fully measure, appearance. Such instru-mental measures of appearance properties are useful only to theextent that they can be correlated to the results of visu
14、alobservations by observers of the appearance phenomena beingevaluated.1This guide is under the jurisdiction of ASTM Committee E12 on Color andAppearance and is the direct responsibility of Subcommittee E12.11 on VisualMethods.Current edition approved July 1, 2016. Published July 2016. Originally ap
15、provedin 1992. Last previous edition approved in 2009 as E1499 97 (2009). DOI:10.1520/E1499-16.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Documen
16、t Summary page onthe ASTM website.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States15.2 Almost invariably, too little attention has been paid toensuring that the essential visual observations have beenproperly obtained to provide the ba
17、sis for correlating visual andinstrumental test results.5.3 This guide provides the means for assessing observers,by outlining the requirements and tests for their selection,evaluation, and training. This guide should be useful to allexperimenters designing or using visual test methods to pro-vide e
18、ither direct results in terms of the observation ofappearance properties, or the experiments correlating suchresults with instrumental measures approximating the sameappearance properties. The user is cautioned to avoid thesubstitution of validated vision tests with replicas of any kind,either print
19、ed, photographed or digitally displayed.6. Selection and Evaluation of Observers6.1 The process used for selecting observers depends a greatdeal upon the type of experiment being carried out, but shouldessentially evaluate the potential capability of the observer toexecute a series of visual evaluat
20、ion tasks (1, 2).3When thesetasks involve appearance attributes, color or related spectralphenomena are often among the task subjects, and if insteadgeometric phenomena such as gloss are involved, many of thesame considerations apply. Accordingly, the emphasis in thisguide is upon selecting observer
21、s for color-related measure-ments. Thus, observers must be screened to rule out those withany color- or task-oriented deficiencies.6.2 Color Vision Tests (3):6.2.1 Pseudoisochromatic PlatesAs a preliminary colorvision test, a pseudoisochromatic-plate test should be admin-istered to the candidate obs
22、ervers. The instructions and scoringtechniques supplied by the manufacturer should be followed.In particular, the illumination level should be kept well withinthe photopic range (1000 1x is recommended as a minimumvalue) and the spectral quality of the illuminating sourceshould be near that of north
23、-sky daylight. Failure to identifycorrectly the required number of the plates in the test usedshould be considered grounds for dismissing the candidateobserver.6.2.2 Farnsworth-Munsell 100 Hue TestThe Farnsworth-Munsell 100 Hue Test(4) should next be administered to thecandidate. While the pseudoiso
24、chromatic-plate tests isolatecertain factors of color deficiency, the Farnsworth-Munsell 100Hue Test measures color discrimination directly and in detail.This test was not designed strictly for pass-fail categorizationof observers but is recommended as an adjunct test for theanalysis of color defect
25、ives. (It is also useful as an observerevaluation test; see 6.3.1.) In the Farnsworth-Munsell 100 HueTest, abnormal color vision is indicated by the observersfailure to place the test chips in correct order. The chips consistof 85 colored papers varying in hue at approximately constantvalue and chro
26、ma, and the observers failure is usually by widemargins in one or more limited regions of the hue circle. Thepresence of such abnormal results of the test should be groundsfor dismissing the candidate observer.6.3 Visual Acuity and Discrimination TestsHaving deter-mined that the candidate observers
27、have normal color vision, itis next necessary to test their level of discrimination of smalldifferences in color or another appearance attribute of interest.6.3.1 Farnsworth-Munsell 100 Hue TestUse of theFarnsworth-Munsell 100 Hue Test as a color-discriminationtest does not require readministration
28、of the test, but merelyreexamination of the test results. For the purposes of assessingcolor (more precisely, hue) discrimination, the test results areexamined for the presence of an approximately constant butsignificant error level in the arrangement of the test chipsthroughout the hue circle. This
29、 may be interpreted as aninability to discriminate the small color differences betweenneighboring chips. While a weakness of this type might, forexample, interfere with an observers ability to participate inthreshold scaling experiments, the observer might still becompetent to perform magnitude scal
30、ing of larger differencesamong specimens.6.3.2 Triangle TestThis test is part of a series known asthe Japanese Color Aptitude Test. The candidate observers areshown, one at a time, a series of 20 sets of three colored chipseach. In each set, two of the chips are identical and the third isslightly di
31、fferent in color. The observer is asked to identifywhich one is different, the differences being so small that thereis considerable uncertainty in the judgment. A lower thanaverage score in this test indicates that the observer does notdifferentiate small differences well.6.4 Magnitude Scaling Tests
32、:6.4.1 Length EstimationA simple magnitude-scaling testmay be devised to familiarize the observer with scalingprocedures and the experimental task of matching a givenanchor scale with a perceived difference in stimuli. In anexample (1), the observer was asked to judge the apparentlength of a line in
33、 comparison to the length of a standard line.The lines were drawn with a heavy black marker on 100 by150-mm index cards, one line to each of 21 cards. The standardor anchor line, 125 mm long, was assigned a value of 10 unitsof length. The other 20 cards had lines of various lengths, bothlonger and s
34、horter than the anchor line. The anchor and one testcard were displayed side by side at a distance of 0.6 m. Ofcourse, no rulers or other aids were allowed. The observerstask was to assign a value to the length of each line relative tothat of 10 units assigned to the anchor. Means of assessing theda
35、ta obtained from a test such as this are discussed by Lodge(5).6.4.2 Color EstimationAnother set of 20 cards from theJapanese Color Aptitude Test may be used to assess thecandidate observers “feel” for the type of judgment required inmagnitude scaling of an appearance attribute. These cards eachcont
36、ain three color chips in a horizontal row. The left-hand chipis identified with the value 1, and the right-hand chip with thevalue 10. The chip in the middle lies between these two endson some color-attribute scale. The task is to assign a scale valuebetween 1 and 10 to the center chip. The color-at
37、tribute scalesof hue, value, and chroma are used randomly in the set.6.5 The results of the above magnitude-scaling tests shouldbe compared to the observers performance on the color-discrimination tests, particularly the Farnsworth-Munsell 1003The boldface numbers in parentheses refer to the list of
38、 references at the end ofthis standard.E1499 162Hue Test, to determine how observers with superior to normalcolor vision perform. Any significant disparities should beexamined to see if they resulted from poor directions orimproper viewing conditions. If this was not the case, theobserver should be
39、asked to repeat the judgments at anothertime. If the results are poor a second time, the conclusion maybe drawn that the candidate does not have a substantial skill forthis type of judgment, and this observer should be dismissed.7. Training of Observers7.1 The importance of following the steps of se
40、lection andevaluation of observers by training sessions using the exactconditions of the scaling experiment in accordance withSection 6 cannot be overemphasized. The details of thesetraining sessions will depend on the experimental design beingused and cannot be stated in general terms here. Example
41、s arefound in Refs (1) and (2). Only by training the observer underthe actual experimental conditions to be used can that observerlearn exactly what is expected in the task.7.1.1 It is recommended that each observer make a dummyset of observations before each observing session. This hasseveral advan
42、tages. The first few observations always exhibitmore “noise,” as the observer refamiliarizes himself with thetask. Use of the same sample set for each of these preliminaryobservation sets allows consistency to be tested and puts theobserver at ease. At least one such set of observations shouldbe one
43、 from which the results are discarded. The experimentershould watch the observer during the first session to be surethat the instructions for the experiment are understood.7.1.2 There appears to be virtually no information reportedin Refs (1 and 2), (6-9) about several important aspects ofobserver t
44、raining, such as how long the training sessions needto be, when they should occur, whether they should be repeatedand at what intervals, and how the experimenter knows whenthe observer is adequately trained. A few comments on thenumber of observers required in a typical case are given in Ref(6), but
45、 this should be considered as part of the experimentaldesign.7.1.3 Specific comments on magnitude scaling are found in6.4.8. Precautions in Using Observers8.1 The report (7) of a conference on color-discriminationpsychophysics points out several areas in which care must betaken to provide optimum wo
46、rking conditions for the observersif their visual scaling results are to be reliable. Referenceshould also be made to the requirements for observers for othersensory testing procedures (8).8.1.1 It is all too easy to overtax the capabilities of observersto make critical judgments over long periods o
47、f time. Sessionsshould be limited in length of time or number of observations,or both. It is not yet possible to place firm limits on thesenumbers or durations. Boredom as well as physical fatiguemust be considered. Some experimenters suggest rewardingobservers for consistent results in order to com
48、bat boredom andensure attention and responsiveness. In any case, the perfor-mance of the observers should be reviewed to ensure that theirlevel of accuracy does not degrade with increase in length ofthe observing session.8.2 Care must be taken in the design of the experiments toensure that all varia
49、bles that might influence the results ormake the visual judgments more difficult are controlled. Someparameters requiring control that might not be immediatelyobvious are the sizes of the test specimens (see, for example,Practice D1729), their proximity, and the nature of any borderor dividing line between them; the nature and level of theillumination; the nature and luminance of the surround; and theabsence of any distraction in the observers field of vision, suchas light reflected from glossy surfaces or brightly coloredclothing.8.2.1 The optimum levels of illum
