ASTM E2263-2012(2018) Standard Test Method for Paired Preference Test.pdf

1、Designation: E2263 12 (Reapproved 2018)Standard Test Method forPaired Preference Test1This standard is issued under the fixed designation E2263; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in

2、 parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This document covers a procedure for determiningpreference between two products using either a two-alternativeforced-choice task, or with the op

3、tion of choosing no prefer-ence. Preference testing is a type of hedonic testing.1.2 A paired preference test determines whether there is astatistically significant preference between two products for agiven population of respondents. The target population must becarefully considered.1.3 This method

4、 establishes preference in a single evalua-tion context. Replicated tests will not be covered within thescope of this document.1.4 Paired preference testing can address overall preferenceor preference for a specified sensory attribute.1.5 The method does not directly determine the magnitudeof prefer

5、ence.1.6 This method does not address whether or not twosamples are perceived as different. Refer to Test Method E2164for directional difference test.1.7 Apaired preference test is a simple task for respondents,and can be used with populations that have minimal reading orcomprehension skills, or bot

6、h.1.8 Preference is not an intrinsic attribute of the product,such as hue is, but is a subjective measure relating torespondents affective or hedonic response. It differs frompaired comparison testing which measures objective character-istics of the product. Preference results are always dependenton

7、 the population sampled.1.9 This standard does not purport to address all of thesafety problems associated with its use, when testing includeshazardous materials, operations, or equipment. It is the re-sponsibility of the user of this standard to establish appropriatesafety, health, and environmenta

8、l practices and to determinethe applicability of regulatory limitations prior to use.1.10 This international standard was developed in accor-dance with internationally recognized principles on standard-ization established in the Decision on Principles for theDevelopment of International Standards, G

9、uides and Recom-mendations issued by the World Trade Organization TechnicalBarriers to Trade (TBT) Committee.2. Referenced Documents2.1 ASTM Standards:2E253 Terminology Relating to Sensory Evaluation of Mate-rials and ProductsE456 Terminology Relating to Quality and StatisticsE1871 Guide for Serving

10、 Protocol for Sensory Evaluation ofFoods and BeveragesE1958 Guide for Sensory Claim SubstantiationE2164 Test Method for Directional Difference Test2.2 ISO Standard:ISO 5495 Sensory AnalysisMethodologyPaired Com-parison33. Terminology3.1 For definition of terms relating to sensory analysis, seeTermin

11、ology E253, and for terms relating to statistics, seeTerminology E456.3.2 Definitions of Terms Specific to This Standard:3.2.1 (alpha) riskthe probability of concluding that apreference exists when, in reality, one does not. (Also knownas Type I Error or significance level.)3.2.2 (beta) riskthe prob

12、ability of concluding that nopreference exists when, in reality, one does. (Also known asType II Error.)3.2.3 common responsesfor a one-sided test, the numberof respondents selecting the product that is expected to bepreferred. For a two-sided test, the largest number of respon-dents selecting eithe

13、r product.3.2.4 one-sided testa test in which the researcher has an apriori assumption concerning the direction of the preference.1This test method is under the jurisdiction of ASTM Committee E18 on SensoryEvaluation and is the direct responsibility of Subcommittee E18.04 on Fundamen-tals of Sensory

14、.Current edition approved Aug. 1, 2018. Published August 2018. Originallyapproved in 2004. Last previous edition approved in 2012 as E2263 12. DOI:10.1520/E2263-12R18.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual

15、Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.3Available from American National Standards Institute (ANSI), 25 W. 43rd St.,4th Floor, New York, NY 10036.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA

16、 19428-2959. United StatesThis international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recommendations issued by the World Trade Organization T

17、echnical Barriers to Trade (TBT) Committee.1In this case, the alternative hypothesis will express that aspecific product is preferred over another product (that is only,A B or A 65 % represents “large” values.8.2 Having defined the required sensitivity for the test using8.1, use Table X1.1 to determ

18、ine the number of respondentsnecessary for a one-sided test, or Table X2.1 to determine thenumber of respondents necessary for two-sided test. Select thesection of the table corresponding to the selected Pmaxvalueand the column corresponding to the selected value. Theminimum required number of respo

19、ndents is found in the rowcorresponding to the selected value of . Alternatively, TableX1.1 can be used to develop a set of values for Pmax, , and that provide acceptable sensitivity while maintaining the num-ber of respondents within practical limits.8.2.1 Using the parameters: = 0.05, = 0.20, and

20、Pmax=60 %, the researcher would use the section of Table X1.1corresponding to Pmax= 60 % and the column corresponding to = 0.20. In the row corresponding to = 0.05, it is found that158 respondents will be needed for the test.8.3 Often in practice, the number of respondents is deter-mined by project

21、constraints (for example, duration of theexperiment, number of respondents available, quantity ofsample, budgetary constraints). The power of the test shouldthen be computed. For this purpose, the following parametersneed to be defined: , observed Pmax, and the number ofrespondents, n. The observed

22、Pmaxcorresponds to the observedproportion of common responses, n is determined by the testrealization, and should be fixed by the experimenter prior tothe test being conducted. With this information, an exact powercomputation can be achieved using appropriate software.However, an approximate value c

23、an already be inferred byreverse lookup using Table X1.1 or Table X2.1, depending onwhether the alternative is one- or two-sided. First, use the valueof Pmaxclosest to the observed one to select a group of rows,then select among these rows the one corresponding to theselected value of . Finally, sel

24、ect the cell having the numberof assessors closest to the actual number of assessors. The4Herz, R. S. and Cupchik, G. C., “An Experimental Characterization ofOdor-evoked Memories in Humans,” Chemical Senses, Vol 17, No. 5, 1992, pp.519-528.5Todrank, J., Wysocki, C. J., and Beauchamp, G. K., “The Eff

25、ects of Adaptationon the Perception of Similar and Dissimilar Odors,” Chemical Senses, Vol 16, No.5, 1991, pp. 476-482.E2263 12 (2018)3corresponding column heading will give a close estimate of theactual power of the test (1-). Lower sample sizes will reducethe power of the test.9. Procedure9.1 Pair

26、ed preference can be used in either CLT (CentralLocation Test) or IHUT (in-home use test) designs. Thefollowing discussion focuses on CLT testing procedures,however, randomizations and data analyses would be similarfor IHUTs.9.2 Prepare serving order worksheet and ballot in advanceof the test to ens

27、ure a balanced order of presentation of the twosamples. Balance the serving sequences of the samples (ABand BA) across all respondents. Serving order worksheetsshould also include complete sample identification informationeither by product name or coded reference for double blindstudies. See Appendi

28、x X1.9.3 It is critical to the validity of the test that respondentscannot differentiate the samples based on the way they arepresented. For example, in a test evaluating flavor differences,one should avoid any subtle differences in temperature orappearance caused by factors such as the time sequenc

29、e ofpreparation. Code the vessels containing the samples in auniform manner, using three digit numbers chosen at randomfor each test. Prepare samples out of sight and in an identicalmanner: same apparatus, same vessels, same quantities ofsample (see Practice E1871, ASTM Serving Protocols).9.4 Presen

30、t the pair of samples simultaneously if possible,following the same spatial arrangement for each assessor (on aline to be sampled always from left to right, or from front toback, etc.). Respondents are typically allowed to evaluate eachsample more than once. If the conditions of the samples restrict

31、reevaluating the samples (for example, if samples are bulky,leave an aftertaste, or show slight differences in appearancethat cannot be masked), present the samples sequentially anddo not allow repeated evaluations.9.5 It is not recommended that more than the preferencequestion be asked about the sa

32、mples, because the selection therespondent has made on the initial question may bias theresponse to subsequent questions. Responses to additionalquestions may be obtained through separate tests foracceptance, degree of difference, etc. See Manual 266.Asection soliciting open-ended comments may be in

33、cludedfollowing the initial preference question.9.6 The paired preference test can either be forced-choice orhave the option of no preference.9.6.1 When using the paired preference test as a forced-choice procedure, respondents are not allowed the option ofreporting “no preference.”Arespondent who h

34、as no preferencefor either of the samples should be instructed to randomlyselect one of the samples, and can indicate in the commentssection that they had no preference.10. Analysis and Interpretation of Results10.1 The procedure used to analyze the results of a pairedpreference test depends on whet

35、her or not a “no preference”option is allowed.10.1.1 If a forced choice procedure is used, analyze asdetailed in 10.2.10.1.2 If a “no preference” option is allowed, then there arevarious ways to handle the data depending on the test objec-tives. Typically the no preference data is split in some mann

36、erbetween “A” and “B.” Regardless of how the no preferencedata are handled, it is always important to report the percentageof no preference responses and take those into account for yourfinal action steps. (Refer to Guide E1958 for decision rulesregarding handling of no preference votes and specific

37、 claims.)10.2 Analysis for PreferenceDifferent analyses are useddepending on whether the number of respondents is equal to orgreater than planned or fewer than planned.10.2.1 If the actual number of respondents is equal to orgreater than planned, refer to Table X1.2 (one-sided) or TableX2.2 (two-sid

38、ed) to analyze the data. If the number of commonresponses is equal to or greater than the number given in thetable, conclude that there is a preference between the products.If the number of common responses is fewer than the numbergiven in the table, conclude that there is no preference. Theconclusi

39、ons, “preference” or “no preference,” are based on thepredetermined , , and Pmaxlevels.10.2.2 When the number of respondents is fewer thanplanned, then the data analysis is the same as 10.2.1 above.Understand that the -risk is now larger than the value chosenbecause a smaller number of respondents p

40、articipated in thetest. A result of “no preference” becomes more likely as Ndecreases.10.3 Analysis for ParityDifferent analyses are used de-pending on whether the number of respondents is equal to orgreater than planned or fewer than planned. There is a directrelationship between sample size (N) an

41、d test sensitivity inparity testing.10.3.1 When the actual number of respondents is equal to orgreater than planned, then the analysis is conducted as outlinedin 10.2.1.10.3.2 When the number of respondents is fewer thanplanned, then data analysis consists of calculating a confidenceinterval. A conf

42、idence interval is calculated because the , ,and Pmaxlevels are different in parity preference testing. Thecalculations are as follows, where c = the number of commonresponses, and n = the total number of respondents:Proportion of common responsesPc5 c/nScstandard deviation of Pc! 5 =Pc1 2 Pc!/nConf

43、idence Limit 5 Pc6zSc10.3.3 zis the critical value of the standard normal distri-bution. Values of zfor some commonly used values of -riskare:6MNL26-2ND Sensory Testing Methods: Second Edition, Chambers, E. andWolf, M.B., Eds., ASTM International, 1996.E2263 12 (2018)4-risk z0.50 0.0000.40 0.2530.20

44、 0.8420.10 1.2820.05 1.6450.01 2.3260.001 3.090Given the values chosen for and Pmax, if the confidencelimit is less than Pmax, then conclude that there is parity (that is,no more than Pmaxof the population would have a preferenceat the -level of significance). If the confidence limit is greaterthan

45、Pmax, then conclude that the products are not at parity.Understand that the -risk is larger than the value chosen whena smaller number of respondents participate in the test.10.4 If desired, calculate a two-sided confidence interval onthe proportion of common responses.11. Report11.1 Report the test

46、 objective, the results, and the conclu-sions. The following additional information is recommended:11.1.1 The purpose of the test and the nature of thetreatment studied;11.1.2 Full identification of the samples: origin, method ofpreparation, quantity, shape, storage prior to testing, servingsize, an

47、d temperature. (Sample information should communi-cate that all storage, handling, and preparation was done insuch a way as to yield samples that differed only in the variableof interest, if at all.);11.1.3 The number of respondents, recruitment criteria, thenumber of selections of each sample, and

48、the result of thestatistical analysis;11.1.4 Test sensitivity parameters: , , and Pmaxlevels,one-tailed or two-tailed test, critical value, decision risk;11.1.5 Respondents: age, gender, frequency of product us-age: typical/usual product consumption in the category (forexample, brand loyal or rotato

49、rs);11.1.6 The test environment: use of booths, simultaneous orsequential presentation and lighting conditions;11.1.7 The location and date of the test and name of the testadministrator;11.1.8 Next steps.12. Precision and Bias12.1 Because results of paired preference tests are a func-tion of individual preferences, a general statement regardingthe precision of results applicable to all populations of respon-dents cannot be made. Unless the demographics of the testpopulation are matched to U.S. census, results cannot beprojected to