1、 ISO 2013 Usability of consumer products and products for public use Part 2: Summative test method Facilit demploi des produits de consommation courante et des produits usage public Partie 2: Mthode dessai sommative TECHNICAL SPECIFICATION ISO/TS 20282-2 Second edition 2013-08-01 Reference number IS
2、O/TS 20282-2:2013(E) ISO/TS 20282-2:2013(E)ii ISO 2013 All rights reserved COPYRIGHT PROTECTED DOCUMENT ISO 2013 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photo
3、copying, or posting on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below or ISOs member body in the country of the requester. ISO copyright office Case postale 56 CH-1211 Geneva 20 Tel. + 41 22 749 01 11 Fax + 41 22 749 09
4、 47 E-mail copyrightiso.org Web www.iso.org Published in Switzerland ISO/TS 20282-2:2013(E) ISO 2013 All rights reserved iii Contents Page Foreword iv Introduction v 1 Scope . 1 2 Conformity 2 3 Normative references 2 4 T erms and definitions . 2 5 General principles . 6 5.1 Type of usage of the pro
5、duct to be tested 6 5.2 Purpose of the test . 6 5.3 Scope of the goals used in the test . 7 5.4 Validity and reliability 9 5.5 Ethical issues 9 6 Test procedure overview 9 7 Test method 10 7.1 Purpose and scope of the test .10 7.2 Product(s) to be tested 10 7.3 Context of evaluation 10 7.4 Check tha
6、t the product is capable of achieving the intended goals for the intended users 15 7.5 Design and perform the test .16 8 Results 20 8.1 General 20 8.2 Goals 20 8.3 Effectiveness 20 8.4 Efficiency .21 8.5 Satisfaction 21 9 Report .21 Annex A (informative) User goals .22 Annex B (informative) Backgrou
7、nd to the method .25 Annex C (normative) Recruiting a representative sample of users .26 Annex D (normative) C onfidenc e int erv als 29 Annex E (normative) Examples of satisfaction scales33 Annex F (normative) Format for test reports .35 Annex G (normative) Specification of usability r equir ements
8、 .41 Annex H (normative) Usability t est pr oc edur e specification 44 Annex I (informative) Feedback on the standard .48 Bibliography .49 ISO/TS 20282-2:2013(E) Foreword ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodie
9、s). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and n
10、on-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization. The procedures used to develop this document and those intended for its further maintenance are desc
11、ribed in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2. www.iso.org/directives Attention is drawn to the
12、 possibility that some of the elements of this document may be the subject of patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any patent rights identified during the development of the document will be in the Introduction and/or on the ISO l
13、ist of patent declarations received. www.iso.org/patents Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement. For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment, as well as
14、 information about ISOs adherence to the WTO principles in the Technical Barriers to Trade (TBT) see the following URL: Foreword - Supplementary information The committee responsible for this document is ISO/TC 159, Ergonomics, Subcommittee SC 4, Ergonomics of human-system interaction. This second e
15、dition cancels and replaces the first editions of ISO/TS 20282-2:2006, ISO/PAS 20282- 3:2007, and ISO/PAS 20282-4:2007, of which it constitutes a technical revision. The main changes are the following: clarification that the test method is only intended to be used when there are a limited number of
16、goals to be tested and it is possible to identify typical contexts of use and criteria for successful goal achievement; provision for a wider range of methods to test achievement of goals; use of ISO 9241-11 terminology: usability, effectiveness, efficiency, and satisfaction; wider range of levels o
17、f confidence with a sample size as low as 10, using the Adjusted Wald statistic; wider range of purposes for use of the test method. ISO/TS 20282 consists of the following parts, under the general title Usability of consumer products and products for public use: Part 1: Design requirements for conte
18、xt of use and user characteristics Part 2: Summative test method (Technical Specification)iv ISO 2013 All rights reserved ISO/TS 20282-2:2013(E) Introduction Many people find some consumer products and walk-up-and-use products, including consumer products provided for public use, difficult to instal
19、l and use, particularly when using them for the first time or at infrequent intervals. This is clearly undesirable for the producers of such products, for organizations that use the products to provide a service, and for the people who use them. Information about the usability of a product would, th
20、erefore, be of great value to producers, as part of development and marketing, to service providers, and to potential purchasers making purchase decisions or comparing alternative products. This would provide an incentive for producing products that are easier to install and use and would enable pot
21、ential purchasers to pay specific attention to usability when selecting a product to buy and use. It is difficult to judge usability in a purchase situation without available comparable usability test results. Usability (see ISO 9241-11) is the extent to which a product can be used by specified user
22、s to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. Effectiveness is fundamental as it is about achieving the intended goal(s). Efficiency is about the resources (such as time or effort) needed by users to achieve their goals so it can be impo
23、rtant. In addition, it is important that users are satisfied with their experience, particularly where users have discretion over whether to use a product and can readily choose some alternative means of achieving their goals. In this part of ISO/TS 20282, accessibility is operationalized as the ext
24、ent to which a product can be used with effectiveness, efficiency, and satisfaction by people from a population with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use. Poor usability and/or accessibility can result in errors that can lead
25、to several types of risks, for example, inconvenience resulting from not achieving a goal or achieving the wrong goal, incurring unexpected costs, or physical injury. In many countries, there are legal requirements to provide accessible products, services, and facilities. EXAMPLE Calling the wrong p
26、erson by mistake with a mobile phone may have the negative consequence of possible undesirable call charges either for the caller or the person called (who may have to pay for the call). In addition to the risks of potential adverse consequences for the user as a result of failing to achieve their g
27、oal or achieving the wrong goal (poor effectiveness), there are other risks such as being late as a result of poor efficiency or users avoiding the use of a difficult-to-use product as a result of poor satisfaction. Formative evaluation using expert inspection or user-based testing to provide feedba
28、ck to improve the usability of the product is an integral part of the iterative human-centred design process recommended in ISO 9241-210. Summative evaluation can be used to validate usability and/or accessibility requirements, to provide a benchmark, or to provide a basis for comparison of differen
29、t products. Although some types of expert inspection methods based on a checklist or a standard can provide summative data, the aspects of usability and/or accessibility that are measured are limited in comparison with the measures of effectiveness, efficiency, and satisfaction provided by user-base
30、d testing. EXAMPLE One study found that only 50 % of the problems encountered on 16 websites by 32 blind users were covered by Success Criteria in the Web Content Accessibility Guidelines 2.0 (WCAG 2.0). 23 Inspection can precede user-based testing to identify (and, if possible, eliminate) easily id
31、entifiable problems and to check that the product is capable of achieving the intended goals for the intended users (see 7.4). To provide reliable data on effectiveness, efficiency, and satisfaction that can be compared, it is desirable to have a standard summative user-based test procedure. This pa
32、rt of ISO/TS 20282 specifies a summative user-based test method that can be used to provide an evaluation of the usability and/or accessibility and ease of unpacking, setting up, and installation of consumer products, and the usability and/or accessibility of products for public use (including walk-
33、up-and-use products). It can be applied to products that are used to achieve goals that have clear success criteria and relate to well-defined types of subject matter. ISO/TS 20282-1 describes in more detail sources of variance in user characteristics that form part of the context of use that needs
34、to be taken into account when designing for usability. This information is also ISO 2013 All rights reserved v ISO/TS 20282-2:2013(E) needed to identify the elements of the context of use required for testing in this part of ISO/TS 20282. Further information about the characteristics of older people
35、 and people with disabilities can be found in ISO/TR 22411.vi ISO 2013 All rights reserved TECHNICAL SPECIFICATION ISO/TS 20282-2:2013(E) Usability of consumer products and products for public use Part 2: Summative test method 1 Scope This part of ISO/TS 20282 specifies a user-based summative test m
36、ethod for the measurement of the usability and/or accessibility of consumer products and products for public use (including walk-up-and- use products) for one or more specific user groups. This test method treats accessibility as a special case of usability where the users taking part in the test re
37、present the extremes of the range of characteristics and capabilities within the general user population. When the test method refers to usability, the method can also be used to test accessibility (unless otherwise specified). This test method is for use when valid and reliable measures of effectiv
38、eness, efficiency, and satisfaction are needed. NOTE 1 Products for public use include walk-up-and-use products that provide a service to the general public. The test method can also be used to assess the usability and/or accessibility of achieving the goals of unpacking, installing, and setting up
39、a consumer product. This part of ISO/TS 20282 is intended to be used for testing the usability and/or accessibility of products when it is possible to identify typical contexts of use that are representative of the use of the product(s), it is possible to identify the criteria for the successful ach
40、ievement of the users goal, and there are a limited number of goals being tested at the same time. While the test method is intended to test consumer products and products for public use, it can also be used to test other products, systems, and services with the characteristics described above. If u
41、se of a product involves interaction with inputs, outputs, or environments that are highly variable and/or complex with variability or complexity that cannot be categorized in well-defined subsets, it is outside the scope as it would not be possible to obtain reliable results. See Annex A for exampl
42、es of products and goals that are within the scope of this part of ISO/TS 20282. EXAMPLE The method could be applied to an office photocopier, a website selling books or train tickets, or a legal advice service. The method would not be appropriate for a complex ecommerce website, a word processor, o
43、r a bicycle. The method is primarily intended for use for assessing completed versions of products, but could also be used for internal purposes during development to judge, assess, and communicate the usability and/or accessibility of functional prototype versions. The results of the summative test
44、 method can be used for the following purposes: to estimate the probability of achieving target values of effectiveness, efficiency, and satisfaction in actual use; to publish information about the usability and/or accessibility of a product; to compare the usability and/or accessibility of several
45、products; ISO 2013 All rights reserved 1 ISO/TS 20282-2:2013(E) to compare the results with a usability and/or accessibility requirements specification; to support procurement. NOTE 2 Annex H lists the information to be included when specifying the procedure used to test whether the usability and/or
46、 accessibility requirements (Annex G) have been met. The intended users of this part of ISO/TS 20282 are people with expertise in the design and management of testing usability and/or accessibility, working within or on behalf of manufacturers, suppliers, purchasing organizations, or third parties (
47、such as test organizations or consumer organizations). 2 Conformity A report of the values for the usability and/or accessibility of a product conforms to this part of ISO/TS 20282 if the test method used conforms to the requirements in Clauses 6, 7, 8, and 9 and Annexes C and D, and the report of t
48、he results contains the information specified in Annex F. A statement of requirements for usability results conforms to this part of ISO/TS 20282 if it conforms to the requirements in Annex G. The specification of a usability test procedure conforms to this part of ISO/TS 20282 if it conforms to the
49、 requirements in Annex H. 3 Normative references The following documents, in whole or in part, are normatively referenced in this document and are indispensable for its application. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO 20282-1, Ease of operation of everyday products Part 1: Design requirements for context of use and user characteristics 4 T erms a nd definiti ons For the purposes of this
