1、ETSI ETR*Zb1-7 96 3400855 0141930 I103 Wv ETSI ETSI T ETR 261-7 October 1996 Source: ETSI TC-HF Reference: DTR/HF-O1028-7 ICs: 33.020 Key words: Keypad, MMI, supplementary service Human Factors (HF); Assessment and definition of a harmonized minimum man-machine interface (MMI) for accessing and cont
2、rolling public network based supplementary services; Part 7: Experimental evaluation of draft ETS 300 738 ETSI European Telecommunications Standards Institute ETSI Secretariat Postai address: F-O6921 Sophia Antipolis CEDEX - FRANCE Off ice address: 650 Route des Lucioles - Sophia Antipolis - Valbonn
3、e - FRANCE X.400: c=fr, a=atlas, p=etsi, s=secretariat - Internet: secretariat Q etsi.fr Tel.: +33 4 92 94 42 O0 - Fax: +33 4 93 65 47 16 Copyright Notification: No part may be reproduced except as authorized by written permission. The copyright and the foregoing restriction extend to reproduction i
4、n all media. Q European Telecommunications Standards Institute 1996. All rights reserved. ETSI ETR*:2bL-7 96 3400855 OL4193L 04T = Page 2 ETR 261-7: October 1996 Whilst every care has been taken in the preparation and publication of this document, errors in content, typographical or otherwise, may o
5、ccur. If you have comments concerning its accuracy, please write to “ETSI Editing and Cornmittee Support Dept.“ at the address shown on the title page. ETSI ETRx2bL-7 96 = 3400855 OL41932 T8b W Page 3 ETR 261 -7: October 1996 Contents Foreword . 5 Introduction 5 Scope 7 References 7 Definitions. sym
6、bols and abbreviations . 8 Introduction to the experiment . 8 Method 9 5.1 Subjects . 9 5.2 Experimental procedure . 9 5.3 Equipment 11 5.4 Data capture 12 Results 12 6.1 Personal data . 12 6.2 Performance 12 6.3 Results for the “Assisted mode“ (Turin sample) 16 6.4 Results for the “Learning mode“ (
7、Berlin sample) . 17 6.5 Overall analysis 18 Detailed analysis . 20 7.1 Error analysis . 20 7.1.1 Cross-service errors 20 7.1.2 Service-specific errors . 21 7.2 Timing 21 7.3 Knowledge transfer 23 7.4 Short questionnaire 23 7.4.1 Assisted Mode . 23 7.4.2 Learning Mode 24 7.5 Long questionnaire 25 7.5
8、.1 Assisted Mode . 25 7.5.2 Learning Mode 26 Discussion of results and conclusion 27 8.1 Effects of the experimental implementation . 27 8.1.1 Meaning of tones . 27 8.1.2 Sub-optimum implementation of the display . 27 8.1.3 Sub-optimum instructions for experimental tasks . 27 Critical issues identif
9、ied during the study . 27 8.2.1 Learning complex command strings . 27 8.2.2 Service definition . 28 8.2.3 User models in 3-Party Conferencing . 28 8.3 Conclusions . 28 8.3.1 Is PBI+ better than PBI+, and PBI+ better than PBI? 28 8.3.2 What effect does the information before the control action have o
10、n usability? 28 8.3.3 What effect do feedback or prompts during and as a result of the control action have on usability? . 29 8.3.4 What effect does the increase in syntax complexity from “activate“ and “de-activate“ to including “interrogate“ have on usability? . 29 8.2 ETSI ETR*261-7 7b 3400855 03
11、43933 912 Page 4 ETR 261-7: October 1996 Annex A: A.l Personal Data . 30 Personal data tables for the German and Italian subjects 30 Annex B: General Instructions . 34 Annex C: Description of the Services 35 Annex D: Tasks and Questionnaires 42 Annex E: Error Analysis . 54 E.l Types of errors vs . T
12、asks for PEI. Sessions 1 and 2. “Assisted Mode“ . 54 E.2 Types of errors vs . Tasks for PBI+. Sessions 1 and 2 (“Assisted Mode“) 55 E.3 Types of errors vs . Tasks for PBI+. Sessions 1 and 2 (“Assisted Mode“) 56 E.4 Types of errors vs . Tasks for PEI. Session 1 (“Learning Mode“) . 57 E.5 Types of err
13、ors vs . Tasks for PBI. Session 2 (“Learning Mode“) . 58 E.6 Types of errors vs . Tasks for PEI+. Session 1 (“Learning Mode“) . 59 E.7 Types of errors vs . Tasks for PEI+. Session 2 (“Learning Mode“) . 60 E.8 Types of errors vs . Tasks for PBI+. Session 1 (“Learning Mode“) . 61 E.9 Types of errors v
14、s . Tasks for PEI+. Session 2 (“Learning Mode“) . 62 Annex F: Responses to Long Questionnaire 63 History . 65 - ETSI ETRa2bL-7 96 m 3400855 0141934 859 m Page 5 ETR 261-7: October 1996 Foreword This ETSI Technical Report (ETR) has been produced by the Human Factors (HF) Technical Committee of the Eu
15、ropean Telecommunications Standards Institute (ETSI). ETRs are informative documents resulting from ETSI studies which are not appropriate for European Telecommunication Standard (ETS) or Interim European Telecommunication Standard (I-ETS) status. An ETR may be used to publish material which is eith
16、er of an informative nature, relating to the use or the application of ETSs or I-ETSs, or which is immature and not yet suitable for formal adoption as an ETS or an I-ETS. Introduction The Technical Committee for Human Factors has prepared this ETSI Technical Report to report publicly its work on th
17、e assessment and definition of a harmonized minimum man-machine interface for the access and control of public network based supplementary services. It is intended to complement ETS 300 738 3. This ETR constitutes part 7 of a multi-part ETR (“Assessment and definition of a harmonized minimum man-mac
18、hine interface (MMI) for accessing and controlling public network based supplementary services“), whose parts have the following titles: Part 1: “General approach and summary of findings“; Part 2: “Literature review - Memory and related issues for dialling supplementary services using number codes“;
19、 Part 3: “Experimental comparison of two MMIs - Simulated UPT access and prototype ISDN supplementary services“; Part 4: “Experimental comparison of the effect of categorized and non-categorized formats within user instructions“; Part 5: “Experimental comparison of the CEPT and GSM codes schemes“; P
20、art 6: “Survey of existing PSTN, ISDN and mobile networks, and a user survey of supplementary service use within Centrex and PBX environments“; Part 7: “Experimental evaluation of draft ETS 300 738“. ETSI ETRa2bL-7 96 3400855 Ol14L935 795 Page 6 ETR 261-7: October 1996 Blank page ETSI ETR*E!bL-7 96
21、3400855 OL4L93b 621 Page 7 ETR 261 -7: October 1996 1 Scope This multi-pari ETSI Technical Report (ETR) presents the results of the research work conducted to develop a European Telecommunication Standard (ETS) defining a harmonized minimum man-machine interface (MMI) for the access and control of p
22、ublic network based telecommunications services, and in particular supplementary services. This part 7 of the ETR presents the results of an experimental evaluation of the harmonized minimum man-machine interface as defined in the draft ETS being produced under ETSI Work Item to define a minimum har
23、monized MMI for the access and control of public network based supplementary services 3. The report describes the methodology, subjects, and a simulation developed to support the MMI, as the common agreed experimental approach to the evaluation. The results and conclusions discussed are based on two
24、 sets of experimental data. 2 References For the purposes of this ETR, the following references apply: I51 71 Pl 91 o1 Allison, G and Hewson, T: “Usability evaluation of 3 protocols for conference calling (9way calling)“. Proc. 15th HFT Symposium, Melbourne, 6-10 March 1995. CEPT: T/CAC 02: “Subscri
25、ber control procedures for supplementary services in modern telecommunication system“. ETS 300 738: “Human Factors (HF); Minimum Man Machine Interface (MMI) to public network based supplementary services“. ETR 261-3: “Human Factors (HF); Assessment and definition of a harmonized minimum man-machine
26、interface (MMI) for accessing and controlling public network based supplementary services; Pari 3: Experimental comparison of two MMIs - Simulated UPT access and prototype ISDN supplementary services“. ETR 261 -4: “Human Factors (HF); Assessment and definition of a harmonized minimum man-machine int
27、erface (MMI) for accessing and controlling public network based supplementary services; Part 4: Experimental comparison of the effect of categorized and non-categorized formats within user instructions“. ETR 261-5: “Human Factors (HF); Assessment and definition of a harmonized minimum man-machine in
28、terface (MMI) for accessing and controlling public network based supplementary services; Part 5: Experimental comparison of the CEPT and GSM codes schemes“. ETR 261 -6: “Human Factors (HF); Assessment and definition of a harmonized minimum man-machine interface (MMI) for accessing and controlling pu
29、blic network based supplementary services; Part 6: Survey of existing PSTN, ISDN and mobile networks, and a user survey of supplementary service use within Centrex and PBX environments“. ETS 300 51 1 : “European digital cellular telecommunications system (Phase 2); Man Machine interface (MMI) of the
30、 mobile station (MS) (GSM 02.30)“. ITU-T Recommendation E.131: “Subscriber control procedures for supplementary telephone services“. ITU-T Recommendation E.161: “Arrangement of figures, letters and symbols on telephones and other devices that can be used for gaining access to a telephone network“. E
31、TSI ETR*2bL-7 96 3400855 0141937 568 H Page 8 ETR 261-7: October 1996 ETR 261-1: “Human Factors (HF); Assessment and definition of a harmonized minimum man-machine interface (MMI) for accessing and controlling public network based supplementary services; Part 1 : General approach and summary of find
32、ings“. 3 Definitions, symbols and abbreviations For the purposes of this part of the ETR, the definitions, symbols and abbreviations given in part 1 ll of the ETR apply. 4 Introduction to the experiment The ETR is complementary to the ETS being drafted in parallel on a harmonized minimum MMI for acc
33、essing and controlling supplementary services 3, and reports a study that was conducted in order to support empirically some of the recommendations made in the draft ETS. In particular, it was intended to provide data to support the approach taken in the draft ETS of focusing on three elements as pr
34、imary factors in usability. These comprise the information needed prior to the control action (e.g. handbooks, prompts, etc.), the control action itself (e.g. a command for activating a service), and the information after the control action (e.g. feedback). The issue addressed with the present study
35、 is whether the level of information (in terms of quantity and modality) specified in the draf ETS is indeed required for the MMI to be usable, or whether less information presented in a single mode only (e.g. through tones) is sufficient. The results of an earlier survey (part 6 of the present ETR
36、?I) showed widespread ignorance about the existence or use of supplementary services. A pilot experimental study (part 3 of the present ETR 4) also showed that the information given prior to the control action is critical to being able to control supplementary services successfully using the minimum
37、 MMI implemented through the 12-key telephone keypad. Other data collected during user experiments showed that using an MMI becomes almost impossible if no or inadequate information or handbook is available. Finally, the study results (part 4 of the present ETR 5) showed that information provided in
38、 handbooks should be carefully structured in order to make explicit the logic underlying the syntax of the MMI. The present study examined three interfaces that varied in terms of the amount of information and the modes in which information was presented prior to and after a control action. All thre
39、e implementations were compliant with the recommendations given in the draft ETS (with the exception of PBI, see below). The reason for only testing compliant implementations lay in the fact that the previous experimental work reported in pari 5 of the present report 6 indicated that the choice of t
40、he code scheme (.e. CEPT or GSM) did not matter for the user as much as the instruction and feedback information, which seemed to be far more critical. It should also be added that the CEPT Service Codes used in the present study (e.g. “21 I for Call Forwarding) were not changed in the different exp
41、erimental conditions. Three different man-machine interfaces were used: - PBI the feedback consisted of network tones only; PBI+ the feedback consisted of tones and verbal announcements; PBI+ the feedback consisted of tones, announcements and visual messages. Three supplementary services were tested
42、 in the present experiment: Call Forwarding Unconditional (CFU), Call Barring (CB) and 3-Party Call (3PTY). They were chosen because they are representative of the various categories of supplementary services. In particular, 3PTY is switching order based, whereas CFU and CB are code scheme based, an
43、d CB is defined in a negative manner (.e. switching on the service bars the outgoing calls) and requires entry of supplementary information (e.g. PIN) in addition to the service code. In addition to studying the effects of different feedback modalities as provided by the three interfaces (PBI, PBI+,
44、 and PBI+), the issue of syntax complexity was addressed. The absolute minimum functionality to be provided by a phone-based interface consists of control sequences for activating and de-activating a service. Any additional functionality increases the syntax in terms of further commands to be learnt
45、 by the user. Interrogating the system on whether a particular service is currently active or not (“interrogate“-function) is from a human factors point of view an additional desirable function that should be implemented as part of the MMI. In order to be able to assess the impact of increasing the
46、minimum functionality (and consequently the syntax complexity) from “activate“ and “de-activate“ to include ETSI ETR*E!bL-7 96 3400855 0141938 4T4 Page 9 ETR 261 -7: October 1996 “interrogate“, two implementations of the PBi condition were tested only one of which included the specific command seque
47、nce for “interrogate“. Finally, the question was addressed whether, after sufficient learning time, the services could be used without supporting instructional material. The experiments were conducted in two separate locations, at CSELT in Turin, Italy, and the Heinrich Hertz Institute in Berlin, Ge
48、rmany, using a common agreed experimental design. Procedurally, each subject was asked to attempt to gain access to a service, interrogate it or deactivate it, on one of the three interfaces, according to specific user instructions supplied. The aim of the experiment was to provide data to help answ
49、er five questions: 1) 2) 3) 4) 5) Is PBI+ better than PBI+, and PBI+ better than PBI? What effect does the information before the control action have on usability? What effect do feedback or prompts during and as a result of the control action have on usability? What effect does the increase in syntax complexity from “activate“ and “de-activate“ to including “interrogate“ have on usability? Can the users learn the procedures after making use of the services twice, or do they have to rely on printed material for a longer period of time? 5 Method 5.1 Subject
