1、 International Telecommunication Union ITU-T Series PTELECOMMUNICATION STANDARDIZATION SECTOR OF ITU Supplement 26(06/2012) SERIES P: TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings ITU-T P-ser
2、ies Recommendations Supplement 26 ITU-T P-SERIES RECOMMENDATIONS TERMINALS AND SUBJECTIVE AND OBJECTIVE ASSESSMENT METHODS Vocabulary and effects of transmission parameters on customer opinion of transmission quality Series P.10 Voice terminal characteristics Series P.30 P.300 Reference systems Seri
3、es P.40 Objective measuring apparatus Series P.50 P.500 Objective electro-acoustical measurements Series P.60 Measurements related to speech loudness Series P.70 Methods for objective and subjective assessment of speech quality Series P.80 P.800 Audiovisual quality in multimedia services Series P.90
4、0 Transmission performance and QoS aspects of IP end-points Series P.1000 Communications involving vehicles Series P.1100 Models and tools for quality assessment of streamed media Series P.1200 Telemeeting assessment Series P.1300 Statistical analysis, evaluation and reporting guidelines of quality
5、measurements Series P.1400 For further details, please refer to the list of ITU-T Recommendations. P series Supplement 26 (06/2012) i Supplement 26 to ITU-T P-series Recommendations Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings Summary Supplement 26
6、 to ITU-T P-series Recommendations concerns subjective quality assessment of telemeeting systems that provide audio-only or audio-visual communication for multiple parties. The term multiple parties refers to more than two meeting participants who can be located at two, or more than two, locations.
7、Evaluation of those systems can focus on audio-only, video-only, or audio-visual quality aspects and non-interactive or conversational quality can be assessed. This supplement provides example test scenarios that can be used to conduct conversation tests for multiparty audio-only telemeetings. Histo
8、ry Edition Recommendation Approval Study Group 1.0 ITU-T P Suppl. 26 2012-06-07 12 ii P series Supplement 26 (06/2012) FOREWORD The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of telecommunications, information and communication technologies (ICT
9、s). The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The World Telecommunication
10、Standardization Assembly (WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid down in WTSA Resolution 1. In some areas of inform
11、ation technology which fall within ITU-Ts purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. NOTE In this publication, the expression “Administration“ is used for conciseness to indicate both a telecommunication administration and a recognized operating agency.
12、Compliance with this publication is voluntary. However, the publication may contain certain mandatory provisions (to ensure, e.g., interoperability or applicability) and compliance with the publication is achieved when all of these mandatory provisions are met. The words “shall“ or some other obliga
13、tory language such as “must“ and the negative equivalents are used to express requirements. The use of such words does not suggest that compliance with the publication is required of any party. INTELLECTUAL PROPERTY RIGHTS ITU draws attention to the possibility that the practice or implementation of
14、 this publication may involve the use of a claimed Intellectual Property Right. ITU takes no position concerning the evidence, validity or applicability of claimed Intellectual Property Rights, whether asserted by ITU members or others outside of the publication development process. As of the date o
15、f approval of this publication, ITU had not received notice of intellectual property, protected by patents, which may be required to implement this publication. However, implementers are cautioned that this may not represent the latest information and are therefore strongly urged to consult the TSB
16、patent database at http:/www.itu.int/ITU-T/ipr/. ITU 2013 All rights reserved. No part of this publication may be reproduced, by any means whatsoever, without the prior written permission of ITU. P series Supplement 26 (06/2012) iii Table of Contents Page 1 Scope 1 2 References. 1 3 Definitions 1 4
17、Abbreviations and acronyms 1 5 Conventions 1 6 Three-party conversation test scenarios 1 7 English examples of three-party conversation test scenarios (3CTs) . 3 7.1 Symbols 3 7.2 Training scenario 4 7.3 Test-Scenarios 8 8 French examples of three-party conversation test scenarios (3CTs) 48 8.1 Symb
18、oles 48 8.2 Scnario dentrainement 49 8.3 Scnarios de test . 53 Bibliography. 93 P series Supplement 26 (06/2012) 1 Supplement 26 to ITU-T P-series Recommendations Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings 1 Scope This supplement concerns subject
19、ive quality assessment of telemeeting systems that provide communication between multiple parties at distant locations. As multiparty telemeetings can differ in a large number of aspects, the assessment of such systems requires a proper selection and control of the considered aspects and a precise d
20、escription when reporting results. Recommendation ITU-T P.1301 provides guidelines to select and if necessary adapt existing methods for subjective evaluation of multiparty telemeetings. Concerning the evaluation of conversational quality of multiparty telemeetings, ITU-T P.1301 provides also descri
21、ptions of conversation tasks. The present supplement complements these with example scenarios for conversation test scenarios for three parties. 2 References ITU-T P.1301 Recommendation ITU-T P.1301, Subjective quality evaluation of audio and audiovisual multiparty telemeetings. 3 Definitions None.
22、4 Abbreviations and acronyms 3CTs Three-party Conversation Test scenario 5 Conventions None. 6 Three-party conversation test scenarios The following conversation test method was developed for assessing audio conferencing quality with three participants. Details of the test scenario development can b
23、e found in b-RAAKE2008, a validation carried out in b-RAAKE2010 showed that subject groups have a higher impact on the conversation duration than the scenarios. The intended dialogue flow of the scenarios is schematically depicted in Figure 1. The scenario instruction sheets given to participants ha
24、ve been developed to support a close-to-natural emerging of the dialogue flow. 2 P series Supplement 26 (06/2012) Figure 1 Targeted conversation structure This supplement contains the scenario instruction sheets, which were originally developed in German and were translated into English and French,
25、including adaptations of names and cultural references (for the United States and France). Notice that adaptation of these scenarios to further cultural contexts requires not only literal translation but also adaptation of names, cultural references (e.g., locations, telephone numbers, etc.), or eve
26、n the scenario theme, in order to ensure that the test subjects experience these scenarios as naturally as possible in their cultural environment. P series Supplement 26 (06/2012) 3 7 English examples of three-party conversation test scenarios (3CTs) 7.1 Symbols The following symbol indicates that y
27、ou are the caller: This symbol indicates that you are the caller. Please wait until the test supervisor asks you to initiate the call. The following symbol indicates that you are a callee: This symbol indicates that you are a callee. Please wait until you hear a ringtone before starting the conversa
28、tion. All three interlocutors have the following symbols: This symbol indicates both the beginning (welcome) and the end of the conversation (goodbye). Next to this symbol, you will find the name related to the role you have in this scenario. Next to this symbol, you will find the reason for your ca
29、ll or a question you would like to ask (e.g., make an appointment). Next to this symbol, you will find a constraint or conflict that you would like to mention (e.g., other conflicting appointments). Next to this symbol, you will find information that your conversation partners might need (e.g., avai
30、lability of a meeting room). This symbol indicates that you should find an agreement (e.g., date and time of the meeting). Next to this symbol, you will find a table or a list of information. Please fill all empty fields with information that you get from your conversation partners. Please share the
31、 information that you have on your side with the others. This symbol indicates that one interlocutor has a question for which neither you, nor any other interlocutor, have an answer. Please discuss this question briefly and find a solution that is acceptable for everyone. 4 P series Supplement 26 (0
32、6/2012) 7.2 Training scenario Overview Scheduling a meeting Participants: Caller: Meyers Company: Brewery BestBeer, Corporate Communications Interlocutor 2: Buchannon Company: TV-Station FootballChannel, Marketing Interlocutor 3: Scott Company: Nibblers Premium Snacks, Public Relations Reason for th
33、e call: Agree on a meeting to prepare a sponsorship offer between the TV-Station FootballChannel, the BestBeer brewery, and the company Nibblers Premium Snacks. Agenda items for the telephone conference: Topic 1: Schedule a meeting: fix a time slot Topic 2: Exchange of e-mail addresses to send an ag
34、enda in advance Any other business: Choose director for commercial spot P series Supplement 26 (06/2012) 5 Conversation guide Scheduling a meeting Caller Name: Meyers Company: Brewery BestBeer, Corporate Communications You dialled the telephone number of Mr/Ms Buchannon from the Marketing Department
35、 of the TV-Station FootballChannel: Welcome Topic 1: Previously you already agreed on: One-hour meeting next Monday Place: meeting room at TV-Station FootballChannel Meeting topic: discuss sponsorship offer To be discussed: At what time? Your appointments on Monday 9.00 10.00 10.00 11.00 11.00 12.00
36、 12.00 13.00 13.00 14.00 14.00 15.00 15.00 16.00 Seminar cannot be postponed 16.00 17.00 Agreed on the time: _ oclock Topic 2: Exchange e-mail addresses: Name Department E-mail address Meyers Corporate Communications Any other business: Discussion Goodbye 6 P series Supplement 26 (06/2012) Conversa
37、tion guide: Scheduling a meeting Interlocutor 3 Name: Scott Company: Nibblers Premium Snacks, Public Relations Mr/Ms Meyers and Mr/Ms Buchannon will ask you to join the call: Welcome Topic 1: You already have some appointments: Your appointments on Monday 9.00 10.00 Weekly group meeting 10.00 11.00
38、11.00 12.00 12.00 13.00 Lunch with a customer 13.00 14.00 14.00 15.00 15.00 16.00 16.00 17.00 Agreed on the time: _ oclock Topic 2: Exchange e-mail addresses: Name Department E-mail address Scott Public Relations Any other business: Invite a director for a commercial spot? Discussion Goodbye P seri
39、es Supplement 26 (06/2012) 7 Conversation guide: Scheduling a meeting Interlocutor 2 Name: Buchannon Company: TV-Station FootballChannel, Marketing Welcome Please ask Mr/Ms Scott from the Public Relations Department of the Company Nibblers Premium Snacks to join the call. Topic 1: The meeting takes
40、place in one of your meeting rooms: Room bookings on Monday 9.00 10.00 booked 10.00 11.00 booked 11.00 12.00 booked 12.00 13.00 booked 13.00 14.00 14.00 15.00 15.00 16.00 16.00 17.00 booked 17.00 18.00 booked Agreed on the time: _ oclock Topic 2: Exchange of e-mail addresses to send an agenda in adv
41、ance? Exchange e-mail addresses: Name Department E-mail address Buchannon Marketing t.buchannonfootball- Any other business: Discussion Goodbye 8 P series Supplement 26 (06/2012) 7.3 Test-Scenarios Scenario 1: Time frame for the appearance at a trade-fair Participants: Caller: Schumaker Company: Med
42、iaHouse, Marketing Interlocutor 2: Pfeiffer Company: MediaHouse, Corporate Communications Interlocutor 3: Moore Company: Starlight Engineering and Consulting Reason for the call: Time frame for building up the booth, technical setup, e.g., lighting and rehearsal of the MediaHouse company at the Cons
43、umer Electronics Show (CES) in Las Vegas Agenda items for the telephone conference: Topic 1: At what time will the lighting and the show lasers be installed? Topic 2: In case of urgent problems: exchange contact information of local partners Any other business: Fog machine P series Supplement 26 (06
44、/2012) 9 Scenario 1: Time frame for the appearance at a trade-fair Caller Name: Schumaker Company: MediaHouse, Marketing You dialled the telephone number of Mr/Ms Pfeiffer from the Corporate Communications Department of your company: Welcome Topic 1: MediaHouse will have a booth at the CES conventio
45、n in Las Vegas: When will the installation of the lighting and the show lasers take place? Time frame of the Las Vegas Exhibition Corporation: CES Las Vegas: Time frame for exhibitors As of 15 August Halls of the Las Vegas Convention Center are open for booth setup Agreed on the time frame _ Topic 2
46、: Contact information of local partners: Company Contact person Telephone no. E-mail address Las Vegas Convention Center Mr Fair 555 43 34 45 55 fairlas-vegas- Millers Booth Constructions Lighting Installations Watts & Co. Any other business: Discussion Goodbye 10 P series Supplement 26 (06/2012) Sc
47、enario 1: Time frame for the appearance at a trade-fair Interlocutor 3 Name: Moore Company: Starlight Engineering and Consulting Mr/Ms Schumaker and Mr/Ms Pfeiffer will ask you to join the call: Welcome Topic 1: Time required for installing lighting and lasers: three days Appointment calendar: Starl
48、ight Engineering and Consulting August 15th-17th Booked: Lighting for a discotheque 18th-20th Available 22nd-27th Booked: Lighting for new studio room at CNN Las Vegas 28th-31st Available Agreed on the time frame _ Topic 2: Contact information of local partners: Company Contact person Telephone no. E-mail address Las Vegas Convention Center Millers Booth Constructions Lighting Installations Watts & Co. Mr Edison 555 87 88 932 Any other busine
