1、 INCITS/ISO/IEC 19796-3:20092010 (ISO/IEC 19796-3:2009, IDT) Information technology - Learning, education and training - Quality management, assurance and metrics - Part 3: Reference methods and metrics Reaffirmed as INCITS/ISO/IEC 19796-3:2009 R2015INCITS/ISO/IEC 19796-3:20092010 PDF disclaimer Thi
2、s PDF file may contain embedded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, parties accept the
3、rein the responsibility of not infringing Adobes licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relative to the file; t
4、he PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. Adopted by INCITS (Inter
5、National Committee for Information Technology Standards) as an American National Standard. Date of ANSI Approval: 11/18/2010 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2010 by Information Technology Industry Council (ITI). All rights r
6、eserved. These materials are subject to copyright claims of International Standardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Council (ITI). Not for resale. No part of this publication
7、may be reproduced in any form, including an electronic retrieval system, without the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1101 K Street NW, Suite 610, Washington DC 20005. Printed in the United States of America ii ITIC 2010 All rights
8、 reserved ISO/IEC 19796-3:2009(E) ISO/IEC 2009 All rights reserved iiiContents Page Foreword iv Introduction.v 1 Scope1 2 Conformance .2 3 Normative references2 4 Terms and definitions .2 5 Mapping quality approaches4 5.1 Quality approaches and ISO/IEC 19796 series.4 5.2 Practical guideline for qual
9、ity approaches.6 6 Quality management / assurance activity model .6 7 Reference model for quality methods .8 7.1 Categories 8 7.2 Elements.10 8 Reference model for quality metrics .14 8.1 Categories 14 8.2 Elements.16 9 Collection of methods and metrics .19 9.1 Collection of methods.19 9.2 Collectio
10、n of metrics .22 Annex A (informative) Description format examples of methods reference model.30 Annex B (informative) Description format examples of metrics reference model.34 Bibliography38 ISO/IEC 19796-3:2009(E) iv ISO/IEC 2009 All rights reservedForeword ISO (the International Organization for
11、Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective orga
12、nization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technol
13、ogy, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to prepare International Standards. Draft International Standards
14、adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is drawn to the possibility that some of the elements of this document may be the subjec
15、t of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. ISO/IEC 19796-3 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 36, Information technology for learning, education and training. ISO/IEC 19796
16、 consists of the following parts, under the general title Information technology Learning, education and training Quality management, assurance and metrics: Part 1: General approach Part 3: Reference methods and metrics The following parts are under preparation: Part 2: Harmonized quality model Part
17、 4: Best practice and implementation guide Technical Report Part 5: How to use ISO/IEC 19796-1 Technical Report ISO/IEC 19796-3:2009(E) ISO/IEC 2009 All rights reserved vIntroduction Quality in the field of distance education and e-learning has become an issue of increasing importance in academia, i
18、nstitutions, and industry. A variety of approaches have been developed and implemented successfully. Generic standards, such as ISO 9000:2005 and/or ISO 14000 have been used also in the educational community. In addition to generic standards related to quality, there are specific quality guidelines
19、that have been developed and used for e-Learning or distance education (such as the ASTD criteria for e-Learning, the BLA Quality Mark, Quality Platform Learning by D-ELAN, or Quality elements by Sloan-consortium). It has become clear that quality management can contribute to improve the performance
20、 of organizations in the field of learning, education, and training (LET). Numerous approaches to quality management and assurance and their different scopes and objectives lead to confusion within communities that depend on information technologies to support and facilitate learning, education, and
21、 training. Therefore, a harmonized quality standard, the ISO/IEC 19796 series, has been developed. ISO/IEC 19796 has five parts as a series. ISO/IEC 19796-1 is the basic framework for quality development in organizations within the field of learning, education, and training (LET). It serves as a fra
22、mework to describe, compare, and analyze quality management and quality assurance approaches. In addition to providing a descriptive model that can be used to compare and analyze quality approaches, ISO/IEC 19796-1 identifies the components of a seven-part process model within the lifecycle of infor
23、mation and communication systems for learning, education, and training. As a framework it may be used to compare different quality assurance and quality management processes. This part of ISO/IEC 19796 provides reference methods and metrics used in the lifecycle process. It also provides examples of
24、 how methods and metrics can be generically described, compared, and used for specific contexts. For each process in the e-learning lifecycle, a set of potential methods and metrics should be specified. This set of methods and metrics can be used during the development of an individual quality appro
25、ach that is based on defined quality objectives. ISO/IEC 19796-1 describes the processes for the e-Learning lifecycle. It is a reference model with a high level of abstraction which has to be adapted to a certain organization and to a certain situation. To facilitate the adaptation procedure, two re
26、ference models for quality management and quality assurance are described in this part of ISO/IEC 19796: reference model for methods; reference model for metrics. To facilitate the adaptation of ISO/IEC 19796-1, as the first step, this part of ISO/IEC 19796 provides reference methods and metrics use
27、d in lifecycle processes with the reference models from ISO/IEC 19796-1. It also provides collections of methods and metrics which are generically described and can be used for specific contexts. For each process in the e-Learning lifecycle, a set of potential methods and metrics should be specified
28、. This set of methods and metrics can be used during the development and definition of an individual quality approach based on certain quality objectives. ISO/IEC 19796-1 provides a description format and a process framework as the reference framework for the description of quality approaches (RFDQ)
29、, for the description and development of quality approaches. The RFDQ framework is only a base for quality development it does not provide specific methods and metrics for particular quality objectives, and also does not provide instruments or procedures. ISO Guide 72 Guidelines mainly distinguish t
30、hree types of management system standards; Type A management systems requirements standards, Type B management systems guidelines standards, and Type C management systems related standards. The consensus is to use ISO Guide 72 Guidelines for the justification and development of management system sta
31、ndards. Accordingly, this International Standard is Type C, a management systems related standard, as neither requirements are defined nor guidelines are given. INTERNATIONAL STANDARD ISO/IEC 19796-3:2009(E) ISO/IEC 2009 All rights reserved 1Information technology Learning, education and training Qu
32、ality management, assurance and metrics Part 3: Reference methods and metrics 1 Scope This part of ISO/IEC 19796 extends the “reference framework for the description of quality approaches” (RFDQ) defined in ISO/IEC 19796-1 by providing a harmonized description of the methods and metrics required to
33、implement quality management and quality assurance systems for stakeholders designing, developing, or utilizing information technology systems used for learning, education, and training. Activities such as quality planning, quality control, and quality improvement are important for quality managemen
34、t implementations. While these three activities are focused on products, processes and their development, quality assurance is focused more on confirmation and indication for internal and external stakeholders. It should be noted that the reference methods and metrics for this part of ISO/IEC 19796
35、include issues related to the implementation of quality management and assurance systems for information technologies that are used for learning, education, and training. This part of ISO/IEC 19796 is to be used to help identify methods and metrics to implement a quality assurance and management sys
36、tem of an IT system used for learning, education, and training. For example, it may be used for quality management systems that help to verify items such as IT system effectiveness, compliance with quality objectives including purposes, customer satisfaction, training in the use of the IT system, co
37、mplaints handling, and auditing. NOTE While not included in the normative references of this part of ISO/IEC 19796 it may be helpful for ITLET (Information Technology for Learning, Education, and Training) stakeholders interested in quality issues to refer to the related standards listed in the bibl
38、iography for further information regarding quality management and quality assurance. During the implementation of quality management and quality assurance systems, using specific methods and metrics are indispensable for the exchange, purchase, management, and archiving of learning courses, systems,
39、 and/or services. Involving all stakeholders (e.g., developers, administrators, government, providers, teachers, and learners) within a framework that is open and inclusive will help to ensure that information technologies for learning, education, and training are both effective for and appropriate
40、to learning and teaching needs. This part of ISO/IEC 19796 has the following components, which are developed in order to indicate and communicate quality approaches. The reference model for methods provides an interoperable formalized description of methods that can be easily implemented and adopted
41、 into a quality management system to realize and facilitate quality management. The reference model for metrics provides an interoperable formalized description of metrics that can be easily implemented and adopted into a quality management system to measure quality in a comparable way. A collection
42、 of methods that can be used to manage and assure quality in different contexts. It supports stakeholders to implement concrete actions to achieve (a set of) quality objectives. ISO/IEC 19796-3:2009(E) 2 ISO/IEC 2009 All rights reserved A collection of metrics and indicators that can be used to meas
43、ure quality in processes, products, components, and services. It supports stakeholders to implement measures (e.g. indicators) for specific quality objectives. Annex A (informative) contains examples of how this part of ISO/IEC 19796 can be used and provides a suggested format to describe a quality
44、method. Annex B (informative) contains examples of how this part of ISO/IEC 19796 can be used and provides a suggested format to describe a quality metric. 2 Conformance The objective of this part of ISO/IEC 19796 is to extend the RFDQ (defined by ISO/IEC 19796-1) leading to a complete quality syste
45、m that supports stakeholders to implement quality management and quality assurance systems. For each quality management and assurance process a set of potential methods and metrics that can be used at the development of individual quality approaches needs to be specified. The harmonized description
46、of these methods and metrics is needed to provide clear understanding, mutual communication and agreement. The collections of methods and metrics such as quality approaches should be extensible. A method or metric is conformant if it uses the corresponding reference model (i.e. Clause 7 for methods
47、and Clause 8 for metrics) or an instantiation in description format (as noted in Table 1 for methods and Table 2 for metrics). A conforming description may contain descriptions of processes or approaches in addition to what is included in this part of ISO/IEC 19796. In other words, it is intended to
48、 be extensible and may contain additional data elements. If it exists in the reference methods or metrics collection then a reference should be provided to facilitate verification that it is conformant. 3 Normative references The following referenced documents are indispensable for the application o
49、f this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies. ISO/IEC TR 9126-2:2003, Software engineering Product quality Part 2: External metrics ISO/IEC 19796-1:2005, Information technology Learning, education and training Quality management, assurance and metrics Part 1: General approach 4 Terms and def
