1、Reference numberISO/IEC 14598-6:2001(E)ISO/IEC 2001Software engineering Productevaluation Part 6:Documentation of evaluation modulesIngnierie du logiciel valuation du produit Partie 6: Documentation des modules dvaluationNational Standard of CanadaCAN/CSA-ISO/IEC 14598-6:02(ISO/IEC 14598-6:2001)Inte
2、rnational Standard ISO/IEC 14598-6:2001 (first edition, 2001-06-01) has been adopted withoutmodification (IDT) as CSA Standard CAN/CSA-ISO/IEC 14598-6:02, which has been approved as a NationalStandard of Canada by the Standards Council of Canada.ISBN 1-55324-911-9 December 2002The Canadian Standards
3、 Association (CSA), The Standards Council of Canada is theunder whose auspices this National Standard has been coordinating body of the National Standards system, produced, was chartered in 1919 and accredited by a federation of independent, autonomousthe Standards Council of Canada to the National
4、organizations working towards the furtherStandards system in 1973. It is a not-for-profit, development and improvement of voluntarynonstatutory, voluntary membership association standardization in the national interest.engaged in standards development and certification The principal objects of the C
5、ouncil are to foster activities. and promote voluntary standardization as a means CSA standards reflect a national consensus of of advancing the national economy, benefiting theproducers and users including manufacturers, health, safety, and welfare of the public, assisting consumers, retailers, uni
6、ons and professional and protecting the consumer, facilitating domestic organizations, and governmental agencies. The and international trade, and furthering internationalstandards are used widely by industry and commerce cooperation in the field of standards.and often adopted by municipal, provinci
7、al, and A National Standard of Canada is a standard whichfederal governments in their regulations, particularly has been approved by the Standards Council ofin the fields of health, safety, building and Canada and one which reflects a reasonableconstruction, and the environment. agreement among the
8、views of a number of capableIndividuals, companies, and associations across individuals whose collective interests provide to theCanada indicate their support for CSAs standards greatest practicable extent a balance ofdevelopment by volunteering their time and skills to representation of producers,
9、users, consumers, andCSA Committee work and supporting the Associations others with relevant interests, as may be appropriateobjectives through sustaining memberships. The to the subject in hand. It normally is a standardmore than 7000 committee volunteers and the 2000 which is capable of making a s
10、ignificant and timelysustaining memberships together form CSAs total contribution to the national interest.membership from which its Directors are chosen. Approval of a standard as a National Standard ofSustaining memberships represent a major source of Canada indicates that a standard conforms to t
11、heincome for CSAs standards development activities. criteria and procedures established by the StandardsThe Association offers certification and testing Council of Canada. Approval does not refer to theservices in support of and as an extension to its technical content of the standard; this remains
12、thestandards development activities. To ensure the continuing responsibility of the accreditedintegrity of its certification process, the Association standards-development organization.regularly and continually audits and inspects products Those who have a need to apply standards arethat bear the CS
13、A Mark. encouraged to use National Standards of CanadaIn addition to its head office and laboratory complex whenever practicable. These standards are subject in Toronto, CSA has regional branch offices in major to periodic review; therefore, users are cautioned centres across Canada and inspection a
14、nd testing to obtain the latest edition from the organizationagencies in eight countries. Since 1919, the preparing the standard.Association has developed the necessary expertise to The responsibility for approving National Standards meet its corporate mission: CSA is an independent of Canada rests
15、with theservice organization whose mission is to provide an Standards Council of Canadaopen and effective forum for activities facilitating the 270 Albert Street, Suite 200exchange of goods and services through the use of Ottawa, Ontario, K1P 6N7standards, certification and related services to meet
16、Canadanational and international needs.For further information on CSA services, write toCanadian Standards Association5060 Spectrum Way, Suite 100Mississauga, Ontario, L4W 5N6CanadaAlthough the intended primary application of this Standard is stated in its Scope, it is importantto note that it remai
17、ns the responsibility of the users to judge its suitability for their particular purpose.Registered trade-mark of Canadian Standards AssociationCAN/CSA-ISO/IEC 14598-6:02 Part 6: Documentation of evaluation modulesSoftware engineering Product evaluation December 2002 Canadian Standards Association C
18、SA/1CAN/CSA-ISO/IEC 14598-6:02Software engineering Productevaluation Part 6: Documentation of evaluationmodulesCSA PrefaceStandards development within the Information Technology sector is harmonized with internationalstandards development. Through the CSA Technical Committee on Information Technolog
19、y (TCIT),Canadians serve as the Canadian Advisory Committee (CAC) on ISO/IEC Joint Technical Committee 1 onInformation Technology (ISO/IEC JTC1) for the Standards Council of Canada (SCC), the ISO memberbody for Canada and sponsor of the Canadian National Committee of the IEC. Also, as a member of th
20、eInternational Telecommunication Union (ITU), Canada participates in the International Telegraph andTelephone Consultative Committee (ITU-T).This International Standard was reviewed by the CSA TCIT under the jurisdiction of the StrategicSteering Committee on Information Technology and deemed accepta
21、ble for use in Canada. (Acommittee membership list is available on request from the CSA Project Manager.) From time to time,ISO/IEC may publish addenda, corrigenda, etc. The CSA TCIT will review these documents for approvaland publication. For a listing, refer to the CSA Information Products catalog
22、ue or CSA Info Update orcontact a CSA Sales representative. This Standard has been formally approved, without modification, bythe Technical Committee and has been approved as a National Standard of Canada by the StandardsCouncil of Canada.December 2002 Canadian Standards Association 2002All rights r
23、eserved. No part of this publication may be reproduced in any form whatsoever without the prior permissionof the publisher. ISO/IEC material is reprinted with permission. Where the words “this International Standard” appearin the text, they should be interpreted as “this National Standard of Canada”
24、. Inquiries regarding this National Standard of Canada should be addressed to Canadian Standards Association 5060 Spectrum Way, Suite 100, Mississauga, Ontario, Canada L4W 5N6 1-800-463-6727 416-747-4044www.csa.caReference numberISO/IEC 14598-6:2001(E)ISO/IEC 2001INTERNATIONALSTANDARDISO/IEC14598-6F
25、irst edition2001-06-01Software engineering Productevaluation Part 6:Documentation of evaluation modulesIngnierie du logiciel valuation du produit Partie 6: Documentation des modules dvaluationISO/IEC 14598-6:2001(E)PDF disclaimerThis PDF file may contain embedded typefaces. In accordance with Adobes
26、 licensing policy, this file may be printed or viewed but shall notbe edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading thisfile, parties accept therein the responsibility of not infringing Adobes licensing policy. The
27、 ISO Central Secretariat accepts no liability in thisarea.Adobe is a trademark of Adobe Systems Incorporated.Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameterswere optimized for printing. Every care has bee
28、n taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely eventthat a problem relating to it is found, please inform the Central Secretariat at the address given below. ISO/IEC 2001All rights reserved. Unless otherwise specified, no part of this publication may be rep
29、roduced or utilized in any form or by any means, electronicor mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISOs member bodyin the country of the requester.ISO copyright officeCase postale 56 Gb7 CH-1211 Geneva 20Tel. + 41 22
30、749 01 11Fax + 41 22 749 09 47E-mail copyrightiso.chWeb www.iso.chii ISO/IEC 2001 All rights reservedISO/IEC 14598-6:2001(E) ISO/IEC 2001 All rights reserved iiiContents Page1 Scope 12 Conformance13 Normative references 14 Terms and definitions .25 The evaluation module concept.26 Format for documen
31、tation of an evaluation module36.1 EM0 : Foreword and introduction 36.1.1 Foreword.36.1.2 Introduction36.2 EM1 : Scope36.2.1 Characteristics.36.2.2 Level of evaluation 36.2.3 Techniques.46.2.4 Applicability46.3 EM2 : References.46.4 EM3 : Terms and definitions.46.5 EM4 : Inputs and metrics 46.5.1 In
32、put for the evaluation .46.5.2 Data elements.56.5.3 Metrics and measures .56.6 EM5 : Interpretation of results56.6.1 Mapping of measures56.6.2 Reporting56.7 EMA : Application procedure .56.7.1 Definition of Technical Terms Used.56.7.2 Resources Required56.7.3 Evaluation instructions .66.7.4 Document
33、ation.6Annex A (informative) Development of evaluation modules 7Annex B (informative) Example of an evaluation module Fault density.8Annex C (informative) Example of an evaluation module Functionality 12Annex D (informative) Example of an evaluation module Usability and Quality in Use .25Bibliograph
34、y31ISO/IEC 14598-6:2001(E)iv ISO/IEC 2001 All rights reservedForewordISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission)form the specialized system for worldwide standardization. National bodies that are members of ISO or IECparticipate in
35、 the development of International Standards through technical committees established by therespective organization to deal with particular fields of technical activity. ISO and IEC technical committeescollaborate in fields of mutual interest. Other international organizations, governmental and non-g
36、overnmental, inliaison with ISO and IEC, also take part in the work.International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3.In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.Draft Internati
37、onal Standards adopted by the joint technical committee are circulated to national bodies for voting.Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote.Attention is drawn to the possibility that some of the elements of this part of ISO/I
38、EC 14598 may be the subject ofpatent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.International Standard ISO/IEC 14598-6 was prepared by Joint Technical Committee ISO/IEC JTC 1, Informationtechnology, Subcommittee SC 7, Software engineering.ISO/IEC
39、14598-6 is intended for use in conjunction with ISO/IEC 9126-1 (in preparation) which will replaceISO/IEC 9126:1991.ISO/IEC 14598 consists of the following parts, under the general title Software engineering Product evaluation:Gbe Part 1: General overviewGbe Part 2: Planning and managementGbe Part 3
40、: Process for developersGbe Part 4: Process for acquirersGbe Part 5: Process for evaluatorsGbe Part 6: Documentation of evaluation modulesAnnexes A, B, C and D of this part of ISO/IEC 14598 are for information only.ISO/IEC 14598-6:2001(E) ISO/IEC 2001 All rights reserved vIntroductionSoftware produc
41、t evaluation depends on a set of evaluation techniques and metrics that provide information aboutthe quality characteristics of the software. Many metrics and associated methods for using the measurementresults can be used for specific software product evaluation. ISO/IEC 9126-2 and ISO/IEC 9126-3 p
42、rovide examplemetrics that correspond to one sub-characteristic. It is difficult to use these metrics consistently in an organisation.It may be necessary to develop new metrics for specific use. Therefore, it is necessary that a supporting function(see 14598-2) in the organisation specifies each met
43、ric for correct and consistent use within the organisation. Theformat for documenting a metric and associated methods, as well as guides for their use, should be standardised.The concept of an evaluation module provides a solution to this need.An evaluation module specifies the evaluation methods ap
44、plicable for evaluating a quality characteristic andidentifies the evidence it needs. It also defines the elementary evaluation procedure and the format for reporting themeasurements resulting from the application of the techniques.A consistent way of documenting evaluation modules has a number of a
45、dvantages:Gbe It provides a common reference in the description of the theoretical basis of evaluation modules.Gbe It identifies a minimal set of requirements for documenting and developing evaluation modules.Gbe It provides a necessary tool in collecting and cataloguing the large number of evaluati
46、on modules anticipated.Evaluation modules provide a flexible and structured approach to making metrics applicable for evaluatingintermediate and completed products. The use of evaluation modules produced according to this part of ISO/IEC14598 helps to ensure that software product evaluations can be
47、repeatable, reproducible and objective.The format for documenting an evaluation module takes into account the following:Gbe It will be applied within the context of the evaluation of software products.Gbe The format supports the need for developing new metrics with respect to state of the art.Gbe Th
48、e format provides a precise definition of metrics and their application.Gbe It provides the information needed for those who will use them for an evaluation.Annex A provides guidance for the development process for new evaluation modules.Annexes B, C and D are examples of evaluation modules.INTERNAT
49、IONAL STANDARD ISO/IEC 14598-6:2001(E) ISO/IEC 2001 All rights reserved 1Software engineering Product evaluation Part 6:Documentation of evaluation modules1 ScopeThis part of ISO/IEC 14598 defines the structure and content of the documentation to be used to describe anEvaluation Module. Evaluation modules are intended to be used within the context of the ISO/IEC 9126 and theISO/IEC 14598 multipart standards.This part of ISO/IEC 14598 is intended to be used by experts in evaluation technology such as testing laborato
