BS ISO IEC IEEE 29119-1-2013 Software and systems engineering Software testing Concepts and definitions《软件和系统工程 软件测试 概念和定义》.pdf

1、BSI Standards Publication BS ISO/IEC/IEEE 29119-1:2013 Software and systems engineering Software testing Part 1: Concepts and definitionsBS ISO/IEC/IEEE 29119-1:2013 BRITISH STANDARD National foreword This British Standard is the UK implementation of ISO/IEC/IEEE 29119-1:2013. It supersedes BS 7925-

2、1:1998 which is withdrawn. The UK participation in its preparation was entrusted to Technical Committee IST/15, Software and systems engineering. A list of organizations represented on this committee can be obtained on request to its secretary. This publication does not purport to include all the ne

3、cessary provisions of a contract. Users are responsible for its correct application. The British Standards Institution 2013. Published by BSI Standards Limited 2013 ISBN 978 0 580 73279 9 ICS 35.080 Compliance with a British Standard cannot confer immunity from legal obligations. This British Standa

4、rd was published under the authority of the Standards Policy and Strategy Committee on 30 September 2013. Amendments issued since publication Date Text affectedBS ISO/IEC/IEEE 29119-1:2013Reference number ISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 IEEE 2013INTERNATIONAL STANDARD ISO/IEC/ IEEE 29119-1

5、 First edition 2013-09-01 Software and systems engineering Software testing Part 1: Concepts and definitions Ingnierie du logiciel et des systmes Essais du logiciel Partie 1: Concepts et dfinitions BS ISO/IEC/IEEE 29119-1:2013 ISO/IEC/IEEE 29119-1:2013(E) COPYRIGHT PROTECTED DOCUMENT ISO/IEC 2013 IE

6、EE 2013 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from ISO, IEC or IEEE at the respective address below. ISO cop

7、yright office IEC Central Office Institute of Electrical and Electronics Engineers, Inc. Case postale 56 3, rue de Varemb 3 Park Avenue, New York CH-1211 Geneva 20 CH-1211 Geneva 20 NY 10016-5997, USA Tel. + 41 22 749 01 11 Switzerland E-mail stds.iprieee.org Fax + 41 22 749 09 47 E-mail inmailiec.c

8、h Web www.ieee.org E-mail copyrightiso.org Web www.iec.ch Web www.iso.org Published in Switzerland ii ISO/IEC 2013 All rights reserved IEEE 2013 All rights reservedBS ISO/IEC/IEEE 29119-1:2013 ISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved iiiContents Pag

9、e Foreword . v Introduction vi 1 Scope 1 2 Conformance . 1 3 Normative references 1 4 Terms and definitions . 1 5 Software Testing Concepts 12 5.1 Introduction to Software Testing . 12 5.1.1 The Role of Testing in Verification and Validation 14 5.1.2 Exhaustive Testing 14 5.1.3 Testing as a Heuristi

10、c . 14 5.2 Software Testing in an Organizational and Project Context . 14 5.2.1 The Test Process . 17 5.3 Generic Testing Processes in the Software Life cycle 19 5.3.1 Development Project Sub-processes and their Results . 20 5.3.2 On-going Maintenance and its Results . 21 5.3.3 Support Processes for

11、 the Software Development Life Cycle . 22 5.4 Risk-based Testing 24 5.4.1 Using Risk-Based Testing in the Organizational Test Process 25 5.4.2 Using Risk-Based Testing in the Test Management processes . 25 5.4.3 Using Risk-Based Testing in the Dynamic Testing processes 25 5.5 Test Sub-process 26 5.5

12、.1 Test Objectives 26 5.5.2 Test Item . 27 5.5.3 Testing of Quality Characteristics . 27 5.5.4 Test Basis . 28 5.5.5 Retesting and Regression Testing 29 5.5.6 Test Design Techniques . 29 5.6 Test Practices 30 5.6.1 Introduction 30 5.6.2 Requirements-Based Testing . 31 5.6.3 Model-Based Testing 31 5.

13、6.4 Mathematical-Based Testing 32 5.6.5 Experience-Based Testing . 32 5.6.6 Scripted and Unscripted Testing . 33 5.7 Automation in Testing . 34 5.8 Defect Management . 34 Annex A (informative) The Role of Testing in Verification and Validation 35 Annex B (informative) Metrics and Measures 36 B.1 Met

14、rics and Measures . 36 Annex C (informative) Testing in Different Life Cycle Models . 37 C.1 Overview . 37 C.2 Agile Development and Testing . 37 C.2.1 Agile Development Principles 37 C.2.2 Test Management in Agile Development 38 C.2.3 Test Sub-processes in Agile Development. 39 C.3 Sequential Devel

15、opment and Testing . 40 C.3.1 Sequential Development Principles 40 BS ISO/IEC/IEEE 29119-1:2013 ISO/IEC/IEEE 29119-1:2013(E) iv ISO/IEC 2013 All rights reserved IEEE 2013 All rights reservedC.3.2 Test Management in Sequential Development .40 C.3.3 Test Sub-processes in Sequential Development .41 C.4

16、 Evolutionary Development and Testing 41 C.4.1 Evolutionary Development Principles .41 C.4.2 Test Management in Evolutionary Development 42 C.4.3 Test Sub-processes in Evolutionary Development 42 Annex D (informative) Detailed Test Sub-process Examples .44 D.1 Overview .44 D.2 Acceptance Test Sub-pr

17、ocess .45 D.3 Detailed Design Test Sub-process .45 D.4 Integration Test Sub-process .46 D.5 Performance Test Sub-process 48 D.6 Regression Test Sub-process 49 D.7 Retest Test Sub-process .51 D.8 Story Set Test Sub-process 51 D.9 Story Test Sub-process 51 D.10 System Test Sub-process .52 D.11 Compone

18、nt Test Sub-process 53 Annex E (informative) Roles and Responsibilities in Testing 54 E.1 Testing Roles .54 E.2 Communication in Testing 54 E.3 Independence in Testing .54 Bibliography 56 BS ISO/IEC/IEEE 29119-1:2013 ISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights

19、reserved vForeword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards thro

20、ugh technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non- governmental, in liaison with ISO and IEC, als

21、o take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating Committees of the IEEE Standards Association (IEEE-SA) Standards B

22、oard. The IEEE develops its standards through a consensus development process, approved by the American National Standards Institute, which brings together volunteers representing varied viewpoints and interests to achieve the final product. Volunteers are not necessarily members of the Institute an

23、d serve without compensation. While the IEEE administers the process and establishes rules to promote fairness in the consensus development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of the information contained in its standards. International Standards ar

24、e drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of ISO/IEC JTC 1 is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International S

25、tandard requires approval by at least 75 % of the national bodies casting a vote. Attention is called to the possibility that implementation of this standard may require the use of subject matter covered by patent rights. By publication of this standard, no position is taken with respect to the exis

26、tence or validity of any patent rights in connection therewith. ISO/IEEE is not responsible for identifying essential patents or patent claims for which a license may be required, for conducting inquiries into the legal validity or scope of patents or patent claims or determining whether any licensi

27、ng terms or conditions provided in connection with submission of a Letter of Assurance or a Patent Statement and Licensing Declaration Form, if any, or in any licensing agreements are reasonable or non-discriminatory. Users of this standard are expressly advised that determination of the validity of

28、 any patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Further information may be obtained from ISO or the IEEE Standards Association. ISO/IEC/IEEE 29119-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC

29、 7, Software and systems engineering, in cooperation with the Software as well as how automation can be used to support testing. The involvement of testing in defect management is also discussed. Annex A describes the role of testing within the larger scope of verification and validation. Annex B pr

30、ovides a brief introduction to metrics used to monitor and control testing. Annex C contains a set of examples showing how to apply the standard in different life cycle models. Annex D provides examples on detailed test sub-processes. Annex E provides additional information on the roles and responsi

31、bilities typically encountered in test groups and tester independence. Finally, the Bibliography is at the end of the document. Note that Title Case is used throughout this part of ISO/IEC/IEEE 29119 to denote processes and documents that are specified in ISO/IEC/IEEE 29119-2 and ISO/IEC/IEEE 29119-

32、3 (e.g. Test Planning Process, Test Plan), whereas lowercase letters are used for documents that form parts of other documents (e.g. the project test strategy is an element of the Project Test Plan). The test process model that the ISO/IEC/IEEE 29119 series of software testing standards are based on

33、 is defined in detail in ISO/IEC/IEEE 29119-2 Test Processes. ISO/IEC/IEEE 29119-2 covers the software testing processes at the organizational level, test management level and for dynamic test levels. Testing is the primary approach to risk treatment in software development. This standard defines a

34、risk-based approach to testing. Risk-based testing is a recommended approach to strategizing and managing testing that allows testing to be prioritized and focused. Templates and examples of test documentation that are produced during the testing process are defined in ISO/IEC/IEEE 29119-3 Test Docu

35、mentation. Software testing techniques that can be used during testing are defined in ISO/IEC/IEEE 29119-4 Test Techniques. Together, this series of international standards aims to provide stakeholders with the ability to manage and perform software testing in any organization. BS ISO/IEC/IEEE 29119

36、-1:2013 INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-1:2013(E) ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved 1Software and systems engineering Software testing Part 1: Concepts and definitions 1 Scope This part of ISO/IEC/IEEE 29119 specifies definitions and concepts in software testin

37、g. It provides definitions of testing terms and discussion of concepts key to the understanding of the ISO/IEC/IEEE 29119 series of software testing international standards. 2 Conformance ISO/IEC/IEEE 29119-1 is informative and no conformance with it is required. The ISO/IEC/IEEE 29119 software test

38、ing series of standards contain three standards where conformance may be claimed: test processes; test documentation; test techniques. Conformance is addressed in ISO/IEC/IEEE 29119-2, ISO/IEC/IEEE 29119-3 and ISO/IEC/IEEE 29119-4. 3 Normative references This document does not require the use of any

39、 normative references. Standards useful for the implementation and interpretation of this part of ISO/IEC/IEEE 29119 are listed in the Bibliography. 4 Terms and definitions For the purposes of this document, the terms and definitions given in ISO/IEC/IEEE 24765 and the following apply. NOTE The foll

40、owing terms and definitions are provided to assist with the understanding and readability of parts 1, 2, 3 and 4 of the ISO/IEC/IEEE 29119 Software Testing standards so some terms defined in ISO/IEC/IEEE 29119-1 will not be used in ISO/IEC/IEEE 29119-1 and will only be used in another standard in th

41、e ISO/IEC/IEEE 29119 series. Only terms critical to the understanding of these standards are included; this clause is not intended to provide a complete list of testing terms. The systems and software engineering Vocabulary ISO/IEC/IEEE 24765 should be referenced for terms not defined in this clause

42、. This source is available at the following web site: http:/puter.org/sevocab. BS ISO/IEC/IEEE 29119-1:2013 ISO/IEC/IEEE 29119-1:2013(E) 2 ISO/IEC 2013 All rights reserved IEEE 2013 All rights reserved4.1 accessibility testing type of usability testing used to measure the degree to which a test item

43、 can be operated by users with the widest possible range of characteristics and capabilities 4.2 actual results set of behaviours or conditions of a test item, or set of conditions of associated data or the test environment, observed as a result of test execution EXAMPLE Outputs to hardware, changes

44、 to data, reports, and communication messages sent. 4.3 backup and recovery testing type of reliability testing that measures the degree to which system state can be restored from backup within specified parameters of time, cost, completeness, and accuracy in the event of failure 4.4 black-box testi

45、ng see specification-based testing (4.39) 4.5 capacity testing type of performance efficiency testing conducted to evaluate the level at which increasing load (of users, transactions, data storage, etc.) compromises a test items ability to sustain required performance 4.6 compatibility testing type

46、of testing that measures the degree to which a test item can function satisfactorily alongside other independent products in a shared environment (co-existence), and where necessary, exchanges information with other systems or components (interoperability) 4.7 coverage item see test coverage item (4

47、.54) 4.8 decision types of statements in which a choice between two or more possible outcomes controls which set of actions will result Note 1 to entry: Typical decisions are simple selections (e.g. if-then-else), to decide when to exit loops (e.g. while-loop), and in case (switch) statements (e.g. case-1-2-3-N). 4.9 dynamic testing testing that requires the execution of the test item 4.10 endurance testing type of performance efficiency testing conducted to evaluate whether a test item can sustain a required load continuously for a specified period of time