BS ISO IEC IEEE 29119-5-2016 Software and systems engineering Software testing Keyword-Driven Testing《软件和系统工程 软件测试 关键字驱动测试》.pdf

1、BS ISO/IEC/IEEE 29119-5:2016 Software and systems engineering Software testing Part 5: Keyword-Driven Testing BSI Standards Publication WB11885_BSI_StandardCovs_2013_AW.indd 1 15/05/2013 15:06BS ISO/IEC/IEEE 29119-5:2016 BRITISH STANDARD National foreword This British Standard is the UK implementati

2、on of ISO/IEC/IEEE 29119-5:2016. The UK participation in its preparation was entrusted to Technical Committee IST/15, Software and systems engineering. A list of organizations represented on this committee can be obtained on request to its secretary. This publication does not purport to include all

3、the necessary provisions of a contract. Users are responsible for its correct application. The British Standards Institution 2016. Published by BSI Standards Limited 2016 ISBN 978 0 580 83376 2 ICS 35.080 Compliance with a British Standard cannot confer immunity from legal obligations. This British

4、Standard was published under the authority of the Standards Policy and Strategy Committee on 30 November 2016. Amendments/Corrigenda issued since publication Date Text affectedBS ISO/IEC/IEEE 29119-5:2016 Software and systems engineering Software testing Part 5: Keyword-Driven Testing Ingnierie du l

5、ogiciel et des systmes Essais du logiciel Partie 5: Essais axs sur des mots-cls INTERNATIONAL STANDARD ISO/IEC/ IEEE 29119-5 Reference number ISO/IEC/IEEE 29119-5:2016(E) First edition 2016-11-15 ISO/IEC 2016 IEEE 2016 BS ISO/IEC/IEEE 29119-5:2016 ISO/IEC 2016 All rights reserved ii IEEE 2016 All ri

6、ghts reserved ISO/IEC/IEEE 29119-5:2016(E) COPYRIGHT PROTECTED DOCUMENT ISO/IEC 2016, Published in Switzerland IEEE 2016 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, includi

7、ng photocopying, or posting on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below or ISOs member body in the country of the requester. ISO copyright office Institute of Electrical and Electronics Engineers, Inc Ch. de Bland

8、onnet 8 CP 401 3 Park Avenue, New York CH-1214 Vernier, Geneva, Switzerland NY 10016-5997, USA Tel. +41 22 749 01 11 Fax +41 22 749 09 47 copyrightiso.org stds.iprieee.org www.iso.org www.ieee.org IEC Central Office 3, rue de Varemb CH- 1211 Geneva 20 Switzerland E-mail inmailiec.ch Web www.iec.ch B

9、S ISO/IEC/IEEE 29119-5:2016 ISO/IEC/IEEE 29119-5:2016(E) Contents Page 1 Scope 1 2 Conformance 1 2.1 Intended usage .1 2.2 Full conformance .1 2.3 Tailored conformance .2 3 Normative references 2 4 Terms and definitions .2 5 Introduction to Keyword-Driven Testing .4 5.1 Overview .4 5.2 Layers in Key

10、word-Driven Testing .7 5.2.1 Overview .7 5.2.2 Domain layer 8 5.2.3 Test interface layer 9 5.2.4 Multiple layers 9 5.3 Types of keywords 10 5.3.1 Simple keywords. 10 5.3.2 Composite keywords 11 5.3.3 Navigation/interaction (input) and verification (output) . 14 5.3.4 Keywords and test result . 14 5.

11、4 Keywords and Data. 15 6 Application of Keyword-Driven Testing . 16 6.1 Overview 16 6.2 Identifying keywords 16 6.3 Composing test cases 17 6.4 Keywords and data-driven testing 18 6.5 Modularity and refactoring 18 6.6 Keyword-Driven Testing in the Test Design Process . 19 6.6.1 Overview 19 6.6.2 TD

12、1 Identify Feature Sets . 20 6.6.3 TD2 Derive Test Conditions . 20 6.6.4 TD3 Derive Test Coverage Items . 20 6.6.5 TD4 Derive Test Cases . 21 6.6.6 TD5 Assemble Test Sets 22 6.6.7 TD6 Derive Test Procedures 22 6.7 Converting non keyword-driven test cases into Keyword-Driven Testing . 22 7 Keyword-Dr

13、iven Testing Frameworks 22 7.1 Overview 22 7.2 Components of a Keyword-Driven Testing framework. 23 7.2.1 Overview 23 7.2.2 Keyword-driven Editor . 25 7.2.3 Decomposer 26 7.2.4 Data sequencer . 26 7.2.5 Manual test assistant . 26 7.2.6 Tool bridge 26 7.2.7 Test execution environment and execution en

14、gine 26 7.2.8 Keyword library . 27 7.2.9 Data 27 7.2.10 Script repository . 27 7.3 Basic attributes of the Keyword-Driven Testing framework 27 7.3.1 General information on basic attributes 27 7.3.2 General attributes . 27 7.3.3 Dedicated keyword-driven editor (tool) 28 7.3.4 Decomposer and data sequ

15、encer . 29 ISO/IEC 2016 All rights reserved IEEE 2016 All rights reserved iiiBS ISO/IEC/IEEE 29119-5:2016 ISO/IEC/IEEE 29119-5:2016(E) 7.3.5 Manual test assistant (tool) 29 7.3.6 Tool bridge . 29 7.3.7 Test execution engine . 29 7.3.8 Keyword library . 30 7.3.9 Script repository 30 7.4 Advanced attr

16、ibutes of frameworks 30 7.4.1 General information on advanced attributes 30 7.4.2 General attributes 30 7.4.3 Dedicated keyword-driven editor (tool) . 31 7.4.4 Decomposer and data sequencer 31 7.4.5 Manual test assistant 31 7.4.6 Tool bridge . 31 7.4.7 Test execution environment and execution engine

17、 . 32 7.4.8 Keyword library . 33 7.4.9 Test data support 33 7.4.10 Script repository 33 8 Data interchange . 33 Annex A (normative) Conventions 34 Annex B (informative) Benefits and Issues of Keyword-Driven Testing . 35 B.1 General benefits of Keyword-Driven Testing . 35 B.2 Benefits of Keyword-Driv

18、en Testing for test automation 35 B.3 Benefits of Keyword-Driven Testing for manual testing . 36 B.4 Possible issues with Keyword-Driven Testing . 36 Annex C (informative) Getting started with Keyword-Driven Testing . 37 C.1 General . 37 C.2 Identifying Keywords 37 C.3 Composing test cases 38 Annex

19、D (informative) Roles and Tasks . 39 D.1 Overview Roles and Tasks 39 D.2 Domain expert 39 D.3 Test designer . 39 D.4 Test automation expert . 40 Annex E (informative) Basic keywords . 41 E.1 Overview . 41 E.2 Basic keywords for a GUI . 41 E.3 Example application of basic keywords . 45 Annex F (infor

20、mative) Examples. 49 F.1 Overview . 49 F.2 Example: test procedure from ISO/IEC/IEEE 29119-3 49 F.3 Example: Test of shopping procedure with low-level keywords 51 F.4 Example for calculator with low-level keywords 52 F.5 Example for calculator with domain level keywords . 52 Annex G Bibliography 54

21、iv ISO/IEC 2016 All rights reserved IEEE 2016 All rights reservedBS ISO/IEC/IEEE 29119-5:2016 ISO/IEC/IEEE 29119-5:2016(E) ISO/IEC 2016 All rights reserved IEEE 2016 All rights reserved v Foreword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Co

22、mmission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of Internat ional Standards through techni cal committees established by the respective organization to deal with particular fields of technical activity

23、. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non- governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committe

24、e, ISO/IEC JTC 1. IEEE Standards documents are developed within t he IEEE Societies and the Standards Coordinating Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its standards through a consensus development process, approved by the American National Standa

25、rds Institute, which brings together volunteers representing varied viewpoints and interests to achieve the final product. Volunteers are not necessarily members of the Institute and serve without compensation. While the IEEE administers the process and establishes rules to promote fairness in the c

26、onsensus development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of the information contained in its standards. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of ISO/IEC JTC 1 is to pr

27、epare Intern ational Standards. Draft International Standards adopted by the joint technical committee are circulated to na tional bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is called to the possibil

28、ity that implementation of this standard may require the use of subject matter covered by patent rights. By publication of this standard, no position is taken with respect to the existence or validity of any patent rights in conn ection therewith. ISO/IEEE is not re sponsible for identifying essenti

29、al patents or patent claims for which a license may be requir ed, for conducting inquiries into the legal validity or scope of patents or patent claims or determining whether any licensing terms or conditions provided in connection with submission of a Letter of Assurance or a Patent Statement and L

30、icensing Declaration Form, if any, or in any licensing agreements are reasonable or non-discriminatory. Users of this standard are expressly advised that determination of the valid ity of any patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Further in

31、formation may be obtained from ISO or the IEEE Standards Association. ISO/IEC/IEEE 29119-5 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Software in particular TD4 (Derive Test Cases), TD5

32、 (Assemble Test Sets) and TD6 (Derive Test Procedures) as here applied to Keyword-Driven Testing. The templates and examples of test documentation as defined in ISO/IEC/IEEE 29119-3 are also applicable to ISO/IEC/IEEE 29119-5. Software test design tech niques that can be used dur ing test design are

33、 defined in ISO/IEC/IEEE 29119-4 Test Techniques. The application of ISO/IEC/IEEE 29119-4 is assumed when designing test cases that are then described by keywords according to ISO/IEC/IEEE 29119-5. This part of ISO/IEC/IEEE 29119-5 has the following structure: terms and definitions can be found in c

34、lause 4 an introduction to Keyword-Driven Testing is given in clause 5 the application of Keyword-Driven Testing is explained in clause 6 frameworks for Keyword-Driven Testing are described in clause 7 data interchange is covered in clause 8 Annex A states naming conventions for keywords Annex B nam

35、es benefits that can be achieved with Keyword-Driven Testing Annex C gives advice on how interested parties wanting to use Keyword-Driven Testing can start Annex D describes roles that can be used in Keyword-Driven Testing Annex E contains examples of basic keywords that can be used to create test c

36、ases Annex F contains examples for keyword test cases BS ISO/IEC/IEEE 29119-5:2016 Software and Systems Engineering Software Testing Part 5: Keyword-Driven Testing 1 Scope This part of ISO/IEC/IEEE 29119 defines an efficient and consistent solution for Keyword-Driven Testing by: giving an introducti

37、on to Keyword-Driven Testing; providing a reference approach to implement Keyword-Driven Testing; defining requirements on frameworks for Keyword-Driven Testing to enable testers to share their work items, such as test cases, test data, keywords, or complete test specifications; defining requirement

38、s for tools that support Keyw ord-Driven Testing. These requirements could apply to any tool that supports the Keyword-Driven appr oach (e.g., test automation, test design and test management tools); defining interfaces and a common data exchange format to ensure that tools from different vendors ca

39、n exchange their data (e.g. test cases, test data and test results); defining levels of hierarchical keywords, and advis ing use of hierarchical keywords. This includes describing specific types of keywords (e.g. keywords for navigation or for checking a value) and when to use “flat“ structured keyw

40、ords; providing an initial list of example generic techni cal (low-level) keywords, such as “inputData“ or “checkValue“. These keywords can be used to specify test cases on a technical level, and may be combined to create business-level keywords as required. NOTE This standard is applicable to all t

41、hose who want to create keyword-driven test specifications, create corresponding frameworks, or build test automation based on keywords. 2 Conformance 2.1 Intended usage The requirements in ISO/IEC/IEEE 29119-5 are contained in Clause 7 and in Annex A. ISO/IEC/IEEE 29119- 5 provides requirements on

42、frameworks supporting the application of Keyword-Driven Testing. It is recognized that particular projects or organizations may not need to use all of the components defined in this standard. Therefore, implementation of ISO/IEC/IEEE 29119-5 typically involves selecting a set of components or parts

43、of components suitable for the organization or project. There are two ways that an organization can claim to conform to the provisions of this standard. The organization or individual shall assert whether full or tailored conformance to this standard is claimed. 2.2 Full conformance Full conformance is achieved by demonstrating that all of the Keyword-Driven Testing requirements (i.e. shall statements) defined in ISO/IEC/IEEE 29119-5 have been satisfied. INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-5:2016(E) ISO/IEC 2016 All rights reserved IEEE 2016 All rights reserved 1