1、BSI Standards PublicationBS 5760-24:2014Reliability of systems,equipment andcomponents Part 24: Guide to the integration ofrisk techniques in the inspection andtesting of complex systemsPublishing and copyright informationThe BSI copyright notice displayed in this document indicates when the documen
2、twas last issued. The British Standards Institution 2014Published by BSI Standards Limited 2014ISBN 978 0 580 74264 4ICS 03.120.01; 21.020; 29.020The following BSI references relate to the work on this document:Committee reference DS/1Draft for comment 13/30240790 DCPublication historyFirst publishe
3、d March 2014Amendments issued since publicationDate Text affectedBS 5760-24:2014 BRITISH STANDARDContentsForeword iiIntroduction iii1 Scope 12 Normative references 13 Terms and definitions 14 Developing a complex system 25 Risk assessment 46 Testing 57 Testing strategies 68 Testing combined complex
4、systems 99 Test documentation 910 Implementation guidance 911 System delivery 11Bibliography 13List of figuresFigure 1 Software product risk assessment workshop 10List of tablesTable 1 Prioritization scoring of likelihood and consequences of risks 5Summary of pagesThis document comprises a front cov
5、er, an inside front cover, pages i to iv, pages1 to 18, an inside back cover and a back cover.BRITISH STANDARD BS 5760-24:2014 The British Standards Institution 2014 iForewordPublishing informationThis British Standard is published by BSI Standards Limited, under licence fromThe British Standards In
6、stitution, and came into effect on 31 March 2014. It wasprepared by Technical Committee DS/1, Dependability. A list of organizationsrepresented on this committee can be obtained on request to its secretary.Relationship with other publicationsThe following parts of BS 5760 have been published or are
7、in preparation: Part 0: Guide to reliability and maintainability; Part 2: Guide to the assessment of reliability; Part 8: Guide to assessment of reliability of systems containing software; Part 10: Guide to reliability testing; Part 12: Guide to the presentation of reliability, maintainability andav
8、ailability predictions; Part 13: Guide to reliability test conditions for consumer equipment; Part 18: Guide to the demonstration of dependability requirements Thedependability case; Part 24: Guide to the integration of risk techniques in the inspection andtesting of complex systems.Information abou
9、t this documentThis part of BS 5760 provides a methodology for applying risk-based techniquesto optimizing the inspection and testing of complex systems.Use of this documentAs a guide, this part of BS 5760 takes the form of guidance andrecommendations. It should not be quoted as if it were a specifi
10、cation or a codeof practice and claims of compliance cannot be made to it.Presentational conventionsThe guidance in this standard is presented in roman (i.e. upright) type. Anyrecommendations are expressed in sentences in which the principal auxiliaryverb is “should”.Commentary, explanation and gene
11、ral informative material is presented insmaller italic type, and does not constitute a normative element.Contractual and legal considerationsThis publication does not purport to include all the necessary provisions of acontract. Users are responsible for its correct application.Compliance with a Bri
12、tish Standard cannot confer immunity from legalobligations.BRITISH STANDARDBS 5760-24:2014ii The British Standards Institution 2014IntroductionThere is no universal definition of a complex system, but from a dependabilitystandpoint its most important feature is that it is composed of interconnectedp
13、arts that as a whole exhibit properties that are not easily discernible from theexplicit properties of the individual parts. This includes most systems containingsoftware.Common features of a complex system include:a) difficulty in determining the boundaries of the system;b) complex systems within t
14、he system that make up the complex system; andc) reuse of system elements from other complex systems.As projects and acquisitions become increasingly complex, companies andgovernments are challenged to find effective ways to manage them. Asprogrammes become more network-centric and complex, business
15、es are forcedto find ways to manage complexity while governments are challenged toprovide effective governance to ensure flexibility and resiliency.The systems of the 1970s were largely stand-alone, analogue and mechanicallycontrolled. In contrast the new systems address problems with more accurate,
16、reliable, interoperable and maintainable systems. Current systems are oftensoftware intensive and network enabled with on-board complex sub-systems.The arising complexities are often the result of interactions among the systemsand sub-systems and therefore cannot be tested and evaluated in isolation
17、.The live testing of complex systems is becoming increasingly expensive,particularly as many impacts and interactions result in the addition of newequipment or of complementary or adversary systems and performance. Testingregimes which seek to test every function of a system are also becomingincreas
18、ingly time-consuming due to the complexity and nature of the systems.These challenges can be mitigated, however, by the careful incorporation of risktechniques in the development, assessment and testing of complex systems,which are covered in this standard.BRITISH STANDARD BS 5760-24:2014 The Britis
19、h Standards Institution 2014 iiiBRITISH STANDARDBS 5760-24:2014This page deliberately left blankiv The British Standards Institution 20141 ScopeThis British Standard gives guidance on defining complex system requirementsand focuses on the important aspects that are needed to establish an efficientte
20、sting regime.It also gives guidance on the inspection and testing of complex systems,including the integration of risk management techniques.This British Standard applies to managers involved in the early development of acomplex system, such as project managers, requirement managers, test managersan
21、d financial controllers.NOTE This British Standard only makes reference to aspects such as riskmanagement, project management and requirements management where suchexplanations aid this guide. Full details on these subjects are not the intent of thisstandard.2 Normative referencesThe following docum
22、ents, in whole or in part, are normatively referenced in thisdocument and are indispensable for its application. For dated references, onlythe edition cited applies. For undated references, the latest edition of thereferenced document (including any amendments) applies.BS 4778-3.2 (IEC 60050-191), Q
23、uality vocabulary Availability, reliability andmaintainability terms Part 3.2: Glossary of international terms3 Terms and definitionsFor the purposes of this part of BS 5760, the terms and definitions given inBS 4778-3.2 and the following apply.3.1 business analystperson who analyses the needs of a
24、company for the future development ofsystems by defining its requirementsNOTE A business analyst is often referred to as a requirement analyst.3.2 complex systemany type of system that is composed of interconnected parts3.3 complex system requirementdesired technical or business outcome of a complex
25、 system, as defined internallywithin a company or externally by a customerNOTE This is also referred to within the standard as a requirement.3.4 customerparty requiring the complex system for implementation and use3.5 designerparty responsible for the design of a complex system3.6 developerparty res
26、ponsible for developing the complex system3.7 end-userperson from the target user group of the complex systemNOTE An end-user might be required for testing of the complex system while indevelopment.BRITISH STANDARD BS 5760-24:2014 The British Standards Institution 2014 13.8 itemsubject being conside
27、redNOTE 1 The item might be an individual part, component, device, functional unit,equipment or system, and consist of hardware, software, people or any combinationthereof.NOTE 2 The item is often comprised of elements that may each be individuallyconsidered.NOTE 3 See BS 5760-0, 2.8.3.9 item testin
28、gtest or series of tests carried out on a single part, component or element of acomplex systemNOTE This might be carried out at any stage in the complex system developmentprocess.3.10 producerparty responsible for developing the complex system3.11 revolutionary developmentsystem where very little (o
29、r none) of the design existed previouslyNOTE Identifying revolutionary development areas can assist in establishing the risklikelihood.3.12 supplierparty responsible for supplying the complex system to a customer3.13 testinspection and testing of a complex systemNOTE Due to the limitations of test s
30、cenarios (such as size, availability, cost,security, safety and environmental concerns), it can be difficult to perform testing ina real life environment and a simulated environment might need to be used.3.14 testerindividual responsible for inspecting and testing items or the whole complexsystem4 D
31、eveloping a complex system4.1 Establishing the requirementsAssessing and understanding the importance of the requirements for a complexsystem can help the customer and producer to define the framework of theproject, outline the development process and to determine the resources thatshould be allocat
32、ed to it. The customer and producer should use this frameworkto determine how and at which points the capabilities and limitations of thecomplex system are tested, and their conclusions should be recorded in a projectplan.NOTE The requirements are bespoke or tailor made if the customer is external t
33、othe company responsible for meeting the requirements. For an internal customer,such as the marketing department, the requirements are tailored with considerationto the marketing demands and the resources and time required to meet thesedemands.BRITISH STANDARDBS 5760-24:20142 The British Standards I
34、nstitution 2014Complex system requirements should be formulated, assessed, documented andconfirmed to be verifiable by the relevant parties, such as the designer, tester,business analyst and customer. Vague statements such as “a high number oftransactions” or “highly secure passwords” should be avoi
35、ded so that thedesigner and tester have a clear definition of the complex system requirements.The designer, tester, business analyst and customer should be involved inclarifying and refining the complex system requirements. This is an appropriatetime to analyse the potential consequences if a partic
36、ular requirement is notmet. Scenarios for unexpected events (beneficial, benign or negative) such ashazards, triggers, probability of occurrence and range of consequence shouldalso be covered in the complex system requirements through a risk assessment(see Clause 5).4.2 CategorizationDuring the asse
37、ssment and refining period, complex system requirements shouldbe grouped into categories depending on their function (such as performance,usability, disaster recovery and consistency of end-user experience) and whether,for example, they have an impact on the infrastructure in which the complexsystem
38、 is embedded. In some cases this might necessitate a requirement to bedivided so that it can be specifically allocated to each relevant function.NOTE To categorize the complex system requirements into functions andsub-functions, a failure mode and effects analysis can be used, see BS EN 60812 forfur
39、ther information.The system and the associated systems are linked to establish items that can bedesigned and developed as separate entities. This might result in a requirementbeing modified as the item might only fulfil one part of an overall function.4.3 Item interfaceIn addition to developing the
40、complex system requirements, the interfacesbetween items should be determined. These interfaces cover those which areinternal to the system (across items) and those which are external to thecomplex system (across the complex system and other associatedsystems/end-users to satisfy the complex system
41、requirements). The riskassessment should also cover the commonality across interfaces and thepossibilities of consolidation.4.4 System deploymentThe deployment of a complex system into an operational environment should beundertaken as a series of releases, each release progressively accommodatingmor
42、e and more functions and associated complex system requirements.For each release, a series of tests should be specified and undertaken in stagesin order to confirm correct working of the complex system.4.5 Test stagesThis stage should include the following types of test:a) item test, to confirm the
43、correct performance of discrete items;NOTE 1 Sometimes referred to as a unit test.b) integration of items test for correct passing of information between items;c) built environment test for correct configuration of the testing platform;d) complex system test reflecting the requirements and total sys
44、tem function asappropriate to test stage (i.e. some test elements might be simulated ifcertain items are in development);BRITISH STANDARD BS 5760-24:2014 The British Standards Institution 2014 3e) associated complex system test (e.g. performance test for the efficiency ofthe complex system);f) opera
45、tional readiness test prior to deployment of the complex system andany associated systems;g) commissioning test to determine the serviceable condition of the complexsystem; andh) deployment test for when placed into a real-life environment.Where different tests can be combined, they should be undert
46、aken.NOTE 2 For example, item testing and integration testing might be combined byconducting the tests in an environment containing all of the items.NOTE 3 This combined approach can be useful when dealing with progressivereleases; however, it is advisable to assess any risks associated with combini
47、ng therelevant tests (such as the difficulty of identifying the origin of a fault) and thetimescales involved before they are undertaken.5 Risk assessmentA risk assessment should be carried out during the complex system requirementdevelopment stage (4.1).Identification, analysis and evaluation of ri
48、sks should be applied to each teststage (4.5) to identify and outline potential consequences to:a) the customer, through loss of business, total complex systems failure ortemporary complex systems failure; andb) the designer/developer/supplier, through minimizing development costs,testing early to i
49、dentify risks and faults that can be more expensive torectify later.NOTE 1 In contrast, testing undertaken late in the development stage where timeconstraints are at their most stretched could result in a pressure to complete, and ifnot done correctly, could lead to loss of reputation and future orders.The consequences of the identified risks should be grouped into categories suchas high impact, medium impact and low impact, as defined, agreed anddocumented by the relevant parties such as the supplier and the customer.Ideally, there should be a reaso
