REG NASA-GB-A301-1990 SOFTWARE QUALITY ASSURANCE AUDITS GUIDEBOOK.pdf

1、SOFTWARE QUALITY ASSURANCE AUDITSGUIDEBOOKNOVEMBER 1990Office of Safety and Mission QualitySoftware Management and Assurance Program (SMAP)Guidebook Working GroupProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Provided by IHSNot for ResaleNo reproduc

2、tion or networking permitted without license from IHS-,-,-PREFACEThe growth in cost and importance of software has made it necessary for NASA toestablish software standards and guidance for use in the development and acquisitionof software. The Software Management and Assurance Program (SMAP), estab

3、lishedin the Office of Safety and Mission Quality of NASA Headquarters, focuses theNASA activities in defining standards for software management, engineering, andassurance. One of the products of the SMAP is a series of guidebooks that defines aNASA concept of the processes that are used to manage,

4、engineer, and assuresoftware.There are three levels of SMAP software guidebooks. Level 1 is reserved for a highlevel guidebook that will describe the NASA view of software and the SMAP. Therewill be three Level 2 guidebooks that will provide an overall picture of the conceptsand practices of NASA in

5、 software management, assurance, and engineering. Level 3guidebooks will focus on specific activities that fall within each of those threesoftware disciplines, and provide more detailed information for the manager and/orpractitioner.This is the Level 3 Software Quality Assurance Audits Guidebook tha

6、t describessoftware quality assurance audits in a way that is compatible with practices at NASACenters. For a more generalized view of how software quality assurance audits relateto Software Assurance, refer to the Level 2 Software Assurance Guidebook,document number SMAP-GB-A201.LEVEL 2SOFTWAREASSU

7、RANCEPLANNEDProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-SOFTWARE QUALITY ASSURANCE AUDITSGUIDEBOOKApprovalsLawrence E. HyattChairman, Software Assur

8、anceGuidebook Working GroupDateDonald W. SovaManager, SMAPDateCarl SchneiderDirector, RMhowever, due to their different purpose and focus, they are not addressed in thisguidebook. For example, the Functional Configuration Audit (FCA) and PhysicalConfiguration Audit (PCA) are configuration management

9、 (CM) activities. Quality(Engineering) Audits and Safety Audits are technical activities that evaluate asoftware product against Quality Engineering and Safety requirements. These typesof audits are not covered in this guidebook.1/(2 blank)Provided by IHSNot for ResaleNo reproduction or networking p

10、ermitted without license from IHS-,-,-Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-II. CONCEPTS AND DEFINITIONSAn SQA audit is an activity that is performed to determine the adherence to, andadequacy of, a projects established software development

11、 standards and proceduresand the effectiveness of their implementation. As used in this guidebook, the mainobjective of an SQA audit is to determine the adherence to established standards andprocedures; checking their adequacy or effectiveness is a secondary objective thatusually is not requested of

12、 an auditor.In the NASA Software Assurance Guidebook, standards are defined as “theestablished criteria to which software products are compared.“ Software standardsinclude documentation standards, design standards, and coding standards. In thatguidebook, procedures are defined as the “established cr

13、iteria to which thedevelopment and control processes are compared.“ Procedures, then, are the step-by-step directions that are to be followed to accomplish some development or controlprocess; for example, CM or nonconformance reporting and corrective action(NRCA). In other words, standards and proce

14、dures are requirements for softwaremanagement, engineering, and assurance; SQA audits verify their existence and assessa projects compliance with them.SQA audits also can compare the actual status of a product with reported status.Status auditing is most effective if there are objective and consiste

15、nt criteria forevaluating the level of product completeness. For example, Unit DevelopmentFolders (UDFs) have a cover sheet for recording the progress of a unit through itsdevelopment stages; the folder contains the actual product. If a project uses UDFs,then an audit can compare the actual product

16、to the cover sheet and to the progressreport.The actual processes and products examined by an audit will vary depending on theobjective of the audit. The objective of the audit can vary, and is determined by theorganization that called for the audit. A general audit provides a comprehensiveoverview,

17、 while a limited audit might be an examination of certain procedures, suchas CM, or a check on a certain requirement, such as “Are coding standards beingfollowed?“An audit may be described as internal or external, depending on the organization oforigin of the auditor(s). An internal audit is an audi

18、t conducted by the SQA staff ofthe software developer. Internal audits are intended to be preventative in nature; todetect problems before they become major.An external audit is one performed by an independent auditor who is outside of thedeveloping organization. External audits are most often reque

19、sted by the acquiringorganization, as a means of obtaining an independent opinion about the work inprogress. External audits tend to be more comprehensive in nature than internalProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-audits, and usually enco

20、mpassa broad area of the development activity. Suchauditsusually are requestedbecausethe acquirer is uncertain of the effectivenessof theinternal program or becauseof lack of information and fears about the quality ofperformance on the part of the developer. An advantageof an external audit is thatt

21、he auditor may be more objective about a project than an internal auditor; however,an external auditor must spendmore time learning about the project and itsdevelopment process.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-III. CONDUCTING AN SQA AU

22、DITAn SQA audit has four phases: planning and preparation, the site visit, reporting, andfollow-up. During the planning and preparation phase, the auditor gains anunderstanding of the project. Based on the scope of the audit, the auditordetermines the specific questions that need to be answered, as

23、well as the persons tobe interviewed and the records and products to be examined to answer the questions.The interviews are conducted, and records and products are examined during the sitevisit. The reporting phase consists of the exit debriefing of the audited project, thepreparation of a written r

24、eport on the audit, and clarifying issues and providingrelated information as needed. Follow-up is done by the project, as the problems anddeficiencies found in the audit are remedied. Follow-up may include reauditing toassess the adequacy of the remedies.The activities conducted during the phases v

25、ary depending on the life cycle phase ofthe project being audited and the scope of the audit. The activities also varydepending on whether the audit is external or internal; an external audit requiresmore extensive preparation and should examine a more comprehensive sample ofmaterial than an interna

26、l audit.Each of the four phases of an audit is described in the following sections. Theactivities of each phase are described as if a general, external audit is to be donesince this results in the greatest detail. Some of the activities may be superfluous toan internal SQA audit and may be omitted.A

27、. Audit Planning and PreparationA general SQA audit should be planned carefully to examine all of the softwareengineering, management, and assurance processes and all of their products.Software management processes include status reporting and CM. Engineeringprocesses include analysis, design, and c

28、ode. Assurance processes include verificationand validation (V a limited audit requires a checklist that is more detailed in specific areas.Guidance on preparing a checklist is given in Chapter VI. A checklist is intended toprovide the auditor with a “road map“ during the site visit. It must be comp

29、lete, sothat the auditor can know that sufficient information has been gathered if all of thechecklist items are completed. The checklist questions help define the individualswith whom the auditor wishes an interview and the types of records that the auditorwill examine.The auditor should schedule t

30、he site visit to the project through its assurance staff orother suitable contact after the preparation is done and the checklist prepared.During this contact with the project, the auditor should specify the intent of the audit,the records to be examined, and which people the auditor wishes to inter

31、view.People to be interviewed will include managers, selected developers, CM staff,assurance staff, and testers. Copies of the checklist may be furnished to increase theprojects understanding. The project should be prepared to provide the auditor witha convenient working area that includes normal of

32、fice facilities, access to all productsand records, and interviews with the identified individuals.6Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-B. The Site VisitThe purpose of the audit site visit is to collect the data necessary to assess that t

33、herequired products are being produced, the degree to which they conform toapplicable standards, how well procedures are being followed, and that the reportedstatus corresponds to the actual status. The audit is intended to uncover anysignificant deviation from standards, procedures, or reported sta

34、tus so that correctiveaction can be taken. The auditor uses two basic techniques: interviews with projectstaff and examination of documentation and records.The site visit should begin with an entrance briefing, involving the auditor and keyproject staff. During this briefing, the auditor should desc

35、ribe the focus of the audit,and identify the interviews to be conducted and the records to be examined. Theentrance briefrag may also be used by the project to brief the auditor on itsprocesses, key staff members, and current status. Time for questions and answersshould be included. The auditor also

36、 should assure the project that an exit interviewwill be held where the auditor will present preliminary findings to the project and theproject may provide any additional information to the auditor. This preliminaryexchange of information can significantly help to allay the fears of the project and

37、tosmooth the course of the site visit.After the entrance briefing, the auditor should proceed with the gathering ofinformation. It is useful to begin the information gathering process with interviews,during which the auditor tries to understand the realities behind the documentedplans and procedures

38、. The auditor should learn which individuals carry out aprocedure, approve a change or fix, keep project records, etc. Each individual shouldbe asked to describe his/her perceptions of and interactions with the process. Theauditor should take notes, annotate or develop procedural flow diagrams, askq

39、uestions to clarify, and make it his/her objective to clearly understand the process.In particular, the auditor should be alert for indications of shortcuts or abbreviationsto the procedure. During interviews, the auditor must remember that data are beinggathered, and that conclusions should wait un

40、til all of the facts are in. This providesa clearer understanding of the actual processes used on the project and easescommunications with the staff. The checklist developed during the preparation phaseis used to guide the discussions during the interview.Once the auditor is sure that the processes

41、and procedures are understood as theyreally exist, he/she should begin examining the tangible parts of the project: itsproducts and records. Products consist of requirements and design documentation,including unit development folders, user manuals, code, etc. Records consist ofmemoranda and forms th

42、at document the events in the life of a product. They comefrom CM, NRCA, and V through analysis of impact and dispositioning;design, code, and testing; updating of documentation; submissionof the modified products to the library; and closure of thechange request. Records to be examined include the c

43、hangerequests as processed by the Change Control Board, the workauthorizing documents issued as a result of approved changes,the code and documentation products that are intended to reflectthe approved changes, and the program library records thatcapture the changes to code and data. Throughout the

44、audit, theauditor should be alert for and document any evidence ofunauthorized changes.The records should show the authorization of each change, theproduct(s) to be changed, and the version numbers of thechanged product. Much of the auditors attention should bedevoted to the Program Library or equiv

45、alent, since this is wherethe various versions of products and the change documentscontrolling those versions are stored. The auditor should checkthe products in the library to ensure that documentation is up-to-date with code changes. The auditor should check the versionnumbering and identification

46、 schemes, and the controldocuments. The records should demonstrate that there areadequate security measures in place to prevent loss andunauthorized changes. The auditor should verify that every itemof code and documentation in the program library was properlyreceived.NRCA AuditWhen auditing the NRC

47、A system, the auditor should look at thecomplete cycle. The auditor should review the nonconformancereports that are filed, to assure that they are completely andcorrectly filled out. The disposition process and board actionsProvided by IHSNot for ResaleNo reproduction or networking permitted withou

48、t license from IHS-,-,-shouldbe recorded, usually on the sameform. Thenonconformancesthat result in product changesshould betracked to the product, and evidenceshould be gathered thatchangesare made, tested or reviewed, and approvals for issuanceare granted. The NRCA procedureswill parallel thoseuse

49、d inCM, and canbe audited in much the sameway, especiallywhenit comesto the program library. In both cases(CM and NRCA),the auditor should pay particular attention to corrected productsto assurethat they still satisfy requirements and standards. V that is, the auditor must be able to determine that a do