1、10/9/2018,An Assessment of Space Shuttle Flight Software Development Processes,Presented by Jun Wufor Reading in Computer ScienceCUNY Graduate Center,Content of this presentation,Information about the reportIntroductionFindings and Recommendations,About This Report,The project that is the subject of
2、 this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The report has been reviewed by a group other than the authors accordi
3、ng to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.,About the report ( cont.),This study was supported by Contract NASW-4003 between the National Academy of Sciences and
4、the National Aeronautics and Space Administration. Chair of the Committee for Review was Nancy G. Leverson Library of Congress Catalog Card Number 93-84549 International Standard Book Number 0-309-04880-X,About Nancy G. Leverson,She was Boeing Professor of Computer Science and Engineering at the Uni
5、versity of Washington. In 2001, She moved to MIT, she is now Professor of of Aeronautics and Astronautics in MIT. Professor Leveson started a new area of research, software safety, which is concerned with the problems of building software for real-time systems where failures can result in loss of li
6、fe or property.,Introduction,In early 1991, the National Aeronautics and Space Administrations (NASAs) Office of Space Flight commissioned the Aeronautics and Space Engineering Board (ASEB) of the National Research Council (NRC) to investigate the adequacy of the current process by which NASA develo
7、ps and verifies changes and updates to the Space Shuttle flight software. The Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes (hereafter, the Committee) was convened in January 1992 to accomplish the following tasks,Tasks,Review the entire flight software dev
8、elopment process from the initial requirements definition phase to final implementation. Review and critique NASAs independent verification and validation process and mechanisms. Determine the acceptability and adequacy of the complete flight software development process, Consider whether independen
9、t verification and validation should continue.,Findings and Recommendations,NASA guidelines and standards Off-nominal cases System-level software V&V The independence of IV&V software safety standards Software safety procedures Personnel,Findings and Recommendations,System-safety organizational role
10、s and responsibilities Organizational roles and responsibilities The role of headquarters S&MQ and the center SR&QA offices Community responsibility Policies, guidelines, and enforcement Final thoughts and future considerations,NASA Guidelines and Standards,Finding #1: Each software development cont
11、ractor provides its own development and coding guidelines for the Shuttle software. These guidelines are not consistent among the developers.,NASA Guidelines and Standards,Recommendation #1: NASA should develop guidelines for software development and V&V procedures and should require contractors to
12、share experiences gained while developing NASA-contracted software. V&V: Verification and Validation,Off-Nominal Cases,Finding #2: V&V inspections by the development contractors pay little attention to off-nominal cases. (i.e., crew/ground errors, hardware failures, or software errors),Off-Nominal C
13、ases,Recommendation #2: The V&V performed by the development contractors should include off-nominal scenarios beyond loop termination and abort control sequence actions and should include a detailed coverage analysis.,System-Level Software V&V,Finding #3: V&V inspections by software development cont
14、ractors focus on verifying the consistency of two descriptions at different levels of detail (e.g., consistency between a modules requirements and the design of its implementation). The correctness of the requirements with respect to the hardware and software platforms on which implementations run a
15、re generally not considered.,System-Level Software V&V,Recommendation #3:NASA should augment the current V&V process to expand the consideration of system-level issues and should provide adequate funding to allow for successful completion of these tasks.,The Independence of IV&V,Finding #4:Independe
16、nce of the IV&V contractor is limited. For example, the functions the IV&V contractor is allowed to investigate are controlled by the Shuttle Avionics Software Control Board, thereby reducing the IV&V contractors ability to fully investigate potential problems. IV&V: Independent Verification and Val
17、idation,The Independence of IV&V,Recommendation #4: In order to provide a greater level of independence, responsibility for IV&V should be vested in entities separate from the Shuttle program structure and the centers involved in the Shuttle software development and operation. However, these organiz
18、ations should continue to conduct activities supporting IV&V.,Software Safety Standards,Finding #5:Current NASA safety standards and guidelines do not include software to any significant degree. A software safety guideline has been in draft form for four years. Decisions are being made and safety-cr
19、itical software is being built without minimal levels of software safety analysis or management control being applied.,Software Safety Standards,Recommendation #5: NASA should establish and adopt standards for software safety and apply them as much as possible to Shuttle software upgrades. The stand
20、ards should be applied in full to new projects such as the space station. NASA should not be building any software without such standards in place.,Software Safety Standards,Recommendation #6:NASA should provide headquarters S&MQ with the authority to approve or reject any tailoring of the software
21、safety standards for individual programs and minimize the differences between the safety programs being followed at different centers within a single program. S&MQ: Safety and Mission Quality,10/9/2018,Software Safety Procedures,Finding #6: The Committee found insufficient coordination between the S
22、huttle system-safety program and the software activity. There is no tracing of system hazards to software requirements and no criticality assessment of software requirements or components (except when they are changed). There is no baseline software hazard analysis that can be used to evaluate the c
23、riticality of software modifications and no documentation of the software safety design rationale.,Software Safety Procedures,Recommendation #7: For the Shuttle software safety process, NASA should provide a software safety program plan that is reviewed and approved by headquarters S&MQ, the SR&QA m
24、anagers at the centers, and the Shuttle program manager. Recommendation #8: NASA should perform a hazard analysis for the Shuttle software. NASA should also implement the other appropriate aspects of the draft software safety guideline and provide a software safety design-rationale document. SR&QA:
25、Safety, Reliability, and Quality Assurance,Personnel,Finding #7: The SR&QA offices at the centers have limited personnel to support software-related activities. The assignment of one civil servant to software safety is not adequate to do more than just attend meetings. Finding #8: There is little ov
26、ersight or evaluation of software development activities by the center SR&QA offices.,Personnel,Recommendation #9: NASA should build up expertise on software and software safety within the center SR&QA groups and headquarters and provide adequate personnel to perform flight software S&MQ activities,
27、System-Safety Organizational Roles and Responsibilities,Finding #9: The reporting relationship between the centers and headquarters S&MQ is ill-defined. There is little interaction between the Johnson Space Center (JSC) SR&QA office and the software development activities within IBM and Rockwell. He
28、adquarters has no enforcement power. Multiple centers on the same program may be enforcing different standards and procedures.,System-Safety Organizational Roles and Responsibilities,Recommendation #10: NASA should establish better reporting and management relationships between developers, centers,
29、programs, and the headquarters Safety Office. Recommendation #11: NASA should consider the establishment of a NASA safety certification panel or board separate from the program offices and also the establishment of a subcommittee of the Aerospace Safety Advisory Panel to deal with software issues.,O
30、rganizational Roles And Responsibilities,Finding #10: The Shuttle flight software maintenance and upgrade process is not adequately documented. There are important aspects of the process that are not described in the available documentation. This lack of visibility represents an increased risk of so
31、ftware-related problems.,Organizational Roles And Responsibilities,Recommendation #12: NASA should continue to enhance the current effort to fully document all aspects of the Shuttle flight software process. The effort should clarify the responsibilities of each contractor and each part of the NASA
32、organization in a concise and readable format.,The Role of Headquarters S&MQ and the Center SR&QA Offices,Finding #11: The headquarters S&MQ Office would have no authority to enforce established guidelines and policies if such existed. Finding #12: The SR&QA offices at the centers do not have the re
33、sources, manpower, or authority to compel the development contractors or other NASA organizations to provide information that is sufficient to assure that the proper process is being followed.,The Role of Headquarters S&MQ and the Center SR&QA Offices,Finding #13: There is a lack of visibility for p
34、otential software problems because there are few requirements or opportunities to report software reliability, quality assurance, or safety problems to the program-level safety organizations or to headquarters. Recommendation #15: The headquarters S&MQ Office and the SR&QA offices at the centers sho
35、uld be given routine access to all software-related problem reports, and all members of the flight software community should be made aware of their responsibility to keep these oversight organizations involved in their activities.,Community Responsibility,Finding #14: Many important functions within
36、 the flight software process appear to be assigned to the flight software community rather than a specific NASA or contractor organization. Recommendation #16: NASA should assign specific responsibilities for each aspect of the flight software process and document them accordingly. Responsibility sh
37、ould be assigned to individuals or offices and not to the community as a whole.,Policies, Guidelines, and Enforcement,Finding #15: There is a lack of accepted policies and guidelines for appropriate implementation of V&V, IV&V, reliability, quality assurance, and safety measures. Recommendation #17:
38、 NASA should establish a process that provides the center and program managers with the opportunity to comment on proposed policies and guidelines, but also gives the appropriate headquarters personnel the authority to approve the policies and guidelines in cases where complete consensus cannot be r
39、eached in a reasonable amount of time.,Policies, Guidelines, and Enforcement,Finding #16: A primary reason for the lack of established policies and guidelines is the absence of sufficient resources, manpower, and expertise devoted to developing them. Recommendation #18: NASA should provide the S&MQ
40、Office at headquarters and the SR&QA offices at the centers with the additional resources needed to build their expertise in software IV&V, safety, reliability, and quality assurance.,Final Thoughts And Future Considerations,Recommendation #19: NASA should undertake an effort to capture the lessons
41、learned in the development, maintenance, and assurance of the Shuttle flight software for use by other programs. The same type of documentation should be routinely prepared for other programs as well.,Final Thoughts And Future Considerations,Recommendation #20: In future procurements, NASA should mo
42、re precisely identify the information that each development and oversight contractor is responsible for making available to each other and to the community as a whole. Recommendation #21: Based on the lessons learned in the Shuttle program, NASA should put in place the mechanisms necessary to ensure
43、 that all existing and future programs are given the information needed to make intelligent implementations of software oversight functions such as IV&V.,Final Thoughts And Future Considerations,Recommendation #22: NASA should upgrade its workforce and management practices to make it a leader in sof
44、tware engineering and software quality. NASA should maintain as much in-house capability as possible to reduce its dependence on contractors and to provide proper assurance that contracted work is done on time and with as much attention to safety and other qualities as future systems require and deserve.,
