REG NASA-LLIS-2050-2009 Lessons Learned NASA Study of Flight Software Complexity.pdf

1、Lessons Learned Entry: 2050Lesson Info:a71 Lesson Number: 2050a71 Lesson Date: 2009-05-5a71 Submitting Organization: JPLa71 Submitted by: David Oberhettingera71 POC Name: Dan Dvorak; Adam Westa71 POC Email: Daniel.L.Dvorakjpl.nasa.gov; adam.d.west-1nasa.gova71 POC Phone: 818-393-1986 (Dvorak); 202-3

2、58-1552 (West)Subject: NASA Study of Flight Software Complexity Abstract: Flight software development problems led NASA to study the factors that have led to the accelerating growth in flight software size and complexity. The March 2009 report on the NASA Study on Flight Software Complexity contains

3、 recommendations in the areas of systems engineering, software architecture, testing, and project management.Description of Driving Event: Since 1968, the flight software (FSW) used aboard NASA human and robotic spacecraft has grown in size by a factor of ten every ten years. The primary FSW for the

4、 NASA Crew Exploration Vehicle (Orion) and the Mars Science Laboratory (MSL) are each expected to exceed one million lines of code (LOC). The growth trend is expected to continue because of increasingly ambitious mission requirements and because of the advantages (e.g., zero mass, flexibility) of si

5、tuating new functionality in software or firmware rather than in hardware. An increase in FSW size is often accompanied by an increase in FSW “complexity“- defined as a state in which the FSW is difficult to understand and verify- and an increase in risk. Problems with FSW during a variety of NASA s

6、paceflight missions, like those recently reported in References (1) and (2), led NASA to conduct a study (Reference (3) of the engineering activities- from requirements development through operations- that lead to growth in FSW size and complexity. The final report from the study attributes NASAs la

7、ck of control over these FSW characteristics to cultural factors such as: Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-1. Cost and schedule pressures that lead FSW managers and developers to reduce or eliminate activities where the benefits are no

8、t as easy to quantify as those from the production of code. 2. Lack of enforcement of FSW standards and local best practices. 3. Pressure to re-use software from a previous mission because it is “flight-proven“ and presumed to be of lower risk. 4. Emphasis on the flight project at hand, with few pro

9、grammatic incentives for project management to contribute to infrastructure and process improvements that benefit future missions. The NASA final reports findings in the areas of systems engineering, software architecture, testing, and project management are summarized in the Lessons Learned section

10、 below, and its corresponding recommendations are summarized in the Recommendations section. References: 1. “MRO Spaceflight Computer Side Swap Anomalies,“ NASA Lesson Learned No. 2041, NASA Engineering Network, December 16, 2008. http:/www.nasa.gov/offices/oce/llis/imported_content/lesson_2041.html

11、 2. “MRO Articulation Keep-Out Zone Anomaly,“NASA Lesson Learned No. 2044, NASA Engineering Network, April 7, 2009. http:/www.nasa.gov/offices/oce/llis/imported_content/lesson_2044.html 3. “Final Report: NASA Study on Flight Software Complexity,“ NASA Office of Chief Engineer, March 5, 2009 4. “NASA

12、 Systems Engineering Handbook,“ NASA SP-2007-6105, December 31, 2007. 5. “Improving Fault Management for Spaceflight Missions,“ NASA Lesson Learned No. 2049, NASA Engineering Network, April 21, 2009. http:/www.nasa.gov/offices/oce/llis/imported_content/lesson_2049.html Lesson(s) Learned: 1. Engineer

13、s and scientists often dont realize the downstream complexity (and cost) that result from their local decisions. Overly stringent requirements and simplistic hardware interfaces can complicate software; FSW descope decisions and ill-conceived autonomy can complicate operations; and inattention to te

14、stability can complicate verification efforts. 2. Unsubstantiated requirements have caused unnecessary FSW complexity where the requirement was either unnecessary or overly stringent. Rationale statements have often been Provided by IHSNot for ResaleNo reproduction or networking permitted without li

15、cense from IHS-,-,-omitted or misused in spite of a best practice that calls for a rationale for every requirement. Reference (4) places great emphasis on the need to provide rationales. 3. Although overall complexity can be reduced through engineering trade studies involving multiple stakeholders (

16、e.g., flight, ground, hardware, software, testing, operations), Reference (3) found that trade studies were often omitted or performed superficially. 4. Good software architecture is the most important defense against incidental complexity in software designs, but good architecting skills are not co

17、mmon. 5. A decision is often made not to implement a FSW function, or not to fix a FSW defect, principally because an operational workaround is feasible. This practice transfers complexity from FSW to mission operations where it is a continuing cost rather than a one-time cost and where it increases

18、 the risk of an operational error. 6. Fault management (aka fault protection) software is among the most difficult FSW to specify, design, and test. Other fault management problems (Reference (5) include inconsistent fault management terminology used by NASA centers and their contractors, mismatches

19、 between mission requirements and the fault management approach, and lack of attention to fault management in university curricula. 7. Commercial off-the-shelf (COTS) software can provide valuable functionality, but it often comes with unneeded features that interact with other features and complica

20、te testing. 8. Because software development can generate at least 1 defect per 10,000 LOC, FSW with 1 million LOC will have 100 defects, some of which will likely manifest during mission operations. Mature software may contain latent defects that may manifest in new applications. Recommendation(s):

21、1. Furnish appropriate educational materials, such as a “complexity primer,“ and submit lessons learned on FSW complexity to the NASA Lesson Learned Information System (LLIS). 2. Where well-substantiated requirements are exceedingly difficult to implement in software, software managers should proact

22、ively inform the project of the impact. In some cases this might stimulate new discussion to relax hard-to-achieve requirements. 3. Project managers should promote the value of multi-disciplinary trade studies in reducing downstream complexity, and project engineers should raise complexity concerns

23、as they become apparent. 4. Allocate a larger percentage of project funds to up-front architectural analysis in order to achieve project cost savings. Increase the number and authority of software architects. Create a professional architecture review board to provide constructive early feedback to p

24、rojects. Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-5. Involve mission operators in flight/ground trade studies and in decisions to descope FSW. Quantify the costs and risks of operational workarounds before reassigning a FSW function or fix to

25、mission operations. 6. Develop a NASA Fault Management Handbook to provide guidance in fault management requirements specification, design, and test. Schedule fault management reviews in proposals and project plans. Sponsor or facilitate the inclusion of fault management courses within university en

26、gineering programs. 7. Analyze a candidate COTS product for the ability to separate its components and features, and weigh the cost of testing unwanted features against the cost of in-house development. 8. To cope with the inevitable defects in new or revised FSW, conduct randomized testing to avoid

27、 bias toward specific sources of error, extend fault protection to cover software faults as well as hardware faults, and employ fault containment techniques to mitigate the likelihood that residual defects will combine to produce complex error scenarios. Evidence of Recurrence Control Effectiveness:

28、 JPL has referenced this lesson learned as additional rationale and guidance supporting Paragraph 6.4.3 (“Engineering Practices: System Engineering“) in the Jet Propulsion Laboratory standard “Flight Project Practices, Rev. 7,“ JPL DocID 58032, September 30, 2008. In addition, JPL has referenced it

29、supporting Paragraph 4.11 (“Flight System Design: Flight Software System Design“) in the JPL standard “Design, Verification/Validation and Operations Principles for Flight Systems (Design Principles),“ JPL Document D-17868, Rev. 3, December 11, 2006.Documents Related to Lesson: N/AMission Directorat

30、e(s): a71 Space Operationsa71 Sciencea71 Exploration Systemsa71 Aeronautics ResearchAdditional Key Phrase(s): a71 Systems Engineering and Analysis.Engineering design and project processes and standardsa71 Systems Engineering and Analysis.Level II/III requirements definitiona71 Systems Engineering an

31、d Analysis.Mission and systems trade studiesProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-a71 Systems Engineering and Analysis.Mission definition and planninga71 Systems Engineering and Analysis.Planning of requirements verification processesa71 En

32、gineering Design (Phase C/D).Software Engineeringa71 Engineering Design (Phase C/D).Spacecraft and Spacecraft Instrumentsa71 Integration and Testinga71 Mission Operations and Ground Support Systems.Mission control Planninga71 Mission Operations and Ground Support Systems.Mission operations systemsa7

33、1 Safety and Mission Assurance.Early requirements and standards definitiona71 Safety and Mission Assurance.Review systems and boardsa71 Additional Categories.Flight Operationsa71 Additional Categories.Payloadsa71 Additional Categories.Risk Management/Assessmenta71 Additional Categories.Softwarea71 Additional Categories.Test & VerificationAdditional Info: a71 Project: variousApproval Info: a71 Approval Date: 2009-06-11a71 Approval Name: mbella71 Approval Organization: HQProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-