REG NASA-LLIS-2218-2009 Lessons Learned Flight Software Engineering Lessons.pdf

1、Lessons Learned Entry: 2218Lesson Info:a71 Lesson Number: 2218a71 Lesson Date: 2009-06-23a71 Submitting Organization: JPLa71 Submitted by: David Oberhettingera71 POC Name: Ronald Kirk Kandta71 POC Email: Ronald.K.Kandtjpl.nasa.gova71 POC Phone: 818-393-0907Subject: Flight Software Engineering Lesson

2、s Abstract: The engineering of flight software is a major consideration in establishing JPL project total cost and schedule because every mission requires a significant amount of new software to implement new spacecraft functionality. Constraints to the development and testing of software concurrent

3、 to engineering the rest of the flight system has led to flight software errors, including the loss of some missions. The findings of several JPL studies and flight mishap investigations suggest a number of recommendations for mitigating software engineering risk.Description of Driving Event: The en

4、gineering of flight software (FSW) for a typical NASA/Caltech Jet Propulsion Laboratory (JPL) spacecraft is a major consideration in establishing the total project cost and schedule because every mission requires a significant amount of new software to implement new spacecraft functionality. FSW dev

5、elopment is performed concurrently along with the design and development of (1) the spacecraft with its many mechanical, electronic, and computational elements, (2) the instruments that comprise the spacecrafts payload, and (3) the testbed software that simulates the spacecraft, its payload, and the

6、 extreme operational environment of space. This is extremely challenging because the complete and accurate spacecraft hardware design documentation needed to complete the design and test the FSW and develop the simulation software is typically unavailable until late in the software development lifec

7、ycle. Because FSW engineers cannot test their software against mature hardware (that reflects the actual behaviors and performance characteristics of the hardware that will operate in space) until they gain access to flight-like testbeds, a significant amount Provided by IHSNot for ResaleNo reproduc

8、tion or networking permitted without license from IHS-,-,-of software testing is usually performed after the spacecraft has launched, during the relatively benign cruise portion of the mission. Software changes are then uploaded to the spacecraft prior to critical mission events- like landing on Mar

9、s. These endemic constraints to FSW design and test impact project cost and schedule and pose a risk to mission success because the FSW must be integrated with the flight system hardware. In addition, FSW is suffering accelerating growth in size, complexity, and difficulty to understand and verify (

10、Reference (1). These factors complicate project cost and schedule estimation/performance in a functional area where an in-flight fault may cause an unrecoverable error. FSW errors had a role in the loss of the Mars Polar Lander and Mars Climate Orbiter missions (Reference (2), in the Mars Global Sur

11、veyor loss of contact (Reference (3), and in recoverable in-flight failures such as the Mars Exploration Rover flash memory anomaly (Reference (4). References: 1. “NASA Study of Flight Software Complexity,” NASA Lesson Learned No. 2050, NASA Engineering Network, May 5, 2009. http:/www.nasa.gov/offic

12、es/oce/llis/imported_content/lesson_2050.html2. “Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions, JPL Special Review Board, JPL D-18709, March 22, 2000.3. “Mars Global Surveyor (MGS) Spacecraft Loss of Contact,” NASA Lesson Learned No. 1805, NASA Engineering Network, September

13、4, 2007. http:/www.nasa.gov/offices/oce/llis/imported_content/lesson_1805.html4. “MER Spirit Flash Memory Anomaly,” NASA Lesson Learned No. 1483, NASA Engineering Network, August 23, 2004. http:/www.nasa.gov/offices/oce/llis/1483.html 5. Ronald Kirk Kandt, “Flight Software Engineering Lessons,” Proc

14、eedings of the 15th Americas Conference on Information Systems (AMCIS 2009), August 06-09 2009, San Francisco. URS208276Lesson(s) Learned: Reference (5) distills the findings of several JPL studies and flight mishap investigations, conducted over the course of two decades, which explored the causes

15、of FSW problems affecting the success of JPL missions.Recommendation(s): Reference (5) suggests the following steps to mitigate the risk from defects in the FSW development process. Some of these have been addressed in JPL software development process procedures: 1. Adopt a risk-based approach to so

16、ftware engineering. Software development resources may not be allocated optimally because of overly optimistic estimates of software reuse, inability to accurately estimate the cost of new code, reallocation of hardware functionality to FSW, Provided by IHSNot for ResaleNo reproduction or networking

17、 permitted without license from IHS-,-,-hardware that operates differently than planned, contracts bid using overly optimistic cost assumptions, a shortened FSW development schedule due to late hardware delivery, and delays in FSW testing due to late delivery of flight-like testbeds. Where resource

18、limitations constrain the projects ability to perform every preferred software development practice in a manner that yields benefits, software practices should be assessed for risk mitigation potential and adopted based on value. Recommendations 2 through 9 may also help to mitigate software enginee

19、ring risk. 2. Involve software engineers in the early, system-level, project decisions that determine flight system characteristics (e.g., performance, reliability, flexibility), flight system cost, and the role of FSW in the performance of spacecraft flight system functions. 3. Provide projects wit

20、h the requisite software development infrastructure prior to project commencement. To avoid disrupting FSW development, make any changes to the software development process- or to tools that are inadequate or poorly integrated- before software engineers are assigned to the project. 4. Develop simula

21、tions of instruments and other custom hardware that interfaces with FSW at the earliest point at which their behavior is understood. Use these simulations in regression tests of FSW logic, and later for testing in the flight-like environment. Avoid deferring all FSW testing to the end of FSW develop

22、ment when there are fewer resources available to fix problems. 5. Before coding begins, prepare an FSW architecture specification for use in developing the conceptual architecture, realizing the architecture as a high-level integrated flight system design, and managing subsequent changes. As code is

23、 written and actual data is obtained on resource usage and performance characteristics, reassess the architecture and the detailed technical decisions based on the variance from earlier estimates. 6. Define a suite of flight system architectures that are each capable of supporting a commonly-flown c

24、ategory of JPL mission (e.g., “Planetary Rover,” “4 Instruments or Less,” and “High-Bandwidth Communications”). This will significantly ease the problem of project requirements and hardware that are unstable and project software simulations and flight-like testbeds that are unavailable. 7. Developme

25、nt of a complete and consistent set of engineering requirements requires a robust systems engineering process that defines performance and resource utilization requirements, traces requirements to higher and lower-level requirements, ensures review of requirements by key stakeholders and by parties

26、independent of the engineering of the requirements, and assesses the requirements using a checklist of questions that address quality concerns. 8. Use objective measures to monitor FSW development progress and to determine the adequacy of software verification activities. To reliably assess FSW prod

27、uction and quality, these measures should include metrics such as the percentage of code, requirements, and defined Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-faults tested, and the percentage of tests passed in both simulation and testbed envir

28、onments. These measures should also identify the number of units where both the allocated requirements and the detailed design have been baselined, where coding has been completed and successfully passed all unit tests in both the simulated and testbed environments, and where they have successfully

29、passed all stress tests. 9. Manage FSW development using an integrated system. Use of separate “stovepiped” commercial or JPL-developed systems for FSW requirements definition, action item tracking, problem reporting, and scheduling inhibits effective management of FSW development.Evidence of Recurr

30、ence Control Effectiveness: JPL has referenced this lesson learned as additional rationale and guidance supporting Paragraph 6.11 (“Engineering Practices: Software Development”) in the Jet Propulsion Laboratory standard “Flight Project Practices, Rev. 7,” JPL DocID 58032, September 30, 2008. In addi

31、tion, JPL has referenced it supporting Paragraph 4.11 (“Flight System Design: Flight Software System Design”) in the JPL standard “Design, Verification/Validation and Operations Principles for Flight Systems (Design Principles),” JPL Document D-17868, Rev. 3, December 11, 2006.Documents Related to L

32、esson: N/AMission Directorate(s): a71 Space Operationsa71 Exploration Systemsa71 ScienceAdditional Key Phrase(s): a71 1.Engineering design and project processes and standardsa71 0.a71 0.a71 1.Level II/III requirements definitiona71 1.Planning of requirements verification processesa71 1.Software Engi

33、neeringa71 1.Spacecraft and Spacecraft Instrumentsa71 1.Flight Operationsa71 1.Flight Equipmenta71 1.Early requirements and standards definitiona71 1.Training and simulation systemsProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-a71 1.Mission control

34、 Planninga71 1a71 0a71 1.Hardwarea71 1.Payloadsa71 1.Risk Management/Assessmenta71 1.Softwarea71 1.Spacecrafta71 1.Test & VerificationAdditional Info: a71 Project: variousApproval Info: a71 Approval Date: 2009-12-21a71 Approval Name: mbella71 Approval Organization: HQProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-