REG NASA-LLIS-3716-2010 Lessons Learned - Testbed Limitations May Impact End-to-End Flight System Testing.pdf

1、Public Lessons Learned Entry: 3716Lesson Info:a71 Lesson Number: 3716 a71 Lesson Date: 2010-09-7 a71 Submitting Organization: JPL a71 Submitted by: David Oberhettinger Subject: Testbed Limitations May Impact End-to-End Flight System Testing Abstract: After 11 year of spaceflight, it was discovered t

2、hat the dual string Stardust/NExT spacecraft was incapable of switching to the redundant flight system. Flight software changes made only 3 weeks before launch had inhibited side swapping, and the testbed that had verified the changes was not capable of simulating redundancy switching. When it is in

3、feasible to test such changes using the flight system integrated with the launch system, assure that the system testbed is fully equipped for end-to-end simulation of the flight system. Description of Driving Event: After 11 years of spaceflight and the conclusion of a successful primary mission, th

4、e NASA/Caltech Jet Propulsion Laboratory (JPL) discovered that the dual string Stardust/NExT (New Exploration of Comet Tempel 1) spacecraft flight system had been flying single string. An established JPL practice (Reference (1) is to strictly avoid in-flight switches to backup units (e.g., for alive

5、ness checks) as long as the prime unit continues to serve mission needs. Nine months prior to the NExT spacecraft encounter with Tempel 1, however, a switch from the Side A flight system to the identical Side B flight system was commanded due to concerns that the Side A gyro laser had degraded over

6、the 11 years. Since Side B had not been used during the spacecrafts Stardust mission or NExT extended mission, its attitude control gyros laser was known to retain enough intensity for continuous use through the comet encounter. When the switch to the Side B flight system computer was commanded by g

7、round controllers, an undetected timing issue between fault protection software and Safe Mode sequence commands prevented Side B from completing the cold boot from an unpowered state. Autonomous fault protection caused the flight system to dead-end on Side A, and Side A functionality was restored wh

8、en the computer completed a cold start and nominal reboot. The root cause of the redundancy switching failure was a very late change (i.e., 3 weeks before launch) to the Safe Mode command file (Reference (2). Two attitude mode commands were inserted to address a Rate Mode configuration issue identif

9、ied during system-level test (ATLO). When these commands were executed during the side swap, the Attitude except for an earlier solar flare incident (Reference (3) that masked the side swap defect, the spacecraft had never been cold booted after the pre-launch change to the Safe Mode file. The abili

10、ty of the spacecraft to switch to the secondary string (Side B) was enabled in June 2010 through uplinked changes to the onboard fault protection software. However, if the Side A flight system had failed at any time during the 11 years following launch, the unavailability of Side B would have doomed

11、 the mission. References: 1. “Design, Verification/Validation and Operations Principles for Flight Systems (Design Principles),“ JPL Document D-17868, Rev. 3, December 11, 2006, Paragraph 9.4 (“Prime/Redundant Hardware Usage“). 2. “S/C failed to boot on Side B,“ JPL Incident Surprise Anomaly (ISA) N

12、o. 25926, May 6, 2010. 3. no title, JPL Incident Surprise Anomaly (ISA) No. Z70478, November 10, 2000.Lesson(s) Learned: When a system testbed that has been in use for development of a particular spacecraft flight system is used to verify software changes to the system, additional equipment may be n

13、eeded in the testbed to properly simulate or test essential functions like redundancy switching. The lack of such necessary equipment may constitute a violation of the “test-as-you-fly“ rule. Recommendation(s): When it is only feasible to test “last minute“ command changes or flight software changes

14、 via simulation, instead of using the flight system that has been integrated with the launch vehicle, assure that the simulation testbed is capable of end-to-end verification of the impact on all flight software functions, including fault protection. Should the system testbed lack high fidelity feat

15、ures such as dual string simulation, the project should identify potential testing shortfalls and address how it will validate the test results. Evidence of Recurrence Control Effectiveness: JPL has referenced this lesson learned as additional rationale and guidance supporting Paragraph 5.7.7 (“Mana

16、gement Practices: Spares, Testbeds, and Models“) in the Jet Propulsion Laboratory standard “Flight Project Practices, Rev. 7,“ JPL DocID 58032, September 30, 2008. Documents Related to Lesson: N/A Mission Directorate(s): Provided by IHSNot for ResaleNo reproduction or networking permitted without li

17、cense from IHS-,-,-a71 Science a71 Exploration Systems a71 Space Operations Additional Key Phrase(s): a71 Engineering Design (Phase C/D).Robotics a71 Engineering Design (Phase C/D).Simulators and Training Systems a71 Engineering Design (Phase C/D).Software Engineering a71 Engineering Design (Phase C

18、/D).Spacecraft and Spacecraft Instruments a71 Integration and Testing a71 Additional Categories. a71 Additional Categories.Flight Equipment a71 Additional Categories.Software a71 Additional Categories.Payloads a71 Additional Categories.Flight Operations a71 Systems Engineering and Analysis.Planning

19、of requirements verification processes a71 Additional Categories.Test & Verification a71 Additional Categories.Test Facility a71 Additional Categories.Spacecraft Additional Info: a71 Project: Stardust/NExT Approval Info: a71 Approval Date: 2010-11-02 a71 Approval Name: mbell a71 Approval Organization: HQ Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-