REG NASA-LLIS-1196-2002 Lessons Learned Test as You Fly Fly as You Test and Demonstrate Margin (1998).pdf

1、Lessons Learned Entry: 1196Lesson Info:a71 Lesson Number: 1196a71 Lesson Date: 2002-01-24a71 Submitting Organization: JPLa71 Submitted by: David OberhettingerSubject: Test as You Fly, Fly as You Test, and Demonstrate Margin (1998) Abstract: Mars Polar Lander had deficiencies in compliance with the p

2、rinciple of “test-as-you-fly,” which requires that ground tests and simulations accurately reflect the planned mission profile, plus margin.Enforce the system-level test principle of “test as you fly, and fly as you test,” and carefully assess any planned violations. When using simulations for syste

3、m-level verification, validate models and include sufficient parametric variations in the simulations to ensure that adequate margins exist. Description of Driving Event: The principle of “test-as-you-fly“ means that ground tests and simulations should accurately reflect the planned mission profile,

4、 plus margin and the appropriate off-design parameters. Although the system level test and verification process for Mars Polar Lander (MPL) was well planned and executed, there were deficiencies in the test program for the parachute, terminal descent, and touchdown phases. MPL test deficiencies incl

5、uded:a71 The touchdown sensing software was not tested with the lander in the flight configuration, leading to the probable cause of the MPL mission loss. (See Lesson #0938)a71 Fault-injection testing of the flight software was not thorough enough to detect all logic errors in post-landing fault-res

6、ponse algorithms. (See Lesson #0939.)a71 The thermal design of the propulsion subsystem was incorrectly characterized in system thermal-vacuum test due to an error in the thermal model, causing major errors in the propulsion thermal design that went undetected until after launch.The test and verific

7、ation process for the Deep Space 2 (DS2) probe, which was co-launched with MPL and did not survive Mars Encounter, also departed from the test-as-you-fly principle in end-to-Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-end testing and analysis:a71

8、 A decision was made to not conduct a system-level impact test of the probe with aeroshell. The risk of structural failure from the dynamic interaction between the aeroshell and the probe was recognized and accepted.a71 Though DS2 was designed to strike the Martian surface at a velocity of 200 meter

9、s per second, there was no impact test of an electrically powered, complete system.a71 The flight battery lot was not subjected to impact tests. Testing performed on an 8-cell predecessor flight-like lot did not provide statistical confidence as it resulted in one structural failure.Adequate system

10、margins were not demonstrated during the MPL terminal descent control system testing. These margins were subject to erosion from propulsion system dynamics (impulse variations due to water hammer or thermal effects), propellant center-of-mass migration, the lack of a high-fidelity fuel slosh model,

11、and nonlinear pulse-width modulation effects. The true margins of the system were not fully characterized in the presence of these effects.References:1. Report on the Mars Polar Lander and Deep Space 2 Missions, JPL Special Review Board (Casani Report), JPL Internal Document D-18709, 22 March 2000,

12、Sections 3.4 and 5.2.2. JPL Corrective Action Notice No. Z69164, Mars Program Investigation Results: “System Engineering/Risk Management/Error Detection,“ 1 May 2000.3. JPL Corrective Action Notice No. Z69165, Mars Program Investigation Results: “Verification and Validation Process,“ 1 May 2000.Addi

13、tional Key Words: Entry, Descent, and Landing (EDL), Environmental Test, Fault Protection, Integration and Test, Risk Assessment, Robust Design, Simulation Accuracy, Software Testing, Spacecraft Test, Technical Margins, Test and Evaluation, Test Errors, Test Fidelity, Test PlanningLesson(s) Learned:

14、 The process of end-to-end system verification (either through testing, simulation, or analysis) may be compromised when it is not consistent with the mission profile (plus margin and the appropriate off-design parameters).Recommendation(s): 1. Enforce the system-level test principle of “test as you

15、 fly, and fly as you test.“ Carefully assess any planned violations of this principle; if they are necessary, take alternate measures such as independent validation. Departures from this principle must be reflected in the project risk management plan, communicated to senior management for concurrenc

16、e, and reported at Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-reviews.2. When using simulations for system-level verification, models must have been validated (e.g., supported by test); and sufficient parametric variations in the simulations mus

17、t be performed to ensure that adequate margins exist.Evidence of Recurrence Control Effectiveness: JPL Corrective Action Notices were assigned (References 2 and 3) and practices will be modified as appropriate.Documents Related to Lesson: N/AMission Directorate(s): a71 Exploration Systemsa71 Science

18、a71 Aeronautics ResearchAdditional Key Phrase(s): a71 Flight Equipmenta71 Hardwarea71 Independent Verification and Validationa71 Launch Processa71 Payloadsa71 Policy & Planninga71 Risk Management/Assessmenta71 Safety & Mission Assurancea71 Spacecrafta71 Test & VerificationAdditional Info: Provided b

19、y IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Approval Info: a71 Approval Date: 2002-05-06a71 Approval Name: Carol Dumaina71 Approval Organization: JPLa71 Approval Phone Number: 818-354-8242Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-