REG NASA-LLIS-1201-2002 Lessons Learned Flight Hardware Damage Due to Inadequate Test Planning and Insufficient QA Involvement (2001).pdf

1、Lessons Learned Entry: 1201Lesson Info:a71 Lesson Number: 1201a71 Lesson Date: 2002-01-01a71 Submitting Organization: JPLa71 Submitted by: Jim NewellSubject: Flight Hardware Damage Due to Inadequate Test Planning and Insufficient QA Involvement (2001) Abstract: Encoder channels on 44 MER motors were

2、 damaged during test due to an improperly inspected cable, lack of safe-to-mate provisions, inadequate test coverage by QA, and failure to verify encoder function following the first of the 44 tests.Recommendations included the need for approved test plans, completion of operational safety action it

3、ems, first item witnessing by QA, and comprehensive functional checks after testing of the first item. Description of Driving Event: During dynamometer testing of Mars Exploration Rover (MER) motors, position indication circuitry (within the encoder portion of the motor) was damaged. A test cable co

4、nnector used to supply power to the motor brush leads was damaged prior to the first motor test. Its use resulted in inadvertent and improper powering of the encoders. Ultimately, damage occurred to at least one encoder channel on each of the first 44 motor assemblies that were tested. These commerc

5、ial motors are used on MER for such purposes as Airbag Retraction Actuators, and they are long lead-time acquisition items. Several errors contributed to the test failure: a71 A special GSE power supply/monitor had been subjected to failure modes and effects analysis (FMEA) and approved for use on t

6、he motors. However, it was intentionally not used during dynamometer testing in order to avoid powering the encoder. Instead, an auxiliary power supply was directly connected with a two-wire cable.a71 The two-wire power cable had one connector with an improperly crimped shell. The cable Provided by

7、IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-was not properly inspected and tested prior to use on this critical hardware.a71 There was no test plan or test procedure written for the dynamometer test operations. Results of the Operational Safety Survey (OSS)

8、required an approved test procedure and Assembly Instruction Data Sheets (AIDS) prior to test commencement. Although step-by-step AIDS were prepared and approved by QA, they contained no safe-to-mate provisions for attachment of the power connector.a71 Although QA witnessed some of the dynamometer t

9、ests, they did not witness the test of the first motor.a71 Encoder function was not verified following dynamometer testing since there was no intention to power the encoders during these tests. Hence, the damage was not detected after test of the first motor, and 43 more motors were damaged in the s

10、ame manner.References: 1. Jim Newell, “Maxon Motor Encoder Damage - Mistakes, Recovery Plan and Lessons Learned,” Internal JPL Memorandum 510-01-076, December 6, 2001.2. 3. JPL Problem/Failure Report No. Z72981, December 3, 2001.4. ”High Energy Spectroscopic Imager Test Mishap”, LLIS #0903.Additiona

11、l Key Words: connector mating, electrical test, functional test, ground support equipment, quality assurance oversight, safe-to-test, test error, test incident, test induced damage, test oversight, test safety Lesson(s) Learned: Flight hardware test shortcuts and workarounds that bypass Systems Safe

12、ty, QA, Reliability, and other oversight monitoring functions may seem expeditious, but can backfire with disastrous results.Recommendation(s): 1. Prepare, review, and approve (including QA) written test plans (when appropriate) and test procedures for all tests involving critical hardware prior to

13、test.2. a71 Strictly observe connector safe-to-mate practices, including continuity and shorting checks, before cables (flight or GSE) are initially connected to critical hardware.2. Prior to testing, review the OSS, and without exception complete all OSS action items.3. QA should witness the setup

14、for testing and witness first item testing of all critical hardware.4. Perform comprehensive functional checks after completion of testing the first item, and after all subsequent testing of identical critical hardware (especially as a final gate prior to delivery of this hardware).Provided by IHSNo

15、t for ResaleNo reproduction or networking permitted without license from IHS-,-,-Evidence of Recurrence Control Effectiveness: Corrective Action Notice No. Z75799 was closed by JPL on September 16, 2003. The measure taken to implement Laboratory-wide corrective action on the above recommendations wa

16、s revision of the JPL procedure, “Develop Hardware Products” (JPL DocID 57752), effective 8/9/2002, and the JPL guideline, “Develop Hardware Products Recommended Practices” (DocID 58033), effective 8/8/2002.Documents Related to Lesson: N/AMission Directorate(s): a71 Aeronautics & Space Transportatio

17、n Technologya71 Earth Sciencea71 Human Exploration & Development of Spacea71 Space ScienceAdditional Key Phrase(s): a71 Flight Equipmenta71 Ground Equipmenta71 Ground Operationsa71 Hardwarea71 Payloadsa71 Safety & Mission Assurancea71 Spacecrafta71 Test & Verificationa71 Test FacilityAdditional Info: Approval Info: a71 Approval Date: 2002-05-20a71 Approval Name: Carol Dumaina71 Approval Organization: JPLa71 Approval Phone Number: 818-354-8242Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-