REG NASA-LLIS-0991-2001 Lessons Learned Planning and Conduct of Hazardous Tests Require Extra Precautions (2000-2001).pdf

1、Lessons Learned Entry: 0991Lesson Info:a71 Lesson Number: 0991a71 Lesson Date: 2001-01-01a71 Submitting Organization: JPLa71 Submitted by: George GreaniasSubject: Planning and Conduct of Hazardous Tests Require Extra Precautions (2000-2001) Abstract: Two ruptures of burst discs and an over pressuriz

2、ation of the cryostat helium tank occurred on the ground during integration and test of SIRTF. Precautions for planning and conducting hazardous tests should include high quality test procedures, documentation of and pre-test training for known test anomalies, easily understood readouts, test readin

3、ess reviews including test procedures confirmation for proper GSE & sensor maintenance, and involvement of quality assurance personnel throughout the pre-test and testing activities.Description of Driving Event: Three failures occurred on the ground over a three-week period during integration and te

4、st of the Space Infrared Telescope Facility (SIRTF) cryostat at a contractor facility: Rupture of a Pressure Release Device (“Burst Disc“) on Cryostat Fill-Side. During a helium top-off operation performed to place the cryostat in a safe standby configuration, unexpectedly slow helium transfer was n

5、oted by test personnel. This was later attributed to a clogged filter on ground support equipment (GSE) that had not been properly maintained. To speed the flow, test personnel intentionally closed the cryostat internal (see V3 in the graphic) and external (V4) fill valves in reverse order, contrary

6、 to the formal test procedure. This caused a pressure surge within the plumbing that ruptured the fill-side burst disc. The test crew was not adequately trained and staffed to deal with this test anomaly. Rupture of Burst Disc on Cryostat Vent-Side. A second incident ruptured the burst disc on the v

7、ent-side of the cryostat. A fitting on the GSE cryogenic servicing equipment had been removed and reinstalled without proper torqueing or a leak check. The leaky fitting allowed air to enter the flight system over a period of several days. Frozen condensates from the leak blocked the vent side plumb

8、ing and caused the failure. The root cause was the lack of a formal procedure for (1) Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-maintaining the GSE, (2) verifying that there were no leaks within the GSE plumbing after connection to the cryostat

9、, or (3) monitoring the GSE for leakage on a continuous basis. Cryostat Helium Tank Overpressurization. The blockage of the vent line described above caused over-pressurization of the helium tank above the maximum design pressure. Due to the blocked flow, the tank pressure increased even as the pres

10、sure downstream of the blockage remained stable. The tank temperature sensor readings should have alerted the test crew, but the readings were considered suspect. This lack of response to a safety-critical sensor indication may be attributed to lack of (1) confidence caused by previous sensor calibr

11、ation and readout problems, (2) red-flag limits in the control software, and (3) careful real time scrutiny of the test data by cryogenic experts outside the core Cryo-Telescope Assembly (CTA) team. Equipment damage was limited to the two ruptured burst discs, but the incident resulted in a signific

12、ant cost and schedule impact to the SIRTF project, and posed a potentially significant risk to personnel safety. refer to D descriptionDGSE Configuration When Fill-Side Burst Disc Was Ruptured Reference: SIRTF Cryostat Operations Failure Report, contractor internal report, March 27, 2001. Additional

13、 Key Words: test facility operations, facility safety, test equipment maintenance, GSE Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-maintenance, QA oversight, test oversight, test plan Lesson(s) Learned: When planning tests that are potentially ha

14、zardous to personnel, flight hardware or facilities (e.g., high/low temperatures or pressures, stored energy, deployables), special measures should be taken to ensure that:1. Test procedures are especially well written, well organized, and easy to understand by both engineering and quality assurance

15、 personnel.2. Known test anomalies that history has shown to be inherent to the test equipment or conditions (including their likely causes, effects, and remedies) are documented and included in pre-test training.3. Readouts of safety-critical test control data are provided in an easily understood f

16、orm (e.g., audible, visible or graphic format).4. Test readiness reviews are held, and test procedures require confirmation that GSE test equipment and sensors have been properly maintained.5. Quality assurance personnel are present and involved throughout the test to ensure procedures are properly

17、followed, including prescribed responses to pre-identified potential anomalies.Recommendation(s): (See Lessons Learned) Evidence of Recurrence Control Effectiveness: N/ADocuments Related to Lesson: N/AMission Directorate(s): N/AAdditional Key Phrase(s): a71 Cryogenic Systemsa71 Facilitiesa71 Ground

18、OperationsProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-a71 Ground Equipmenta71 Hardwarea71 Human Factorsa71 Human Resources & Educationa71 Industrial Operationsa71 Pressure Vesselsa71 Safety & Mission Assurancea71 Test Facilitya71 Test & VerificationAdditional Info: Approval Info: a71 Approval Date: 2001-10-15a71 Approval Name: Carol Dumaina71 Approval Organization: JPLa71 Approval Phone Number: 818-354-8242Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-