REG NASA-LLIS-1393--2003 Lessons Learned Planning for Failure in High Risk Systems.pdf

1、Lessons Learned Entry: 1393Lesson Info:a71 Lesson Number: 1393a71 Lesson Date: 2003-09-11a71 Submitting Organization: JSCa71 Submitted by: Dave T. Loyd/Angela Luna/Ronald A. MontagueSubject: Planning for Failure in High Risk Systems Abstract: Systemic management and technical failures led to the cat

2、astrophic rupture of a test system. 3 lessons are published regarding this specific incident; this lesson addresses contingency planning and assuring that risk controls remain in place throughout a systems life.Description of Driving Event: On September 11, 2003, at approximately 2:16 PM, the steam

3、line on the Large Altitude Simulation System (LASS) suffered a catastrophic failure of the 24-inch expansion loop located adjacent to the Test Stand 401 steam ejector exhaust duct. The LASS was being operated to verify its readiness to support an altitude firing of the 4th stage of a Minuteman ICBM

4、scheduled for September 25, 2003. The failure occurred approximately 16 seconds after the LASS was commanded to go to full steam, and resulted in a rupture of a portion of the line that dislocated several pieces of the 24-inch pipe in various directions away from the test stand. The proximate cause

5、of the event was pipe wall corrosion that led to selective thinning of the wall, which resulted in the pipe rupture under pressure. The investigation resulted in the following conclusions and observations: 1. Piping containing high mass flow rates must be examined for the capacity to withstand the f

6、orces generated during a pipe wall failure, and potential moment arms must be kept short enough that the pipe strength is not exceeded during a failure.2. Piping designed to “leak-before-burst” capability that has experienced wall thinning will ultimately transition to a “burst-before-leak” situatio

7、n if the wall thinning is not arrested.3. Instrumentation data sample rate and data quality must be high enough to catch the possible events that the transducer might measure, not just the expected events.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,

8、-,-4. System operational capability, such as design life, must be established and documented at the initial design to allow appropriate decisions as the system approaches its capability limits.Lesson(s) Learned: All systems must be regarded as having a limited operational life with adequate design a

9、nd operational provisions made for any credible failure scenario.Recommendation(s): 1. Assure that the system contains voids, supports, guides, and restraints that would allow the steam lines to handle thermal expansion and thrust loading.2. 3. Implement remote safing capability for facility systems

10、.The following recommendations are likely unique to the specific test facility where the event occurred and are provided for additional information: 3. Assess the availability of PPE and atmospheric monitoring equipment in manned test monitoring facilities such as blockhouses.4. Assess the human fac

11、tors design of control room console layouts, and facility system status displays to aid personnel in assessing facility conditions.5. Implement measures to renew the appreciation of the hazards associated with high energy systems, and to the importance of understanding all aspects of high energy fac

12、ility systems condition, among design, test, and maintenance personnel.6. Assure that Failure Modes Effect Analysis (FMEA) and Hazard Analysis (HA) reports are up to date and accurate for the system prior to system pressurization. The HA must be reviewed as part of a Test Readiness review (TRR) or S

13、ystem Requirements Review (SRR.) Updates to the HA must be incorporated whenever changes are made to the hardware or operating procedures. Most importantly, the controls specified in the Hazard Analysis must be verified to be in place before the activity commences.7. Assure thoroughness and follow-t

14、hrough in the Systems Requirements Review process for facilities.8. Assure rigor exists in the Pressure Systems inspection program, including a process for tracking and follow-up on corrections for non-compliances.9. Ensure that the integrity of design principles such as “leak before burst” are pres

15、erved throughout the system life cycle and verified by hard data (as a complement to standards and code compliance).Evidence of Recurrence Control Effectiveness: N/AProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Documents Related to Lesson: NPR 7120

16、.5, NASA Program and Project Management Processes and Requirements; NPR 8831.2D Facilities Maintenance Management; and NPR 8000.4 Risk ManagMission Directorate(s): a71 Exploration Systemsa71 Sciencea71 Space Operationsa71 Aeronautics ResearchAdditional Key Phrase(s): a71 Configuration Managementa71

17、Emergency Preparednessa71 Energetic Materials - Explosive/Propellant/Pyrotechnica71 Energya71 Facilitiesa71 Flight Equipmenta71 Flight Operationsa71 Ground Equipmenta71 Ground Operationsa71 Hardwarea71 Human Factorsa71 Human Resources & Educationa71 Independent Verification and Validationa71 Industr

18、ial Operationsa71 NASA Standardsa71 Parts Materials & Processesa71 Policy & Planninga71 Pressure Vesselsa71 Program and Project Managementa71 Research & Developmenta71 Risk Management/Assessmenta71 Safety & Mission Assurancea71 Softwarea71 Spacecrafta71 Standarda71 Test & Verificationa71 Test Articl

19、ea71 Test FacilityProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Additional Info: Approval Info: a71 Approval Date: 2004-04-26a71 Approval Name: Ronald A. Montaguea71 Approval Organization: JSCa71 Approval Phone Number: 281-483-8576Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-