REG NASA-LLIS-1183--1989 Lessons Learned - Orbiter Processing Facility Bay 2 Water Deluge System Flow Mishap Investigation Board Report of October 25 1989.pdf

1、Lessons Learned Entry: 1183Lesson Info:a71 Lesson Number: 1183a71 Lesson Date: 1989-10-25a71 Submitting Organization: KSCa71 Submitted by: Thomas Utsman/ Eric RaynorSubject: Orbiter Processing Facility Bay 2 Water Deluge System Flow Mishap Investigation Board Report of October 25, 1989 Description o

2、f Driving Event: The Orbiter Processing Facility (OPF) Bay 2 Firex Deluge Zone 3 Viking arm valve AV1.3-2 was leaking, and on September 21, 1989, Problem Report (PR) number PV-6-140247 was initiated to replace defective parts. During system leak check operations following valve repair, water flow fr

3、om zone 3 (directly over the Orbiter) occurred in OPF Bay 2. The time of first flow was established as 10:28 AM on September 24, 1989, as recorded by OPF Water Pump House instrumentation due to diesel pump startup. The Zone 3 water flow over the Orbiter was at a reduced rate, with only a partial noz

4、zle discharge pattern.Shuttle Processing Contract (SPC) Water Systems technicians were cycling the manual arming and firing valves while leak checking the system. Only a few minutes had passed from the time they cracked open (2 turns) the system manual supply valve until they heard about the water f

5、low problem in Bay 2. The SPC Lead Water Systems technician ran into the OPF to see what was wrong and noted the deluge flow. He returned to the manual activation station behind the OPF and found that two SPC ground support equipment (GSE) technicians had opened the manual arming and firing valve to

6、 Zones 1, 2, 3, 4, and 5. (It is presumed Zone 3 did not flow additional water and had been deactivated by this time.) The Lead Water Systems technician then closed all the open manual arming and firing valves, as well as the manual supply isolation valves to stop all Bay 2 water flow.The scenario t

7、hat lead to the two GSE technicians being at the deluge system manual flow control valve station is as follows:After the flow from Zone 3 occurred, the SPC OPF Bay 1 Site Division Manager asked a GSE technician to find someone who could turn the system off. This GSE technician, accompanied by a co-w

8、orker, proceeded to the manual activation station behind Bay 2. Upon arrival, they proceeded to Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-position the manual arming and firing valves to Zones 1, 2, 3, 4, and 5 to the “on“ position, thinking the

9、y were turning the firex system off. This turned on deluge system Zones 1, 2, 4, and 5 (Zone 3 had already been deactivated) to the “full on“ mode. The Lead Water Systems technician then returned to the manual valve panels, proceeded to turn the manual arming and firing valves to the “off“ position,

10、 and then helped his technicians close the manual supply isolation valves to Zones 1 through 5, which stopped all water flow.Lesson(s) Learned: The primary cause of the mishap was failure to follow the procedural instructions in PR number PV-6-140247, in which: (1) facility water technicians added a

11、 flow test on the Zone 3 system after valve diaphragm repair, and (2) in not fully closing or in improperly reopening the manual riser isolation valve, water was allowed to flow into Zone 3 of the OPF Bay 2 water deluge system.Contributing causes included the following:1. The OPF GSE technicians, wh

12、o are members of the “Contingency Team,“ had not been adequately trained in the operation of the OPF water deluge system. This led to the OPF GSE “Contingency Team“ technicians activating Zones 1, 2, 3, 4, and 5 of the Bay 2 deluge system, while thinking they were turning the system off.2. A thoroug

13、h pre-task briefing (PTB) was not held for the water deluge system valve repair task, which could have prevented deviation from the repair procedure by the facility water technicians.3. The labeling on the remote manual arming and firing valve activation station of the OPF water deluge system was mi

14、sinterpreted. This contributed to the OPF GSE technicians activating Zones 1, 2, 3, 4, and 5 of the Bay 2 deluge system when they believed they were turning the system off.4. The remote manual arming and firing valves are in the “on“ position when perpendicular to the water lines, which is not the i

15、ndustry norm for hydraulic systems. This contributed to the OPF GSE technicians believing they were closing the valves by positioning them perpendicular to the water line when, in fact, they were opening the valves. This human factor problem was not addressed in the System Assurance Analysis for the

16、 Water System at the OPF, SAA09FY09-007, Revision B, dated July 1987.5. The manual arming and firing valves behind the OPF are easily accessed by simply opening the individual valve box doors, which destroys the paper integrity seals. Essentially, there was no physical barrier to the valves.6. Even

17、though redundancy exists for the Viking valves in the deluge system, these valves are open when no dome pressure is applied. Failures which remove dome pressure could result in the flow of water.7. There are significant configuration differences between the OPF Bay 1 and 2 water deluge systems. The

18、OPF Bay 1 configuration does not include the manual riser isolation valve. Engineering Order E084-79KO5423, which installs the manual riser isolation valve for both bay configurations, has not yet been accomplished for Bay 1. This should be implemented for Provided by IHSNot for ResaleNo reproductio

19、n or networking permitted without license from IHS-,-,-Bay 1 after OV-103 rollout for STS-33 and prior to another Orbiter occupying Bay 1. There are only 6 boxes for the manual arming and firing valves in the Bay 1 configuration, with the arming and firing valves for a particular zone in the same bo

20、x. The Bay 2 configuration has 12 boxes, one for each valve. Signs on the boxes are not uniform between OPF Bays 1 and 2.8. The NASA Test Director (NTD) evacuated OPF Bay 2 via a public address announcement. He could not use the area warning system because OPF Bay 1 would have also been evacuated, w

21、hich was not required.9. As a result of the inadvertent activation of the firex deluge system, Orbiter GSE was damaged (i.e., the Mechanical Control Interface Unit for the Remote Manipulating Arm and the Cargo Verification Unit). No redundancy is available for the Mechanical Control Interface Unit.1

22、0. The OPF Bay 2 evacuation during this incident was in accordance with established procedures. Security control of Bay 2 was properly maintained. All personnel and emergency service actions were professional and properly carried out.Recommendation(s): 1. Emphasize to all individuals involved in the

23、 work task that proper procedure execution is mandatory.2. Access to the OPF deluge system Zones 1 through 6 should be positively controlled (e.g., a lock and key arrangement). Personnel with access should be fully trained and certified in the operation of the system.3. A policy should be developed

24、outlining which operations/ activities require PTBS. This policy should be rigorously implemented when developed.4. As part of the reassessment of the use of this system, all control panels should be properly labeled. A positive position for open versus closed should be readily visible on all valves

25、.5. The manual arming and firing valves should be oriented to the industry norm and provided with specific open and closed position markings.6. Access to OPF deluge system Zones 1 through 6 should be positively controlled (e.g., a lock and key arrangement). Personnel with access should be fully trai

26、ned and certified in the operation of the system.7. The policy of free access to the water deluge control panel key should be reviewed.8. Evaluate the risk of inadvertent OPF deluge system water flow with the current system design.9. The water deluge system valve repair task was not observed by Engi

27、neering, which could have prevented the deviation to written procedure by the facility water technicians.10. When malfunctions on critical facility/GSE systems occur which could affect flight hardware, a clear policy is required that outlines what level of management approval is needed to initiate r

28、epairs.11. The OPF area warning system should be split between Bay 1 and Bay 2, enabling the NTD to only evacuate one OPF bay when the emergency does not warrant evacuating both bays.12. Non-redundant GSE should be identified, and workarounds developed if this GSE is destroyed or incapacitated for a

29、n extended period of time.Evidence of Recurrence Control Effectiveness: Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-N/ADocuments Related to Lesson: N/AMission Directorate(s): a71 Space Operationsa71 Exploration SystemsAdditional Key Phrase(s): a7

30、1 Emergency Preparednessa71 Facilitiesa71 Fire Protectiona71 Ground Equipmenta71 Ground Operationsa71 Hardwarea71 Human Factorsa71 Logisticsa71 Mishap ReportingMishap Report References: Orbiter Processing Facility Bay 2 Water Deluge System Flow Mishap Investigation Board Report of October 25, 1989Additional Info: Approval Info: a71 Approval Date: 2002-04-17a71 Approval Name: Gena Bakera71 Approval Organization: KSCa71 Approval Phone Number: 321-867-4261Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-