REG NASA-LLIS-1184--1991 Lessons Learned - Orbiter Fuel Cell Power Down Mishap Investigation Board Report of August 12 1991.pdf

1、Lessons Learned Entry: 1184Lesson Info:a71 Lesson Number: 1184a71 Lesson Date: 1991-08-12a71 Submitting Organization: KSCa71 Submitted by: Jackie E. Smith/Eric RaynorSubject: Orbiter Fuel Cell Power Down Mishap Investigation Board Report of August 12, 1991 Description of Driving Event: On August 12,

2、 1991, starting at approximately 3:58 a.m. EDT, two of the three fuel cells installed on Orbiter Atlantis, OV104, were possibly damaged when inadvertently left connected to the orbiter main busses for 16 hours and 36 minutes without water removal capability. The accumulation of water has the potenti

3、al for severe damage to the fuel cells. Atlantis landed at the Kennedy Space Center (KSC) on August 11, 1991, at 8:24 a.m. EDT after completing the STS-43 mission. The vehicle was undergoing deservice and safing operations in the Orbiter Processing Facility (OPF) High Bay 2 as part of the turnaround

4、 activities for its next mission, STS-44. The work effort was being accomplished by a NASA contractor.The mishap occurred when helium was inadvertently ingested into the fuel cell oxygen (02) supply causing a sudden performance loss by Fuel Cells 2 and 3 resulting in the loss of power to orbiter bus

5、ses B and C. This necessitated the implementation of the Emergency Power Down Procedure. The Emergency Power Down Procedure consisted of 6 crew module switch actions and was intended to electrically isolate the fuel cells from the orbiter main busses. The Emergency Power Down Procedure did not addre

6、ss the loss of multiple fuel cells with no vehicle ground power and, therefore, did not accomplish the required isolation of Fuel Cells 2 and 3 from the orbiter main busses. Emergency power down procedures have been used many times during shuttle processing, most often due to loss of orbiter cooling

7、. The Emergency Power Down Procedure did not include the provision that the only functioning bus should be the last commanded off. Since it did not address this, there was no power available to drive the Fuel Cell 2 and 3 to main bus motor switches to the open (isolate) position.During first shift a

8、nd early second shift on August 12, 1991, there were reports of occasional alarms and noises, such as fans operating, emanating from the “powered down“ orbiter. These reports, at first discounted, were investigated and determined to be resulting from the power generation of Fuel Provided by IHSNot f

9、or ResaleNo reproduction or networking permitted without license from IHS-,-,-Cells 2 and 3 which were still connected to the Orbiter Main Bus B and Main Bus C respectively. The fuel cells were removed from the busses at approximately 8:34 p.m. EDT August 12, 1991 by use of ground power through the

10、Orbiter ground umbilicals.Lesson(s) Learned: The chemical reaction within a fuel cell continues to generate electrical power and water when reactants (02 and H2) are present and electrical loads connected. In this case the Fuel Cells 2 and 3, with electrical loads connected, were at reduced voltage

11、levels because helium had mixed with the 02 in quantities sufficient to reduce the efficiency of the fuel cells.The helium ingestion was a result of an inadequate test procedure that performed ground support equipment (GSE) setups for controlled venting, fuel cell power down, cryogenic drain. This p

12、rocedure had been reviewed and approved by the appropriate engineering organizations. STS-43 was the third KSC landing since the Challenger accident and the first time the GSE back pressure regulator confidence test was performed while the fuel cells were powered. The pressure specified in the test

13、procedure for the GSE back pressure regulator exceeded the vehicle 02 manifold pressure allowing GSE helium to enter the fuel cell 02 supply. The power reactanta storage and distribution fuel cell (PRSD/FC) Engineer thought the GSE pressure could be controlled by additional valve cycling not covered

14、 in the procedure and elected to perform the back pressure regulator test instead of implementing a real time change to the test procedure. The GSE pressure exceeded the orbiter onboard flight disconnect specification cracking pressure by 12 psi and 6 psi for five seconds and eight seconds respectiv

15、ely. GSE pressure levels commanded by the PRSD/FC engineer were within limits defined in the test procedure.Ingestion of helium into the fuel cells does not harm them, although it does degrade performance. This performance loss initiated a sequence of events which led to potential damage to Fuel Cel

16、ls 2 and 3.The Shuttle emergency power down procedures must react to all possible contingencies and configure the hardware to an absolute safe condition. The test teams reaction to unplanned events must be preplanned and implemented without hesitation or deliberation. A fuel cell is considered the m

17、ost reliable power source available and therefore multiple failures were not considered to be a realistic occurrence. Engineering and management, both government and contractor, had developed a feeling of security regarding emergency power down. The Emergency Power.Down procedure was developed in 19

18、79 for the STS-1 mission, reviewed as part of the post-Challenger Return-to-Flight activities, and has remained essentially unchanged. In retrospect, the technical community should have recognized that these were unusual circumstances, ie., loss of 2 fuel cells, no ground power, etc., and evaluated

19、this specific use of the emergency power down procedure. Without this recognition, the obscure nature of this procedural inadequacy makes it improbable that the failure to disconnect the fuel cells would have been discovered.Provided by IHSNot for ResaleNo reproduction or networking permitted withou

20、t license from IHS-,-,-No one was injured in the mishap. Extent of damage to Fuel Cells 2 and 3 will be determined after evaluation is accomplished at the original equipment manufacturer. Fuel Cells 2 and 3 had used approximately 500 hours and 1500 hours respectively of the 2000 hours of specificati

21、on operating life. Other orbiter electrical components that were exposed to off nominal bus voltages are under evaluation.For additional information on this topic, please also see LLIS lessons #41 and #134.Recommendation(s): PRIMARY CAUSE:Fuel Cells 2 and 3 were not isolated from orbiter main busses

22、 when commanded. The emergency power down procedure did not address the loss of multiple fuel cells with no vehicle ground power and therefore did not accomplish the required safing.RECOMMENDATION: Review emergency power down procedures to assure that fuel cell failures are appropriately addressed R

23、eview should consider: Precautionary note regarding sequence of fuel cell removal from busses; Use of crew module panel indicators as part of configuration verification; and Precautionary note regarding possibility of invalid data.CONTRIBUTING CAUSE: Sudden performance degradation of Fuel Cells 2 an

24、d 3 as a result of helium ingestion into the 02 supply caused the loss of Main Busses B and C. Helium was ingested via the oxygen horizontal drain disconnect as a result of an inadequate test procedure. The pressure specified in the test procedure for the GSE Back Pressure Regulator (BPR) exceeded t

25、he orbiter pressure allowing GSE helium to enter the fuel cell 02 supply.RECOMMENDATIONS: Consider a positive means to isolate GSE from vehicle during GSE pressure testing. Review OMI V1091, “Orbiter PRSD Cryo Drain“ to: Maintain GSE helium pressure below vehicle reactant pressure when fuel cells ar

26、e operating. Include caution note prior to GSE pressurization steps. Record vehicle pressure at appropriate locations. Alert electrical power distribution (EPD) and other appropriate systems of critical fuel cell operations.OBSERVATIONS:1) The PRSD/FC Engineer, aware of vehicle pressure, attempted t

27、o limit Oxygen Horizontal Drain (OHD) pressure during the Back Pressure Regulator (BPR) adjustment by cycling GSE supply valves instead of initiating a procedural deviation.RECOMMENDATION: Engineers should be cautioned to critically evaluate borderline conditions.2) Due to the obscure nature of the

28、failure, the problem analysis did not recognize that Fuel Cells 2 Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-and 3 were still connected to main busses after power down.RECOMMENDATION: Vehicle configuration should be carefully verified after reco

29、very from emergencies or anomalous conditions.3) While aware of the need to prevent helium ingestion, the PRSD/FC Engineer assumed pressure instrumentation was adequate to indicate presence of helium in quantities harmful to fuel cell performance.RECOMMENDATION: Re-emphasize to PRSD/FC operators the

30、 sensitivity of fuel cells to any helium ingestion. Evaluate the use of vehicle instrumentation that is more responsive to fuel cell performance changes.4) Vehicle data stopped updating prior to vehicle power down. Most test team members were unaware of vehicle data loss and that the motor switches

31、did not open.RECOMMENDATION: Consider implementation of an obvious flag/color change within data display system to indicate data has stopped updating.5) When fuel cells are operating without ground power, flight hardware is vulnerable to GSE activities.RECOMMENDATION: Review operations and GSE desig

32、n to minimize times when fuel cells are operating without redundant ground power.6) Although not referenced in the OMI, the PRSD/FC engineer relied upon a 20 PSID operating specification value for the horizontal drain flight poppet to prevent flow into the orbiter.RECOMMENDATION: Instruct engineers

33、not to use delta pressure across system components in determining safe GSE to vehicle pressure margins.7) Orbiter fire alarm activation (due to low bus voltage) provided false clues to the nature of the problem.RECOMMENDATION: Alert test team of this condition.Evidence of Recurrence Control Effectiv

34、eness: N/ADocuments Related to Lesson: Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-N/AMission Directorate(s): a71 Space Operationsa71 Exploration SystemsAdditional Key Phrase(s): a71 Configuration Managementa71 Cryogenic Systemsa71 Energetic Mate

35、rials - Explosive/Propellant/Pyrotechnica71 Energya71 Flight Equipmenta71 Ground Equipmenta71 Ground Operationsa71 Hardwarea71 Human Factorsa71 Launch Processa71 Logisticsa71 Mishap Reportinga71 Packaging Handling Storagea71 Test & Verificationa71 Test ArticleMishap Report References: Orbiter Fuel C

36、ell Power Down Mishap Investigation Board Report of August 12, 1991Additional Info: Approval Info: a71 Approval Date: 2002-04-30a71 Approval Name: Gena Bakera71 Approval Organization: KSCa71 Approval Phone Number: 321-867-4261Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-