REG NASA-LLIS-1340-1992 Lessons Learned SSME 2032 Test 901-674 Shutdown Mishap Investigation Board Report of March 3 1992.pdf

1、Lessons Learned Entry: 1340Lesson Info:a71 Lesson Number: 1340a71 Lesson Date: 1992-03-03a71 Submitting Organization: SSCa71 Submitted by: Eric Raynor/ O.K. GoetzSubject: SSME 2032 Test 901-674 Shutdown Mishap Investigation Board Report of March 3, 1992 Description of Driving Event: Space Shuttle Ma

2、in Engine (SSME) test 901-0674 was conducted on November 6, 1991 at 11:31 am CDT. The scheduled 400 second test was terminated by the Command and Data Simulator (CADS) at engine start plus 3.72 seconds when the Low Pressure Fuel pump (LPFP) discharge pressure sensor was disqualified by exceeding its

3、 maximum qualification limit of 300 psia. The controller responded by issuing a Major Component Failure (MCF) which initiated the CADS cutoff.The main objectives of test 901-0674 were: reacceptance of flight engine 2032 with replaced oversized piston ring seal wave spring, facility flow meter calibr

4、ations, and the greenrun of the following flight hardware: Controller U/N F48, High Pressure Fuel Turbopump U/N 2226, Low Pressure Fuel Turbopump U/N 4018, Low Pressure Oxygen Turbopump U/N 2035, and Chamber Coolant Valve Actuator (CCVA) S/N 037-71008. The propellant system chilidown and prestart en

5、gine conditioning phase of the test was normal.Post test inspections indicated no external engine or facility damage. Internal borescope inspections revealed heavy erosion of the High Pressure Fuel Turbopump (HPFTP) first stage nozzle and turbine blades. No other internal damage was noted. Post test

6、 inspection of the CCV/CCVA assembly revealed the coupler which links the CCVA to the CCV was missing. Inspection of the CCV also revealed the valve to be fully closed throughout the test.Lesson(s) Learned: With the actuator coupling component left out, the actuator movement did not translate into a

7、ctual Chamber Coolant Valve movement. This resulted in additional fuel flow to the MCC and Nozzle coolant circuit and reduced fuel flow to the preburners. The increased MCC coolant flow resulted in an increased LPFP turbine flow which increased its speed and its discharge pressure which in turn Prov

8、ided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-exceeded the LPFP discharge pressure sensor qualification limit and resulted in an MCF. The lack of sufficient fuel to the preburners caused the high turbine discharge temperatures which approached the redl

9、ines. The shutdown due to the MCF occurred via the standard hydraulic sequence and it was a safe shutdown.For additional lessons learned please also see the findings listed under the “recommendation“ section below.Recommendation(s): 1. Finding: The procedures at Stennis Space Center (SSC) are design

10、ed to provide maximum flexibility and adaptability. Each procedure attempts to anticipate all possible situations which leads to very voluminous and complex procedures that are difficult to follow.1. Recommendation: Restructure and re-format the SSC procedures to reduce complexity, ambiguity and to

11、enhance the probability of being understood by technicians and inspectors prior to performing the work. Consider individual planning for unique jobs with the aid of computers and include/adopt either the KSC or Canoga Park system or elements thereof.2. Finding: The initial release of a SSC procedure

12、 is formally controlled, however, subsequent modifications in the field are not, and the responsibility for establishing the options is unclear. The system relies on redlining and DNAing major sections by technicians, inspectors, engineering, and quality engineering to custom fit the procedure to th

13、e specific circumstances at hand. Technicians and inspectors routinely DNA and redline procedures. In some cases, the hardware is not subsequently under the control of the person selecting the DNA option.2. Recommendation: Procedures should be reviewed and approved by engineering for accuracy and re

14、vision as required for the identified task prior to release to the floor. Decisions affecting hardware configuration should be made by engineering in the course of the review; not left up to the floor technicians and inspectors. DNA/NA options to be exercised on the floor should be limited to those

15、actions where results are immediately verifiable and obvious. DNA/NA options which require decisions where the results are not under the control of the affected line personnel should be disallowed, and if situation changes, new planning should be issued. A procedure deviation system for initiating d

16、eviations to correct errors/oversights would be helpful. Deviations should be reviewed prior to the use of a procedure. Redlines should be limited to minor pen and ink changes not affecting intent, e.g., typographical errors.3. Finding: Disciplined implementation of existing procedures was inadequat

17、e by floor technicians and quality assurance personnel. Technicians performed five procedural steps in the CCVA procedure (Rework #49) that, if properly accomplished, would have disclosed the missing hardware. The QA inspector stamped four of these five. Three other steps were performed in this proc

18、edure that would have possibly disclosed the deficiency. The procedure Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-was clear in these areas and the careful accomplishment of these actions with all parties properly fulfilling their roles would hav

19、e prevented this problem.3. Recommendation: Personnel performing operations must fully understand the importance of their tasks and consequences of incorrect action(s). They must understand the significance of their stamps on the planning whereby they certify and agree to the accomplishment/verifica

20、tion of the specific task(s). Technicians and inspectors should not stamp work they have not personally witnessed. Reinforcement of these responsibilities through additional training/orientation by supervision appears to be warranted to assure the proper level of discipline is in place.4. Finding: P

21、erformance in accordance with the current procedural system does not assure that the hardware configuration is under control through all phases of disassembly/reassembly.4. Recommendation: The procedural control of hardware configuration needs to be addressed and changes instituted to assure the har

22、dware configuration is known and under control at all times. This should apply to the transfer of hardware status from one type of paper to another, to procedures controlling the hardware transfer, storage, issuance, and to the procedures for verifying configuration integrity.5. Finding: A major SSC

23、 planning document deficiency is the absence of a list of parts required to start the job and of parts to be left over after the job.5. Recommendation: Incorporate in planning a list of required parts and a list of parts to be left after the job.6. Finding: The existence and location of flight and t

24、est hardware on test stand A-1 appears to be uncontrolled. The uninstalled coupling was found on level 5. There was a line assembly on level 5 that belonged on level 8. On level 8, a shim was found that should have been left in building 3202.6. Recommendation: Institute a system that inventories and

25、 controls the location of test and flight hardware on all test stands. A preferred approach would be a centralized location rather than multiple locations. The parts crib should be a controlled access area with an individual responsible for inventory of incoming parts and issue to technicians. Parts

26、 should be segregated and identified in the crib in such a manner that they can be unambiguously located and retrieved for a particular job/engine.7. Finding: Flight engine 2032 was shipped from building 3202 to the A-1 test stand with a considerable amount of open work and a sizable box approximate

27、ly 4 X 8 X 4 feet with loose parts. The exact configuration of the engine as shipped was not clearly defined and the absence of parts lists did not force reacting to an unaccounted left-over part like the coupling.Provided by IHSNot for ResaleNo reproduction or networking permitted without license f

28、rom IHS-,-,-7. Recommendation: Discontinue the practice of shipping flight engines with open work unless the work is related to high pressure pumps. Provide an inventory list of loose parts shipped with the engine and to be installed, and a list of parts to be removed from the engine and centrally c

29、ollected prior to hot fire testing. Institute a system that reacts to parts left over and unaccounted for. If parts to be installed are shipped separately, they are to be identified on the inventory list not as being in the shipping box but as still to be delivered. A verification system should be i

30、n place at the engine position that provides assurance of hardware configuration.8. Finding: Technicians and inspectors interviewed did not know that the actuator which was to be installed was new and being installed for green run purposes. It appeared that the working people in general are not bein

31、g briefed as to long term objectives.8. Recommendation: Establish an information system such that also technicians are aware of long term objectives and not just near term objectives of getting dates, paragraphs stamped off, and the paper closed.9. Finding: No clear identification of a responsible d

32、evelopment engineer having full responsibility for the successful accomplishment of a given task.9. Recommendation: Assure the clear identification of a lead development engineer and full delineation of his responsibilities for each job.10. Finding: Integration of Government Agency and contractor pr

33、ocedures is inadequate as evidenced by the existence of conflicting instructions concerning mandatory inspection points.10. Recommendation: The Government Agency letter dated November 24, 1987, concerning a post activity closed paper review of DNA/NA operations having a Government Agency review with

34、in them should be rescinded.11. Evidence of Recurrence Control Effectiveness: N/ADocuments Related to Lesson: N/AMission Directorate(s): a71 Space OperationsProvided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-a71 Exploration SystemsAdditional Key Phrase(

35、s): a71 Administration/Organizationa71 Communication Systemsa71 Configuration Managementa71 Facilitiesa71 Flight Equipmenta71 Ground Equipmenta71 Ground Operationsa71 Hardwarea71 Human Factorsa71 Human Resources & Educationa71 Logisticsa71 Packaging Handling Storagea71 Parts Materials & Processesa71

36、 Policy & Planninga71 Safety & Mission Assurancea71 Test & Verificationa71 Test Articlea71 Test Facilitya71 Training Equipmenta71 TransportationAdditional Info: Approval Info: a71 Approval Date: 2003-06-26a71 Approval Name: Buddy Newbolda71 Approval Organization: SSCa71 Approval Phone Number: 228-688-3152Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-