REG NASA-LLIS-1219--2000 Lessons Learned - Integrated Configuration Control Troubleshooting.pdf

1、Lessons Learned Entry: 1219Lesson Info:a71 Lesson Number: 1219a71 Lesson Date: 2000-03-08a71 Submitting Organization: JSCa71 Submitted by: Dale Huls/ Ronald A. MontagueSubject: Integrated Configuration Control/Troubleshooting Description of Driving Event: The description that follows will demonstrat

2、e that pre-flight troubleshooting relied on history of a similar failure and did not check the actual configurations of certain subsystems or their components. Because troubleshooting did not go “outside the box“, there was no attempt to think beyond historical failures. This was compounded by a des

3、ign that did not provide a way to directly verify power was being delivered to the specific component - in this case specific channels to an electronic card.During the STS-88 (Flt 2A) Node 1 ECLSS equipment checkout, the Node 1 Sample Distribution System (SDS) forward and starboard valves failed to

4、change position when electrically commanded from the ground. Due to a previous failure history, it was incorrectly assumed that the SDS valve failures were attributed to internal binding. A subsequent troubleshooting and repair activity was authorized for STS-96 (Flt 2A.1) to try to repair the valve

5、s. This activity was unsuccessful.However, it was determined from data reviews post STS-96 procedure that the Remote Power Controller (RPC) was open rather than closed during all previous tests. The RPC provides power to the Solenoid Driver Output (SDO) card channels controlling these SDS valves. An

6、alysis has determined that the RPC power feed to the SDO had not been closed for the FWD and STBD SDS valves. Previously, the NADIR and SELECTOR SDS valves had been commanded on both the 2A and 2A.1 missions, with success both times. It turns out that these SDS valves are connected to an SDO card in

7、 which the RPC is always kept closed due to SDO card requirement to power the Multiplexer / Demultiplexer (MDM) heaters. A subsequent test conducted at GMT 320/1999 verified that the valves could be commanded to change position from the ground if power was applied to the appropriate SDO card. Flight

8、 Note EFCN934 SDS Valve Checkout - Inc B provided the proper power configuration. This problem is documented in PR# 988.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Root cause: An analysis has determined that an RPC power feed to the SDO card had

9、not been closed for the Node 1 FWD and STBD SDS valves preventing the valves from actuating by command. This was due to an oversight of the ISS 2A Operations Checklist to verify that the correct power configuration was enabled to perform the SDS valve checkout procedures. The Operations Checklist in

10、dicated what command would be sent to change the position of the SDS valve and the initial / final valve position indications, but no additional commanding to power the appropriate SDO cards. This appears to be a cross-subsystem problem where the ECLSS engineering knew how to command the valves and

11、the EPS engineering knew which RPCs were open or closed, but in the Flight Operational Readiness (FOR) review of these documents, this issue was missed by both.Lesson(s) Learned: When faced with an anomaly that cannot be explained from previous failure history, troubleshooters must be trained to “th

12、ink outside the box“ using integrated multi-disciplinary teams to investigate possible failure modesRecommendation(s): In developing future command procedures, the Mission Operations Directorate (MOD) and other mission support specialist (SE Subsystem Engineering, etc.) should take additional care t

13、o ensure that upstream power requirements are met in order to ensure success of hardware command (activation, checkout, test, etc.) activities. Evidence of Recurrence Control Effectiveness: Action taken at Huntington Beach: A multi-disciplinary team of design engineers reviews operational procedures

14、 before they are published. This ensures that expected configurations (such as the RPC switch closure and SDO switch closure) are accounted for. Also, the design of most of HB software involving EATCS valve closure includes two-step commanding (one step that in effect closes/opens the RPC switch and

15、 the second step that closes/opens the SDO switch). Finally, the design teams review the NASA MOD procedures as part of the FOR. Any missing steps by that time should be caught as part of the FOR.Action taken at Huntsville: Current processes used here in developing Operations procedures, require tha

16、t all SDO cards be powered up when the MDM is powered up and remain on while the MDM is operating. This will preclude a similar problem from occurring. If MOD does not want to keep all the SDO cards powered, then it is recommended that a step be added to verify/activate the appropriate cards as part

17、 of their operations procedures. Since this was an off-nominal test, a specific operations procedure was probably not developed beforehand for this condition. This type of scenario can/will occur again so it is imperative that the MER coordinate closely with the ESR hardware/software experts to work

18、 off-nominal procedures before their execution. This will insure correct procedures are implemented.Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Documents Related to Lesson: N/AMission Directorate(s): a71 Exploration Systemsa71 Space Operationsa71

19、 Aeronautics ResearchAdditional Key Phrase(s): a71 Configuration Managementa71 Hardwarea71 Independent Verification and Validationa71 Policy & Planninga71 Safety & Mission Assurancea71 Softwarea71 Test & VerificationAdditional Info: Approval Info: a71 Approval Date: 2002-06-27a71 Approval Name: Ronald A. Montaguea71 Approval Organization: JSCa71 Approval Phone Number: 281-483-8576Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-