REG NASA-LLIS-6216-2011 Lessons Learned - MSL Mobility Assembly Lift Mishap.pdf

1、Public Lessons Learned Entry: 6216 Lesson Info: Lesson Number: 6216 Lesson Date: 2011-06-21 Submitting Organization: JPL Submitted by: David Oberhettinger Subject: MSL Mobility Assembly Lift Mishap Abstract: JPL flight hardware was damaged during a lift operation preceding a test of the MSL rover Mo

2、bility Assembly due to an undetected mechanical interference that was signaled by an anomalous load cell reading. The load cell reading expected during the lift should be documented and communicated in pre-lift briefings, and the readings should be monitored continuously during the lift. Cognizant e

3、ngineers responsible for flight hardware lifts should receive intensive training in lift operations. Description of Driving Event: In January 2010, a characterization test was performed at the NASA/Caltech Jet Propulsion Laboratory (JPL) on the Mars Science Laboratory (MSL) Mobility Assembly (Refere

4、nces (1) and (2). The assembly was stored on a ground support equipment (GSE) Mobility Cart used for storage and transportation of this Mars rover flight hardware. The purpose of the test, using only one three-wheeled side of the rover suspension, was to move the bogie pivot through its full range o

5、f motion in order to verify bogie resolver functionality and obtain resolver calibration data. MSL has six wheels, each with its own motor (actuator). Like the previous Mars rovers, the MSL suspension connecting the drive wheels to the rover body employs a “rocker-bogie“ design that allows the rover

6、 to drive over obstacles while keeping the rover body balanced. With no axles or springs in the suspension, the three wheels on each side of the rover are connected by rockers, bogies, and pivots that distribute the load over the terrain (Figure 1). Resolvers are assemblies that sense the position o

7、f the actuator output shafts and mobility pivots; they are monitored during driving to halt the vehicle if the resolver readings fail to stay within an expected range. Figure 1. Initial MSL test configuration with the hardware mounted on the Mobility Cart and the “steering wheel” MGSE installed. A “

8、rocker-bogie” design features a two-wheeled rocker arm on a passive pivot attached to a one-wheeled bogie. During rover driving, this design allows the terrain to lift one wheel vertically while the other two wheels remain in contact with the ground. Provided by IHSNot for ResaleNo reproduction or n

9、etworking permitted without license from IHS-,-,-The test (Figure 2) took place in the JPL Spacecraft Assembly Facility (SAF), which is well equipped for mechanical testing. An overhead crane was used to raise the aft rocker assembly high enough off of the GSE cart that the bogie can be moved throug

10、h its full range of motion without making contact with the cart. During the lift, the cognizant engineer (CogE) for the Mobility Assembly was standing at the front of the test article at the rocker deploy pivot, and a flight hardware technician was standing at the back of the test article- also at t

11、he rocker deploy pivot. An additional mobility engineer was on hand calling the lift. Additional flight hardware technicians were stationed at the bogie pivot, on the crane, at the crane control (as the lift operator), at the aft wheel of the test article, and at the forward wheel. A quality assuran

12、ce engineer and a safety engineer were also present. Because the flight harness was new to testing, emphasis was placed on watching the harness during the lift to ensure that there were no pinch points and no over stressing of cables. Figure 2. Lift configuration with lifting sling attached to the f

13、light hardware When a soft rubbing sound was heard during the lift, the hardware CogE halted the operation and investigated. The sound was attributed to a zip tie that secured the cable service loop at the rocker deploy pivot rubbing on the “steering wheel“-a mechanical GSE (MGSE) clamshell ring att

14、ached to the Mobility Assembly support stand tower (Figure 3). The CogE directed the flight technicians to cut and remove the zip tie. The lift operation was continued, the sound was heard again, and the CogE stopped the lift. This time the sound was attributed to the flight harness because the harn

15、ess had not been present in previous lifts. The many service loops were checked for the location of the sound, as were the rocker deploy pivot and the bogie pivot, with no visible evidence that anything was amiss. The lift was resumed. One of the technicians noticed that the load cell read 330 lbs.,

16、 and asked if that was expected. The CogE nodded, performed a quick calculation to determine that the load was over twice the nominal reading, and began to call a halt to the lift. Just then the hardware made a loud pop and the load cell reading decreased to 150 lbs. Searching for the location of th

17、e pop, the team discovered that a nut on the back side of the latch pin that goes through the aft fitting of the rocker deploy pivot interfered with the “steering wheel“ MGSE (Figure 4). The pop sound and the high load resulted from the nut embedding in the MGSE, and the load was released when this

18、nut sheared off. Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Figure 3. Position of latch pin relative to MGSE in the flight hardware test Figure 4. Test failure, in which the “steering wheel“ MGSE (red fitting) has sheared the latching bolt and j

19、ammed the nut and washer, which blocked initial attempts to lower the flight hardware back onto the Mobility Cart The hardware was repaired. Stress analysis indicated it was unlikely that damage occurred outside of the localized region where the latch pin and aft fitting were in contact during the l

20、oading event. The lack of a disciplined methodology to identify potential test interferences between the flight hardware and the MGSE was an error in test preparations that contributed to the mishap. Previous bogie resolver testing on the rover chassis using the development test model (DTM) of the v

21、ehicle had been performed without the “steering wheel” MGSE present. The location of the interference was not clearly visible during the flight hardware lift activity and would not have been noticed unless it was disclosed during test preparations. A second error in preparing for the test was that t

22、he maximum expected load cell reading was not calculated by the CogE prior to the activity and communicated to the lift team. References: 1. “Mobility Rocker Deploy Pivot Hinge Pin Failure,” JPL Problem/Failure Report No. 15868, January 21, 2010. 2. “MSL Mobility Assembly Mishap Summary,” January 25

23、 2010. 3. J. Waydo, “Mobility Rocker Deploy Pivot: Path Forward,” January 25, 2010. 4. “MSL Backshell Crane Incident,” NASA Lesson Learned No. 5796, NASA Engineering Network, June 28, 2011. 5. “Aquarius EGSE Shipping Mishap,” NASA Lesson Learned No. 2456, NASA Engineering Network, February 16, 2010

24、 6. “NOAA-N Prime Mishap,” NASA Lesson Learned No. 1580, NASA Engineering Network, January 18, 2005. 7. “Genesis Canister Lift Incident,” NASA Lesson Learned No. 0914, NASA Engineering Network, July 20, 2000. 8. “MRO Articulation Keep-Out Zone Anomaly,” NASA Lesson Learned No. 2044, NASA Engineerin

25、g Network, April 7, 2009. Lesson(s) Learned: 1. Damage still occurs to NASA spacecraft flight systems or to other critical equipment that is preventable with Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-proper adherence to good handling practices.

26、 References (4) through (7) list other such incidents. Reference (8) further illustrates the need to identify range-of-motion interferences. 2. Neither drawings, nor the experience from previous tests of the developing system, can be relied upon to identify interference points or close approaches du

27、ring lift operations. In the case of the MSL mobility assembly lift, discovering the interference point between the flight hardware pin and the GSE would have required personnel watching the rotation from the rear of the GSE and knowing exactly where to look. 3. Errors in the planning and implementa

28、tion of the MSL test indicate that training in lift operations is important. JPL classes to train CogEs in lift operations were once scheduled periodically. Since the trainer retired and these classes were discontinued, these skills may have deteriorated. Recommendation(s): 1. The CogE for the fligh

29、t hardware under test is fully responsible for planning and conducting the test, including lift operations. The CogE should assure that: o The JPL Assembly and Inspection Data Sheet (AIDS) or procedures for the lift always list a step for verifying and recording the expected load cell reading. o Pre

30、lift briefings always contain a communication of the expected maximum load cell reading and identification of any expected hardware close approaches. Also, conduct an inspection of the test setup by the lift team to familiarize these personnel with the hardware and lift sequence. o When load cells

31、cannot be used, the CogE employs and communicates alternative processes to ensure hardware safety is not compromised. 2. Instead of placing total reliance on computer-aided design (CAD) drawings to identify interference points or close approaches that may be manifested during lift operations: o It i

32、s always prudent to perform a fit or rotation check prior to engaging critical hardware (flight hardware or GSE). o Assign an individual throughout the lift operation to observe the load cell and call out the numbers to assure that the expected load is not exceeded. o During this time, the CogE shou

33、ld perform a very active role, calling out “lift and hold, lift and hold” while moving around the test article and verbalizing possible anomalies. 3. Center-wide training classes should be held periodically to train JPL CogEs in conducting lift operations, covering such topics as mandatory lift proc

34、edures and lessons learned, participant qualifications and responsibilities, system safety and personnel safety practices, Center points-of-contact, etc. Evidence of Recurrence Control Effectiveness: JPL has referenced this lesson learned as additional rationale and guidance supporting Paragraph 6.1

35、2.5.3 (“Engineering Practices: Protection and Security of Flight Hardware”) in the Jet Propulsion Laboratory standard “Flight Project Practices, Rev. 7,” JPL DocID 58032, September 30, 2008. In addition, JPL has issued a Corrective Action Notice, CAN #1647, “Lift Operations Training / Good Handling

36、Practices,” to track JPL-wide, closed-loop action on this mishap. Documents Related to Lesson: N/A Mission Directorate(s): Science Exploration Systems Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Additional Key Phrase(s): Integration and Testing S

37、afety and Mission Assurance.Advanced planning of safety systems Additional Categories.Flight Equipment Additional Categories.Ground Equipment Additional Categories.Hardware Additional Categories.Test Article Additional Categories.Test & Verification Additional Categories.Spacecraft Additional Catego

38、ries.Safety & Mission Assurance Additional Categories.Payloads Additional Categories.Lifting Devices Systems Engineering and Analysis.Human factors planning Additional Categories.Test Facility Additional Info: Project: Mars Science Laboratory Approval Info: Approval Date: 2011-11-07 Approval Name: mbell Approval Organization: HQ Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-