REG NASA-LLIS-6536-2012 Lessons Learned - Communicate Requirements Associated with Analysis Tied to Required Verification Data.pdf

1、Public Lessons Learned Entry: 6536 Lesson Info: Lesson Number: 6536 Lesson Date: 2012-04-2 Submitting Organization: MSFC Submitted by: Jennifer Stevens Subject: Communicate Requirements Associated with Analysis Tied to Required Verification Data Abstract: A specific analysis was required to be perfo

2、rmed by a contractor in support of design, development, verification, and validation of ARES 1-X hardware. The Systems Engineering and Integration (SE&I) group of the prime contractor failed to adequately communicate the requirements associated with an analysis, including a requirement to use specif

3、ic, approved sources for material capability properties with test-verified certainty boundaries (variation dispersions) for use in design analysis to the subcontract engineering group that performed the analysis. The analysis was completed without being verified by the subcontractor or prime contrac

4、tor that the analysis methodology and assumptions complied with contractually stipulated verification data deliverable requirements. The error was found during a late life-cycle review, and the government engineering team was tasked to complete the proper analysis and documentation under extreme tim

5、e constraints to ensure proper verification of the design. The additional unplanned work added cost and time delay to the budget and schedule. Description of Driving Event: In the specific discipline area, the project office contracted with a prime contractor to produce the necessary engineering ana

6、lysis reports to verify that the hardware met the government furnished equipment (GFE) requirements at a component level. The prime contractor subcontracted the work to another company, who then assigned it to their analysis group. The analysis group already had previous experience performing analys

7、is on similar GFE, but for a different prime contractor. Prior to a major review, the prime contractor indicated their intention to verify subsystem safety requirements at the subsystem level rather than at the component level for this particular discipline. Verification compliance was to be summari

8、zed in a single report on a single Verification Data Requirements Sheet (VDRS). At that time, it was agreed that the subcontractor would document the GFE compliance status in a section of their analysis report, and that the prime contractor would include a reference to this report in their subsystem

9、-level integrated compliance report. This arrangement would eliminate the need for an additional set of component VDRSs. When the subcontractor analysis report was released, the subcontractor and prime contractor failed to ensure that the subcontractors work was consistent with the agreed upon verif

10、ication plan. The documentation was submitted to NASA without the agreed-upon component-level verification compliance matrix. The failure to review the product at the subcontractor and prime contractor levels resulted in a significant cost and schedule risk that threatened to delay the mission becau

11、se it was identified very late in the development lifecycle. The non-compliance was discovered during a major review by the governments insight/oversight team during final verification. Expedited and unplanned last minute effort was required to mitigate the impact to the schedule, at an unplanned co

12、st and repetition of technical analytical work to the program. It then became the governments responsibility to perform the compliance analysis. MSFC engineers identified several discrepancies with the report, primarily the use of material design allowable properties from unapproved sources, which h

13、ad to be addressed under deadline by MSFC Engineering. When the prime contractor released the subsystems Analysis Inspection Report, it did not include a section (or even placeholder) to document the GFE component compliance. Due to the time constraints, the MSFC component SE&I team was forced to in

14、itiate a separate set of VRDSs to document the component level verification and use the VRDSs to capture the compliance analysis that had not been included in the subcontractors report. Instead of reducing the level of effort and number of documents, the agreed-upon approach resulted in additional V

15、RDSs that added to the total number that had to be generated, reviewed, and approved by engineering and the project office. Lesson(s) Learned: 1. Communication of verification requirements is essential through all levels of hierarchy and to all organizations performing analysis or producing engineer

16、ing products that support verification and validation rationale. Responsibility for ensuring that work products comply with, and meet the standards of quality, validity and requirements, reside at each level of management and designated governing authority in the hierarchy of authority. Provided by

17、IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-2. Formal detailed verification requirements (DVRs) need to be strongly linked with individual tests and analyses. 3. Insight/oversight teams need to be reviewing technical work by contractors and subcontractors on

18、 an ongoing basis. This lesson also emphasizes the need for technical competence at the government level to monitor and verify that contractors comply with the customers requirements, and for the government to retain the capability to mitigate any failures of contractors to perform work as agreed to

19、. Recommendation(s): 1. Clear links between requirements and verification activities, including specific tests and analyses taken to satisfy the requirements, should be clearly established and maintained through all levels of hierarchy, from top level decision makers to those performing engineering

20、analysis. 2. Formal Detailed Verification Requirements (DVRs) should be strongly linked to test requests, contracts for analyses, and engineering products that support Verification and Validation of system and component design. 1. Including actual DVR references on test requests or work orders that

21、are issued prior to each test or analysis provide concrete and clear traceability and guidance for both the planning of the test or analysis and for the disposition and destination of engineering reports generated from test and analysis. 2. Traceability of detailed verification requirements is the r

22、esponsibility of each SE&I organization at each level of responsibility. Prior planning and consideration of how task orders and test requests are documented can support and clearly reference the relevant requirements underlying each technical activity. Prior planning prevents poor performance. 3. T

23、he risk of not establishing clear links between verification requirements and engineering products should be discussed and understood between project, engineering, and contractor management prior to execution of relevant work tasks. 1. If insight/oversight models or contractor task order agreements

24、preclude contractor-level managerial or designated governing authority review of deliverable products, the government program/project manager and government designated governing authority should establish a priori agreement of the risk acceptance and plan for risk mitigation. 2. Key decision makers

25、and stakeholders need to be aware of the potential risks of shortcutting traceability and review tasks. Evidence of Recurrence Control Effectiveness: Current project management/engineering leadership on another project have requested that the prime contractors SE&I and system development/test groups

26、 provide stronger links between the formal Detailed Verification Requirements (DVRs) and the individual subsystem and component tests that they are currently performing. Much of this work has been done, but the test-specific links are still not completely integrated at the time this lesson learned w

27、as written, increasing the likelihood that disconnects might not be identified until it is too late to address them. Documents Related to Lesson: N/A Mission Directorate(s): Exploration Systems Aeronautics Research Science Additional Key Phrase(s): Program Management.Program level review processes P

28、rogram Management.Contractor relationships Program Management.Configuration and data management Program Management.Risk management Program Management.Role of civil service technical staff versus contractor staff Missions and Systems Requirements Definition.Configuration control and data management M

29、issions and Systems Requirements Definition.DDT/E Engineering Design (Phase C/D).Lander Systems Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Engineering Design (Phase C/D).Entry Systems Systems Engineering and Analysis.Systems analysis - cost anal

30、ysis Systems Engineering and Analysis.Planning of requirements verification processes Systems Engineering and Analysis.Level II/III requirements definition Systems Engineering and Analysis.Engineering design and project processes and standards Missions and Systems Requirements Definition.Review boar

31、ds Missions and Systems Requirements Definition.Requirements critical to costing and cost credibility Program Management.Communications between different offices and contractor personnel Engineering Design (Phase C/D).Propulsion Engineering Design (Phase C/D).Spacecraft and Spacecraft Instruments Sa

32、fety and Mission Assurance.Early requirements and standards definition Safety and Mission Assurance.Review systems and boards Additional Categories.Parts, Materials, & Processes Additional Categories.Program and Project Management Engineering Design (Phase C/D).Launch Systems Additional Info: Project: Any launch vehicle, space craft, or development project at NASA Approval Info: Approval Date: 2012-06-25 Approval Name: ceng Approval Organization: HQ Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-