REG NASA-LLIS-4436-2011 Lessons Learned from Radioisotope Power Systems (RPS) Programmatic Review.pdf

1、Public Lessons Learned Entry: 4436 Lesson Info: Lesson Number: 4436 Submitting Organization: GRC Submitted by: Trudy Kortes Subject: Lessons Learned from Radioisotope Power Systems (RPS) Programmatic Review Abstract: The Radioisotope Power Systems (RPS) Program Office recently completed a System Req

2、uirements Review (SRR), System Definition Review (SDR) (per NPR 7123.1) and an Independent Life Cycle Review (ILCR) Program Approval Review (PAR) (per NID 7120.5) from May, 2010 - August, 2010 in preparation for Key Decision Point (KDP) I which takes the Program from a formulation phase to an implem

3、entation phase. During these reviews, some lessons learned noted were; Use review success criteria as a basis for the review agenda Ensure that the scope of the review is well communicated to all parties Arrange for a neutral party to observe during the review and provide feedback to the program/pro

4、ject office. Description of Driving Event: The Radioisotope Power Systems Programmatic SRR/SDR (May - August, 2010) and PAR (8/30/10 - 9/2/10) were conducted at the Glenn Research Center (GRC) in preparation for KDP I (11/30/10). Lesson(s) Learned: Planning: The Program utilized its draft Program Pl

5、an as an outline for the PAR based on benchmarking with other loosely coupled Agency programs. In hindsight, we would have served the Standing Review Board (SRB) better if we had developed the outline and agenda based on the review success criteria as detailed in NPR 7120.5 Section 2.5.4 and any suc

6、cess criteria noted in the Terms of Reference (TOR) Document. Know your audience! The SRB is an independent board formulated by the HQ Independent Program Assessment Office (IPAO). Once we had the names and titles of people confirmed as SRB members, it would have served us well to have a detailed me

7、eting with all presenters so that they were well aware of who would be in attendance, their areas of focus, and the key issues they might look to understand. The more dry runs, the better. The Program held several dry runs, a table top review with each individual presenter as a first glance at draft

8、 charts, a dry run of charts with the entire Program Office present, an “in-situ“ dry run held at the actual location of the event (very helpful to ensure your administrative support is familiar with AV equipment and that support need requests can be submitted in a timely fashion), and a chart run-t

9、hrough with Directorate management. However, it is recommended to include someone with Program or Project management experience who is familiar with Program or project reviews and completely independent from the office to take a fresh look at the chart package and how it will be presented. (If done

10、right, this could take up a persons entire day, so make sure that person has time and is willing to commit that time to helping out.) Another dry run that could be scheduled is specifically a “tough questions“ session in order to prepare presenters and also come to a common office understanding of h

11、ow to answer particular questions. Determine early in the planning process what is in and out of scope of the review and hold to that. SRB members may want to investigate an area that is outside of the scope of the review and it is completely appropriate to ensure that the scope of the review is und

12、erstood and adhered to. IPAO can assist with this agreement in the Terms of Reference Document. Make sure there is a lead integrator who knows the content of the presentation extremely well. Program management is focused on answering questions, and this function is necessary to facilitate the meetin

13、g especially when the SRB may want to skip topics or jump to particular topics. Scheduling an outside activity or two for an independent board was encouraged and appreciated, but do this early in the review. Once they get into the main part of the process, they will get busy and probably work well P

14、rovided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-into the evenings to complete the review on time, depending on the agenda laid out for them. Ensure all stakeholders/Program or project participants or representatives of those participants are present f

15、or a review of this proportion. This may mean Headquarters personnel (Program Executive or Director) or outside external agencies or other centers or matrixed personnel who are pivotal and key to your Program or project. If possible, have a neutral party present during the review. This should be som

16、eone who knows something about the Program or project but isnt necessarily directly involved. This allows things like real-time feedback so that adjustments can be made and also ensures valuable post-review feedback. Provided a pre-brief of the PAR material to functional line and programmatic line m

17、anagement, such as the HQ Program Executive and Director. Pay attention to repeat information. A certain amount of that is fine (for example showing the same chart a couple of times in different locations) to make sure the audience is familiar with key information. But, too much repetition could lea

18、d to a perception of “glossing over“ certain key information, so strike a good balance with chart integration. Emphasize to presenters that they thoroughly know the content of what they are presenting. Ensure that a comprehensive chart package is pulled together using the same formatting and templat

19、e. Our review had multiple presenters providing multiple chart packages. Provide a format up front for all to follow, but then have a chart making expert on staff put the final package together. Do not underestimate the time it takes to do this and do it well. RPS started producing charts for this r

20、eview in early July for an August 30 review and it was a difficult to complete in that timeframe. Allow 2 or more months for chart preparation. The amount of time and effort to plan and execute a review of this magnitude was underestimated. Review: If an SRB wants to spend more or less time on parti

21、cular topics, be flexible. Be prepared, but be willing to “call an audible“ real time in order to ensure the SRB is focusing on key topics. One of the unwritten goals of a review like this is to build credibility with the SRB which means being forthcoming in answering questions and providing informa

22、tion. Recommendation(s): Use review success criteria as a basis for the review agenda. Negotiate and finalize review success criteria with the IPAO early in the process. Ensure it is documented in the TOR. Inform the presentation team/program or project office of the audience and its focus. Perform

23、as many dry runs as possible within the constraints of the review schedule. Ensure that the scope of the review is well communicated to all parties. Ensure proper stakeholder participation during and at the review. Arrange for a neutral party to observe during the review and provide feedback to the

24、program/project office. Provide a pre-brief of the PAR material to functional line and programmatic line management, such as the HQ Program Executive and Director. Ensure that the Lead Integrator serves as the facilitator for the duration of the planning process and review. Allow enough time in the

25、schedule to develop a comprehensive, professionally produced chart package. Recommend a minimum of 3 months. Begin the planning process as early as possible. Recommend 6 months for a major review. Evidence of Recurrence Control Effectiveness: N/A Documents Related to Lesson: NPR 7120.5 Section 2.5.4

26、 and success criteria noted in the Terms of Reference (TOR) Document Mission Directorate(s): Aeronautics Research Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-Additional Key Phrase(s): Program Management Program level review processes Review boards Mission definition and planning Review systems Additional Info: Project: Radioisotope Power Systems Approval Info: Approval Date: 2011-01-26 Approval Name: mbell Approval Organization: HQ Provided by IHSNot for ResaleNo reproduction or networking permitted without license from IHS-,-,-