1、EIA STANDARD Systems Engineering Capability Model Appraisal Method EIA-731.2 (Formerly EMS-731.2) AUGUST 2002 ELECTRONIC INDUSTRIES ALLIANCE GOVERNMENT ELECTRONICS AND INFORMATION TECHNOLOGY ASSOCIATION Copyright Government Electronics however, it is not the intent of this document to be a substitut
2、e for appraisal training materials. How this Appraisal Method should be used This Appraisal Method description was written to support an appraisal using the SECM as a reference model. This Appraisal Method description was not written to meet the needs of a third-party evaluation or audit. An apprais
3、al of an organizations systems engineering capabilities prmarily provides information useful for improving those capabilities vis-vis the SECM. This Appraisal Method can be performed as a self-appraisal or conducted by or with the assistance of an outside organization or consultant. Appraisal Method
4、 developers This Appraisal Method was developed by the EIA G47 SECM Working Group that developed EIA-73 1 , Part 1. V Copyright Government Electronics e.g., training, customers, and suppliers). Practitioners are a source of data, primarily via the focus groups, and also as a feedback validation chec
5、k of the appraisal findings. Practitioners complete all or parts of the questionnaire. The group that is responsible for defining and maintaining engineering processes and assessment methods within the organization being appraised. This term also includes the external organization responsible for ma
6、intaining the SECM standard (EIA.) Individual providing the resources for the appraisal and the commitment to the process improvement effort. It is particularly important for the sponsor to show commitment by attending both the opening and closing briefings of the On-Site phase, and especially accep
7、ting and indicating action that will occur as a result of the outbrief Familiarizing the appraisal team with the Appraisal Method, Presenting the appraisal findings and developing the appraisal report. Obtaining facilities for the On-Site phase, Scheduling activities during the On-Site phase, Admini
8、stering and collecting questionnaires, and Ensuring personnel attendance, as appropriate. Product life cycle leaders (who know all or most of what is performed in the organization; i.e., interviewees that complete a questionnaire), and Practitioners (technical, support, and management). Completes th
9、e Appraisal Method questionnaire, Participates in an interview, or question and answer session, and Is a primary source of feedback on the validity of the findings. Copyright Government Electronics 1.5 Integrate system 1.6 Verify System 1.7 Validate System Levei2 I Level 3 Level 4 Level 5 Figure 2-3
10、. Sample Format for Depicting Focus Area Ratings (Technical Category) 2.5 How to Read the Process Activity Summaries All process activity descriptions are similarly formatted, and contain some or all of the information blocks described in Table 2-5. For each major phase (Preparation, On-Site, Post-A
11、ppraisal), the summary description contains the process activities that comprise that phase; for the process activities, the summary description contains the actual description of that activity. Table 2-5 - Process Activity Summaries Organization Copyright Government Electronics select training mate
12、rials; and train appraisal team. Administer questionnaire; review any appropriate documents and artifacts; complete initial practice baseline on spreadsheet. Output Appraisal resources Agreement to proceed and expend resources for the planning and conduct of the appraisal Appraisal goals related to
13、business goals Sponsor role understood Preliminary appraisal plan that defines appraisal parameters based on appraisal goals Approved appraisal plan Identified extent of necessary appraisal team training materials Appraisal tools and Practice baseline Document and artifact review notes Table 3-2 lis
14、ts the primary roles involved in the preparation phase and a summary of their activity during this phase. Table 3-2 - Participants for Preparation Typical Duration: Preparation requires two to eight weeks, depending on the complexity of the appraisal selected and other site parameters. Copyright Gov
15、ernment Electronics engaging the sponsor in dialogue to determine hisher goals for the appraisal; and negotiating a commitment for resources, including personnel and funding for the appraisal activities. A primary aspect of obtaining sponsor commitment is ensuring that the sponsor understands hisher
16、 role in the appraisal. For tailoring guidance, read the related paragraphs in Section 6. For checklist and recommended approach, see Annex F. Major Participants: Table 3-3 lists the primary roles involved in this process activity and a summary of their tasks during this process. Copyright Governmen
17、t Electronics commits resources for the appraisal. Engages sponsor in dialogue to understand the appraisal method, outcomes, and expected sponsorship activities. Provides sponsor with information needed to make a commitment decision on the appraisal. Provides a link between the appraisal sponsor and
18、 participants. Typical Duration: This activity typically takes 1 to 3 hours. Tailorable Parameters: Method of interaction with sponsor: All face-to-face meetings, Etc. Number of interactions with sponsor Combination of telephonelvideo and face to face meetings, Combination of written communication a
19、nd face-to-face meetings, Exit Criteria: Appraisal goals established Sponsor commitment to provide appraisal resources obtained Sponsor commitment to appropriate behavior during the appraisal obtained Notes: This is a golno-go decision point. If sponsor commitment for the appraisal is not obtained,
20、no further process activities related to the Appraisal Method will be performed. In particular, the sponsor must agree to accept the findings, thank the team and participants, and indicate what helshe intends to do to act on one or more of the findings. Guidance on using business goals and organizat
21、ion context for tailoring the appraisal is addressed in Section 6. 3.2.2 Select Appraisal Parameters Purpose: Select Appraisal Parameters determines how the Appraisal Method needs to be tailored to meet the goals established for the appraisal with the sponsor. Copyright Government Electronics engage
22、s the organization to determine the makeup of the appraisal team and key appraisal participants. Produces appraisal outline. Engages in dialogue with site coordinator andor facilitator and provides appropriate input on characteristics of team and organizational units to be included in appraisal. Typ
23、ical Duration: Tailoring requires one to two weeks, depending on the complexity of tailoring. Tailorable Parameters: Extent of appraisal plan documentation Exit Criteria: Appraisal team selected Key appraisal participants selected Preliminary appraisal outline developed Product lines and organizatio
24、nal units to be included are selected Notes: This is the activity where tailoring decisions are made and documented, based on the goals of the appraisal. Guidance on tailoring the appraisal to support the organizations business goals is addressed in Section 6. Copyright Government Electronics requir
25、ementddesigners; production/integration/testers; and product field support/disposal), organizational boundaries (organizational support elements together; engineering/ production; and marketing/delivery and retirement), or in other ways that will provide synergy of the information obtained. Groups t
26、hat have feuds probably should not be grouped together. A program manager group might be set up if they cross organizational boundaries. The point is to use the particular organizational context to the best advantage to corroborate data obtained via the questionnaires and interviews. Middle manager
27、buy-in may be obtained by having a focus group before the On-Site period, to provide an opportunity to engage them as “owners” of the action plan recommendations. In addition, in some organizations, marketing or business development will “own” product line evolution, and should be included in practi
28、tioner groups or formed into a mini-focus group. Any useful documents, such as policies, process descriptions (often in training materials), standards, even meeting minutes or action item lists, may be requested, collected, and stored for use by the appraisal team. External appraisal teams typically
29、 sign non-disclosure agreements with the appraised organization. A team may request and use documentation generated for or by Focus Area activities before or during the appraisal On-Site period. Sufficient time should be allocated for this task. To the extent possible, pre-On- Site reading of summar
30、ies or documentation should be accomplished. Templates or outlines for what information to capture and how to capture it should be determined with the facilitator and communicated to team members and those providing the information. See Section 6 and Annex F, Site Coordination Checklist, for details
31、 on preparing for the logistics of an appraisal. 3.2.4 Collect Questionnaire Data Purpose: Collect Questionnaire Data obtains and records profile information on the appraised organizational units. Summary Description: Collect Questionnaire Data involves administering and collecting results from the
32、questionnaire and organization profile instruments. The appraisal team leader provides training for the appraisal participants and administers the questionnaire. The questionnaire rephrases the Specific and Generic Practices of the reference model into questions that can be answered with: yes; no; d
33、ont know; or not applicable. The questionnaire responses are transferred to the Data Tracking Sheet, either in this activity or in the “Analyze Questionnaire” activity (paragraph 4.1.3). For tailoring guidance, read the related paragraphs in Section 6. Copyright Government Electronics & Information
34、Technology Association Reproduced by IHS under license with GEIA Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-EIA-73 1.2 Page 16 Major Participants: Table 3-6 lists the primary roles involved in this process activity and the summary of their tasks during this pr
35、ocess. Table 3-6 - Participants for Collect Questionnaire Data Site Coordinator Appraisal team leader or Summary The selected participants provide data on programs and the organization via the questionnaire. The questionnaire may be administered in various ways, e.g., all participants together in a
36、face- to-face meeting, as an on-line response activity, or mail-ins. It is recommended that the questionnaire be administered to the group, with a moderator, site coordinator, or other person with knowledge about the model present to answer questions of terminology, etc. Determines respondents of th
37、e questionnaires, makes arrangements for them to complete the questionnaire, and may oversee administration. Conducts any meetings related to introducing the data collection activity. Ensures that all participants are made available for data collection. Typical Duration: Collect Questionnaire Data r
38、equires two to three hours per participant. Tailorable Parameters: Method of administering questionnaires Contents of questionnaire, based on appraisal goals (e.g., if only appraising the management Focus Areas, only use those questionnaire items) Who completes what parts of the questionnaire Amount
39、 of analysis performed prior to On-Site phase Transference of questionnaire responses to the Data Tracking Sheet (DTS) Exit Criteria: Appraisal scope information (what different parts of the organization provide, tailored model to be appraised against, participants, etc.) received by appraisal team
40、Filled-in questionnaires received by site coordinator Optional: If it is decided to transfer questionnaire responses to the Data Tracking Sheet prior to the On- Site phase, the transfer should be completed at the end of this process step Notes: See the questionnaire distribution table in Section 6.6
41、 and the Appraisal Method questionnaire instructions in Annex F (Site Coordination Checklist) and Annex I (Appraisal Questionnaire) for information on using the questionnaire to collect data. See paragraph 4.1.5 for details on transferring the questionnaire responses to the Data Tracking Sheet (DTS)
42、. Copyright Government Electronics & Information Technology Association Reproduced by IHS under license with GEIA Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-EIA-73 1.2 Page 17 4 On-Site Phase Activities 4.1 On-Site Phase Overview Purpose: The On-Site phase exp
43、lores the results of the preliminary data analysis, provides an opportunity for practitioners at the organizational units being appraised to participate in the data gathering and validation process, and provides practitioners and management with input on the results of the appraisal. Figure 4-1 show
44、s the activities in the on-site phase in a sequential manner. In some cases, the team may be able to perform some activities in parallel. The On-Site phase may occur as several site visits. Annex C shows some alternative On-Site schedules that may be followed. Conduct opening Fam iliarizc tcam Analy
45、zc Consolidate data O n-S ite Phase Activities Consolidatc data prclim inary Rcvicw prclim inary Figure 4-1. Diagram of On-Site Phase Summary Description: k Prcscnt Bricf sponsor Conduct Table 4-1 lists the major activities of the On-Site phase and the expected output of each. Each element is descri
46、bed more fully in the summaries that follow. Copyright Government Electronics & Information Technology Association Reproduced by IHS under license with GEIA Not for ResaleNo reproduction or networking permitted without license from IHS-,-,-EIA-73 1.2 Page 18 Table 4-1 - Summary Description of the On
47、-Site Phase ID 4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6 4.2.7 4.2.8 4.2.9 4.2.10 4.2.1 1 Activity Conduct opening meeting Familiarize team with the AM Analyze questionnaire Interview product life cycle leaders or project leaders Consolidate data from interviews Interview practitioners Consolidate data fr
48、om practitioners Develop preliminary findings Review preliminary findings Develop the ratings Develop draft findings Description The opening meeting provides participants with an overview of the appraisal and the context of the appraisal (e.g., schedule and confidentiality). The sponsor expresses su
49、pport of the appraisal and subsequent process improvement activities. The team leader instructs the appraisal team on the detailed conduct of the appraisal activities and introduces the model as tailored for this appraisal. The appraisal team analyzes the responses to the questionnaire and formulates a set of follow-on, exploratory questions for the interviews and focus OUDS. Through structured interview techniques using the exploratory questions, the appraisal team gathers corroborating data regarding the project practices. The te
