4.2.1 Coordinating the Evaluation
When a specific software development project has been identified for
evaluation, the customer organization is required to designate a person to be
the Site Coordinator. The Site Coordinator is critical to the success of the
software risk evaluation, serving as the single interface between the
Evaluation Team Coordinator and the project being reviewed. The Site
Coordinator is armed with a SRE methodology support product, the Site
Coordinator's Guide, to serve as a checklist of tasks to complete in
preparation for the SRE. This guide also summarizes the purpose of software
risk evaluations, and the roles and responsibilities of the
The Site Coordinator requires ready access to the identified project
manager and must be able to speak with the authority of the project manager
when working to synchronize the schedules of the participants involved in the
evaluation. Furthermore, the Site Coordinator is strongly encouraged to
identify points of contact in the affected organizations (e.g., acquisition,
program management, contractor project management) to share the logistics
4.2.2 Building the Evaluation Team
The Software Risk Evaluation team is made up of both technical experts and
customer organization (e.g., acquisition) personnel. This membership gives the
team a technical, product-oriented focus and equips the team to work in detail
with staff members at ail levels of the project. If the evaluation is an SEI-
assisted SRE, then the team will also include SEI membership. Field test
experience suggests the total evaluation team size should not exceed eight (8)
members, nor fall below four (4) members. Any decision to staff a team with
less than four members should be regarded as a trade-off between the cost of
resources and the objectivity of the team.
The team members must possess a working knowledge of software development
processes [Paulk], the technology used on the project, the application domain.
and the government acquisition system. Team members must also possess
excellent communication skills. The most critical requirement is that all team
members must receive training in the SRE methodology.
The Site Coordinator arranges for read-ahead materials to be forwarded to
the evaluation team members to familiarize them with the technical,
managerial, and political background of the software development project being
evaluated. A complete listing of the type of materials required is included in
the Site Coordinator's Guide.
4.2.3 Setting the Evaluation Stage
A critical input required by the Evaluation Team Coordinator is the
customer's documented perception of the objectives of the SRE. This
information drives the SRE charter and, as such, sets the tone for the entire
evaluation. The Site Coordinator coordinates with responsible management in
the customer organization to produce this information, then forwards it as
input to the Evaluation Team Coordinator.
Finally, the Site Coordinator is responsible for coordinating the SRE
schedule of events. Figure 4-2 represents what the SEI Risk Program considers
to be an optimal schedule in terms of quality of results and economy of
application. The Site Coordinator works with on-site points of contact to
determine the appropriate audience for each of the SRE sessions (i.e..
Orientation Briefings, Team Training Briefing, Interview Sessions, Risk
Ranking, Results Outbrief) and to schedule needed staff members for these
sessions. Although some flexibility to tailor the duration of events does
exist, adherence to the complete process and a disciplined schedule is
critical to a verifiable evaluation. Schedule tailoring options include
combining interview sessions for organizationally equal peer groups, and
shortening the time allotted for the interview sessions.