Abstract:
This document represents a deliverable of the Software Engineering Institute (SEI) to the Director Defense Research and Engineering (DDR&E) and the Defense Advanced Research Projects Agency (DARPA) Software Action Plan Working Group (SWAP-WG). Work was performed under the task terms of the SEI - DDR&E/DARPA Technical Objectives and Plans (TO&P) number 2-151 dated June 17, 1992, as amended. The document includes a description of a model and methodology for applying a software risk assessment in an evaluative environment. This Software Risk Evaluation (SRE) methodology offers a validated, user-compatible approach to providing insight into the areas of potential technical risk inherent in software intensive acquisitions. This insight into software technical risk supports the acquisition decision-maker's ability to draw conclusions concerning the target software development effort's potential for success. The SRE methodology employs the SEI Risk Program's taxonomy-based questionnaire for identifying the potential software technical risks, and also provides supporting methodologies for transforming this data into actionable findings and conclusions.
Introduction
Because of the size and complexity of today's software and systems, the software-producing community (technical and management) needs a systematic approach for managing the technical uncertainty inherent in software development. Programs continue to be surprised by unanticipated technical problems in the production of software, which usually manifest themselves in cost overruns, schedule delays, and poor product quality. However, these undesirable impacts are only symptoms of the underlying root causes. For example, changing requirements, ad hoc development processes, new technology adoption, and unprecedented engineering applications all introduce uncertainty and, therefore, program risk to developing software systems. To be successful, programs must anticipate risks early in development before they become problems, plan and execute mitigation strategies to lessen risk impact, and effectively cope with those problems that do materialize.
In the summer of 1991, the Acting DDR&E chartered the SWAP and established the SWAP-WG to identify software-related issues that could provide high leverage through timely application of resources. One of the issues that met these criteria was transitioning a user-friendly independent review methodology for identifying technical risks in software intensive acquisitions. Upon examining the independent review issue, the SWAP-WG recognized the opportunity to meld its efforts with the ongoing SEI Risk Program work. This leverage opportunity led to this document.
Independent reviews are effective for discovering risks in programs. However, "expert" reviews, as currently applied, typically rely on the individual expertise assembled for a given review. These reviews tend to be ad hoc, and lack the benefits of a disciplined process. Such discipline is critical for passing on lessons learned and for making improvements to the independent review process. A goal of this work is to replace the current expert review system with independent risk assessment methods and processes that 1) systematically identify and analyze software technical uncertainty such that quantitative status information may be presented to decision makers in a timely fashion and 2) gain common acceptance across all DoD components at all levels - from the program manager through the oversight staff to the acquisition decision makers.
The purpose of this document is to define the individual components of an SRE methodology designed to meet the requirement for objective, consistent independent reviews. Embodied in this document are the initially developed processes and techniques that have proven effective during field trials conducted by the SEI Risk Program. The methodology described is the first in a series of evolutionary steps designed to result in the creation and refinement of a predictive decision model to support risk management in software intensive acquisitions.
