A scorecard approach, which would be useful in assessing the
status of cross-service C4I interoperability, is recommended. This approach
would support problem prioritization, diagnosis, and correction, as well as
operationally based assessment of the state of C4I interoperability for the
use of system managers (e.g., the Office of the Assistant Secretary of Defense
for C3I and the Military Communications and Electronics Board) and operational
users (e.g., CINCs or joint task force commanders). The scorecard approach
cannot provide sophistication and quantitative theoretically grounded
measures. Rather, the fundamental motivator is to move to a point that
interoperability is analyzed, assessed, and driven top-down by considerations
of operational significance.
Recommendation I-3.1: The Assistant Secretary of
Defense for C3I and the Joint Chiefs of Staff should develop a set of
"interoperability scorecards" as a basis for management, covering the spectrum
from compliance with standards to successful end-to-end mission
The approach uses three scorecards--technical, systems, and operational--corresponding to the elements of the architectural triad. The technical compliance scorecard would be used to assess a system's implementation from a technical interoperability perspective. It would take the form of a system profile or list for scoring each system implementation with respect to compliance or non-compliance with relevant standards and guidance. The systems interoperability scorecard would measure system-to-system interoperability and would take the form of a matrix displaying the ability of all pairs of systems to interact with each other. The operational interoperability scorecard would be used to assess the ability of a set of systems to satisfy specific node-to-node information flow requirements as well as the collective set of information flows needed to satisfy a defined mission or mission slice.
Assessment requires that responsibility is assigned for (a) the development and definition of criteria, (b) actual conduct of the assessment, as well as (c) responsibility for ensuring that the results of the assessments translate into actions to remedy issues uncovered in the assessment. Responsibility should be assigned as follows:
a. Development and definition of criteria. For assessment of technical and systems interoperability the committee believes that the Assistant Secretary of Defense for C3I should have the lead in development of criteria, and definition of operational interoperability criteria should be a joint responsibility of the Joint Chiefs of Staff and the combat commands.
b. Conduct of interoperability assessments. Given the Defense Information Systems Agency's Joint
Interoperability Test Center's role in testing compliance with the Joint
Technical Architecture, it is appropriate that these organizations conduct the
technical interoperability assessment. System developers and the organization
proposed in Recommendation I-2 above are the logical responsible organizations to assess system interoperability. The operational interoperability assessments should be conducted by an appropriate organization as tasked by the Joint Chiefs of Staff (e.g., the Joint Theater Air Missile Defense Organization for theater missile defense). The responsible organizations should ensure that users play a key role in making such assessments. Without a process for continual assessment for interoperability, the user will have little sense for what interoperability problems need fixing and what their impact might be.
c. Accountability for the results of the assessments. Accountability and responsibility for remedying shortcomings uncovered should lie with the Assistant Secretary of Defense for C3I in the case of the technical and system assessments and with the Joint Chiefs of Staff and the operational commands in the case of operational interoperability assessments.