C3.6.1. OT&E shall determine the
operational effectiveness and suitability of a system under realistic
operational conditions, including combat; determine if the thresholds and
objectives in the approved ORD and the COIs have been satisfied; and assess
impacts to combat operations. The following procedures shall apply:
C188.8.131.52. The DoD Component
OTA shall be responsible for OT&E.
C184.108.40.206. OT&E shall use threat or
threat representative forces, targets, and threat countermeasures,
validated by DIA or the DoD Component intelligence agency, as appropriate,
and approved by DOT&E2 . DOT&E shall oversee threat
target, threat simulator, and threat simulation acquisitions and
validation to meet developmental, operational, and live fire test and
C220.127.116.11. Information assurance testing
shall be conducted on information systems to ensure that planned
and implemented security measures satisfy ORD and System Security
Authorization Agreement (SSAA) requirements when the system is installed and operated
in its intended environment. The PM, OT&E test authority, and designated
approving authority shall coordinate and determine the level of risk
associated with operating the system and the extent of security testing
required. (See section C6.6.
) Any requirements to reconstitute or recover information
system capabilities damaged by information assurance threat agents should also
be tested during OT&E.
C18.104.22.168. Typical users shall operate
and maintain the system or item under conditions simulating combat stress
and peacetime conditions.
C22.214.171.124. The independent
OTAs shall use production or production representative articles for
the dedicated phase of OT&E that supports the full-rate production decision (or for
ACAT IA or other acquisition programs, the deployment decision).
C126.96.36.199. Test planning shall consider M&S. OT&E should leverage M&S used during DT&E to
improve its credibility and reduce M&S development time and costs. Whenever
possible, an OA shall draw upon test results with the actual system, or
subsystem, or key components thereof, or with operationally meaningful
surrogates. When actual testing is not possible to support an OA, such
assessments may utilize computer modeling and/or hardware in the loop,
simulations (preferably with real operators in the loop), or an analysis
of information contained in key program documents, consistent with section
, above. The TEMP shall explain the extent of M&S supporting OT&E.
(See subparagraph C188.8.131.52. above.)
C184.108.40.206. The OTA shall
test and evaluate all hardware and software alterations that
materially change system performance (operational effectiveness and
suitability). This includes system upgrades and changes to correct
deficiencies identified during T&E.
C220.127.116.11. Naval vessels, the major
systems integral to ship construction, and military satellite programs
typically have development and construction phases that extend over long
periods of time and involve small procurement quantities. To facilitate
evaluations and assessments of system performance (operational
effectiveness and suitability), the PM shall ensure the independent OTA is
involved in the monitoring of or participating in all relevant testing to
make use of any/all relevant results to complete OAs. The OTA shall
determine the inclusion/exclusion of test data for use during OAs and
shall determine the requirement for any additional operational testing
needed for effectiveness and suitability.
C18.104.22.168. OTAs shall conduct an
independent, dedicated phase of OT&E before full-rate production to
evaluate operational effectiveness and suitability as required by 10
U.S.C. 2399 (reference (bd)
) for ACAT I and II programs.
OTAs shall participate in early DT&E and M&S to provide operational insights to the PM,
requirements developers, and acquisition decision makers.
For systems with joint interoperability requirements,
all available interoperability assessments (e.g., OAs, JITC
interoperability assessments, standards conformance reports) should be reviewed during the OTRR before
conducting IOT&E. Potentially critical interoperability problems shall be
highlighted for assessment during OT&E.
C22.214.171.124. OT&E shall evaluate
potentially adverse electromagnetic environmental effects (E3) and
spectrum supportability situations. Operational testers shall use all
available data and shall review DD Form 1494, .Application for Equipment
Frequency Allocation,. to determine which systems need field assessments.
weapon, Command, Control, Communications, Computers, Intelligence,
Surveillance, and Reconnaissance (C4ISR), and information programs that are
dependent on external information sources, or that provide information to
other DoD systems, shall be assessed for information assurance. The
level of information assurance testing depends on the system risk
and importance. Systems with the highest importance and risk shall be
subject to penetration-type testing prior to the beyond LRIP decision.
Systems with minimal risk and importance shall be subject to normal
National Security Agency security and developmental testing, but shall not be
subject to field penetration testing during OT&E.
C126.96.36.199. OT&E shall take maximum
advantage of training and exercise activities to increase the realism and
scope of OT and reduce testing costs.
C188.8.131.52. DOT&E shall determine
the quantity of articles procured for OT&E for MDAPs; the cognizant
OTA shall make this decision for non-MDAPs (10 U.S.C. 2399 (reference (bd)
C184.108.40.206. The operational
effectiveness of MDAPs for large-scale training systems shall be
determined based on their demonstrated training effectiveness.
C220.127.116.11. Each DoD
Component shall provide weapons effectiveness data for weapons in the
acquisition process to DOT&E for use in the Joint Munitions Effectiveness Manuals. The DoD
Component shall provide the data prior to the weapon achieving initial
operational capability, and shall prepare the data in coordination with
the Joint Technical Coordinating Group for Munitions Effectiveness.
C18.104.22.168. DOT&E shall assess the adequacy of OT&E and LFT&E, and evaluate
the operational effectiveness, suitability, and survivability, as applicable,
of systems under DOT&E oversight.
C3.6.2. OT&E Plans
C22.214.171.124. The DoD Components shall brief DOT and E on concepts for an OT&E or OA 120
days prior to start. They shall submit the T&E plan 60 days prior, and shall
report major revisions as they occur. Test plans shall include test
objectives; MOEs, MOPs, and measures of operational suitability;
planned operational scenarios; threat representations; targets; resources;
test limitations; and methods of data gathering and certification, reduction,
and analysis. The detail of the planned test events shall permit DOT&E to assess
C126.96.36.199. DOT&E shall approve, in writing, the adequacy of OT&E plans (including
project funding) for all ACAT I programs, selected ACAT IAM programs, and
other programs under DOT&E oversight (identified on the "Designation of Programs
for OSD Test and Evaluation (T&E) Oversight" memorandum), prior to starting OT&E.
(See section C3.3.
) DOT&E shall approve plans for all OAs in OSD T&E-oversight
programs, prior to execution. This approval requirement shall apply to
major revisions, as well.
C188.8.131.52. DOT&E-oversight programs beyond LRIP, shall
require continued DOT&E test plan approval, monitoring, and Follow-On Operational
Test and Evaluation (FOT&E) reporting to complete IOT&E activity; to refine IOT&E estimates;
to verify correction of deficiencies; to evaluate significant changes to
system design or employment; and to evaluate whether or not the system
continues to meet operational needs and retain operational effectiveness
in a substantially new environment, as appropriate.
C3.6.3. Use of Contractors in
Support of OT&E
C3.6.4. OT&E Information Promulgation