D.4 Criteria definitions. The following paragraphs
provide definitions for the criteria in Figure 6 that may not be
self-explanatory. The criteria are listed in alphabetical order, matching as
closely as possible the wording used in Figure 6.
D.4.1 Accurately describes (an item). This criterion, applied to user/operator/programmer instructions and to the "as built" design and version descriptions, means that the instructions or descriptions are correct depictions of the software or other item described.
D.4.2 Adequate test cases, procedures, data, results. Test cases are adequate if they cover all applicable requirements or design decisions and specify the inputs to be used, the expected results, and the criteria to be used for evaluating those results. Test procedures are adequate if they specify the steps to be followed in carrying out each test case. Test data are adequate if they enable the execution of the planned test cases and test procedures. Test or dry run results are adequate if they describe the results of all test cases and show that all criteria have been met, possibly after revision and retesting.
D.4.3 Consistent with indicated product(s). This criterion means that: (1) no statement or representation in one software product contradicts a statement or representation in the other software products, (2) a given term, acronym, or abbreviation means the same thing in all of the software products, and (3) a given item or concept is referred to by the same name or description in all of the software products.
D.4.4 Contains all applicable information in (a specified DID). This criterion uses the DIDs to specify the required content of software products, regardless of whether a deliverable document has been ordered. Allowances are to be made for the applicability of each DID topic. The formatting specified in the DID (required paragraphing and numbering) are not relevant to this evaluation.
D.4.5 Covers (a given set of items). A software product "covers" a given set of items if every item in the set has been dealt with in the software product. For example, a plan covers the SOW if every provision in the SOW is dealt with in the plan; a design covers a set of requirements if every requirement has been dealt with in the design; a test plan covers a set of requirements if every requirement is the subject of one or more tests. "Covers" corresponds to the downward traceability (for example, from requirements to design) in the requirement, design, and test planning/description DIDs.
D.4.6 Feasible. This criterion means that, in the knowledge and experience of the evaluator, a given concept, set of requirements, design, test, etc. violates no known principles or lessons learned that would render it impossible to carry out.
D.4.7 Follows software development plan. This criterion means that the software product shows evidence of having been developed in accordance with the approach described in the software development plan. Examples include following design and coding standards described in the plan. For the software development plan itself, this criterion applies to updates to the initial plan.
D.4.8 Internally consistent. This criterion means that: (1) no two statements or representations in a software product contradict one another, (2) a given term, acronym, or abbreviation means the same thing throughout the software product, and (3) a given item or concept is referred to by the same name or description throughout the software product.
D.4.9 Meets CDRL, if applicable. This criterion applies if the software product being evaluated is specified in the CDRL and has been formatted for delivery at the time of evaluation. It focuses on the format, markings, and other provisions specified in the CDRL, rather than on content, covered by other criteria.
D.4.10 Meets SOW, if applicable. This criterion means that the software product fulfills any Statement of Work provisions regarding it. For example, the Statement of Work may place constraints on the operational concept or the design.
D.4.11 Presents a sound approach. This criterion means that, based on the knowledge and experience of the evaluator, a given plan represents a reasonable way to carry out the required activities.
D.4.12 Shows evidence that (an item under test) meets its requirements. This criterion means that recorded test results show that the item under test either passed all tests the first time or was revised and retested until the tests were passed.
D.4.13 Testable. A requirement or set of requirements is considered to be testable if an objective and feasible test can be designed to determine whether each requirement has been met.
D.4.14 Understandable. This criterion means "understandable by the intended audience." For example, software products intended for programmer-to-programmer communication need not be understandable by non-programmers. A product that correctly identifies its audience (based on information in Block 3 of the corresponding DID) and is considered understandable to that audience meets this criterion.