In general, testing related to maintainability can be
grouped into five basic areas: functional, performance, verification,
demonstration, and evaluation. Functional testing is performed to verify that a product, or product function, is behaving as intended. Such testing typically involves applying a known stimulus or set of stimulus to the test item and comparing the item response to a known response or set of responses.
Performance testing goes beyond functional testing to verify that the level of performance of the product functions meet the requirements. It is not sufficient, for example, to verify that for a given input signal, the product provides the right kind of output signal; the characteristics of the signal (amplitude, noise level, and so forth) and the reliability, maintainability, safety, and so forth of the product must meet the requirements. Special types of performance testing are design limit tests, life tests, software tests, electromagnetic interference (EMI) tests, and reliability growth tests.
Functional testing and performance testing are
performed throughout various phases of product development and may include the
use of models, simulations, testbeds, and prototypes or Full Scale Development
models of the product. While such testing is almost always performed as part
of the product design and development process, testing of the maintainability
features of the product design, such as diagnostics, must also be planned for
in a similar fashion. Diagnostic and other maintainability performance testing must be an integral part of all testing. This is important to evaluate performance, uncover deficiencies and implement corrective action while the product is still in development.
Verification testing is performed on a continuous basis throughout product development to determine the accuracy of and update the analytical data obtained from engineering analysis. Verification is typically performed prior to any planned demonstration or evaluation test to provide assurances that the maintainability of the product can be achieved and demonstrated. Note that all kinds of test data collected, such as from a functional test of the diagnostics, should be used for verification of maintainability analyses and requirements.
Demonstration testing is usually a formal process
conducted by the product developer and end customer to determine whether
specific maintainability requirements that have been specified have been
achieved. Such testing will involve development of a formal test plan, using
defined methods of analysis to determine compliance. Details of demonstration
test plans and procedures, as well as verification and evaluation test data
can be found in Appendix
, Test Methods.
Evaluation testing is the process to determine, at all levels of maintenance and product design, the impact of the operational and maintenance and support environments on the maintainability parameters of the product. Such testing should involve performance of defined maintenance tasks in the product's actual use environments. Maintainability evaluation testing, as with other forms of testing, should be integrated with testing designed to evaluate other product parameters. Evaluation testing is one area that stands to benefit from virtual reality capabilities. Such capabilities would allow the testing of some maintenance tasks (such as manual ones) in a simulated usage environment, rather that the actual one. Some obvious cost savings are possible
with this approach.