|
Original Date: 04/29/2002
Revision Date: 01/18/2007
Best Practice : Measures of Effectiveness
Since 1991, the Maintenance and Logistics Command-Atlantic has maintained Measures of Effectiveness by collecting and analyzing data in eight categories: Acquisition Process, Financial Management, Product/Service Timeliness, Naval Engineering Support Unit, Availability Readiness, Inventory Accuracy, Customer Service, and Casualty Response.
The Maintenance and Logistics Command-Atlantic (MLCLANT) uses Measures of Effectiveness (MOEs) to help focus on important issues and drive improvement within their key success factors. MOEs are in various formats and part of MLCLANT’s Total Quality Management initiative. These measures are grouped into eight categories: Acquisition Process, Financial Management, Product/Service Timeliness, Naval Engineering Support Unit (NESU), Availability Readiness, Inventory Accuracy, Customer Service, and Casualty Response. MOEs were developed as a reporting system to measure critical processes within the Naval Engineering Program. The MOEs evolved out of MLCLANT’s desire to standardize its measurement activities. MOEs include the following areas: Acquisition, Process, Discipline, and Future Goals; Results; Cutter Reliability; and
Configuration Compliance.
MLCLANT uses a format which requires that the measures be Specific, Measurable, Agreed Upon, Realistic, Time- framed (SMART). The Acquisition Process measures four success rates:
Contract Award and Start Date
MLCLANT’s success rate of 30 days between contract award and start date is an important goal. In FY99 and FY00, MLCLANT realized a dramatic improvement; however, FY01 brought a decline in this measure. This was caused by several factors including the initiation of a new “best-value” contracting method, which complicated the bid evaluation. This measure now has a positive trend for FY02. Overall, 56% of availabilities were awarded at least 30 days prior to the start date.
Containing Contract Growth
Since FY97, MLCLANT has consistently contained contract growth to less than 20% for more than 80% of the contracts (Figure 2-3). In FY01, 94% of the contracts had less than 20% contract growth.
Meeting Contract Start Dates
Since FY97, MLCLANT has demonstrated a high level of success in starting availabilities on time and had an overall success rate of 96% in FY01.
Meeting Contract Completion Dates
Since FY96, completion rates have been on a steady rise (Figure 2-4), with an on-time success rate in FY01 of 80% for completing availability contracts.
MLCLANT’s product timelines are controlled by an availabilities milestone objective established through the Fleet Logistic System (FLS) workflow. Information is reviewed quarterly and feedback is provided on the following timelines: sending the availability project start message; and timeliness in receiving work list (WL) submission, completing draft package, sending final specification to type desk managers, and bid openings. An Availability Start Day Message informs a unit of an availability project commencement and the start of the planning process. Prior to FY99, MLCLANT’s success rate in this measure was high. In FY01, the overall success rate for sending out the Availability Start Day Message was 59%. After analysis, MLCLANT determined that the problematic implementation of FLS caused the drop.
The measure, timeliness in completion of draft package, compares planned vs. actual dates on appropriate project schedules. The draft package describes the repairs needed and the specifications used at the second Acquisition Team (A-Team) meeting review. The goal is to ensure that the draft is completed in accordance with the established timelines. In FY01, the on-time success rate for the completion of draft specification packages was 72%, and 90% for completion of draft specifications within five days of the planned date. This represents a moderate improvement over prior years.
Other measures include:
Final Specifications timeliness in sending final specifications to the Support Branch with established timelines. Since FY98, MLCLANT has seen consistent increases in this measure. For FY01, the success rate was 89%, and 98% within five days.
Timeliness in Bid Openings ensures the bid openings are completed in accordance with established timelines. In FY01, the overall success rate for opening bids on time was 74%, and 77% within five days.
Customer Satisfaction for the MLCLANT and Contractor derived from the number of positive and negative responses from the MOEs’ status report submitted by the procurement section. The goals are to evaluate and compare the level of customer satisfaction with the availability process, and measure, analyze, and maintain a high level of customer satisfaction. Since FY97, MLCLANT has exceeded a 95% customer satisfaction rate.
Naval Engineering Post Availability Scores measures the level of customer satisfaction with the Naval Engineering Division’s contribution to the availability process. Twenty-eight questions are asked, and the score is derived from the average numeric scores given for the Naval Engineering Division from the Post Availability Survey. Here, the goal is to measure, analyze, and improve the level of customer satisfaction. The Naval Engineering Division has provided its customers with adequate service for the last four years. In FY01, the customer satisfaction average was 3.89 out of a possible five. In the third and fourth quarters FY01, MLCLANT maintained an average score of four.
By expanding and improving its measures, MLCLANT can identify trends and process problems, support and track improvement efforts, benchmark and conduct risk assessments, demonstrate results, and focus on significant issues.
Figure 2-3. Monitoring Contract Growth
Figure 2-4. Planned Completion Dates
For more information see the
Point of Contact for this survey.
|
