Attachment 6 - TSE RMA Metrics Terms and Definitions.pdf

Transportation Security Equipment (TSE)

Metrics Terms and Definitions

Reliability, Maintainability & Availability (RMA) Metrics

March 8, 2013

Table of Contents

1. PURPOSE

2. SCOPE

3. CORRECTIVE MAINTENANCE

4. MAINTENANCE ACTIVITY COLLATION PROCESS

4.1 MAINTENANCE ELEMENTS

4.2 TSE FAILURE CRITICALITY

4.3 TSE FAILURE RELEVANCY

4.4 MAINTENANCE ACTION TIME POINTS

4.5 MAINTENANCE ACTION LOW-LEVEL METRICS

4.6 MAINTENANCE ACTION REPAIR CYCLES

4.7 LOW LEVEL METRICS COUNTING

5. OPERATIONAL RMA METRICS

5.1 TSE RELIABILITY METRICS

5.1.1 Mean Time Between Critical Failures (MTBCF)

5.1.2 Mean Uptime (MUT)

5.2 TSE MAINTAINABILITY METRICS

5.2.1 Mean Downtime (MDT)

5.2.2 Mean Time to Repair (MTTR)

5.3 TSE AVAILABILITY METRICS

5.3.1 Operational Availability (Ao)

6. DEPOT TURN AROUND TIME METRIC

1. Purpose

This document has two purposes: 1) To define the process Transportation Security Administration (TSA) will use to identify, collect, record, and report each corrective maintenance activity conducted on Transportation Security Equipment (TSE), and 2) To establish the process for developing the Reliability, Maintainability, & Availability (RMA) metrics that TSA will use to evaluate the performance of TSE during its operational life cycle.

An effective RMA metrics program for the operational TSE will provide TSA management with technical insight into the quality of the development and fielding of TSE.

2. Scope

This document delineates the maintenance elements that TSA uses to document the corrective maintenance activities conducted on TSE, to include time points and low-level metrics from those time points to document each corrective maintenance action; and identifies the requirements for recording these maintenance elements in the TSE Database. It also provides the process TSA uses to develop RMA metrics from the low-level metrics for evaluating and analyzing the operational performance of deployed TSE throughout their life cycle. While not specifically discussed in this document, the same maintenance elements defined in this document support the process TSA uses to develop metrics to evaluate the performance of Contractors as identified in TSA contracts. These Contractor performance metrics will be delineated in a separate contract document.

TSA uses two types of maintenance activities to support TSE: (1) Scheduled preventive maintenance to improve the reliability of TSE, and (2) Unscheduled corrective maintenance to repair or restore TSE to full mission capability. While the contract will dictate the requirements for both, this document focuses only on unscheduled corrective maintenance actions and the accurate recoding of them in the TSE Database.

3. Corrective Maintenance

For TSA, corrective maintenance can be either performed on site or at a depot facility. For this purpose, TSA considers three discrete levels of corrective maintenance as defined below.

• Level I Corrective Maintenance – This is corrective maintenance that is performed as needed to effect minor repairs to TSE that do not require Field Service Technicians (FST) (e.g., some bag jams, reboots, etc.). These activities are normally performed by TSA personnel or their designees and are not addressed under this document.

• Level II Corrective Maintenance – This is unscheduled corrective maintenance activities performed as needed to effect repairs that always require FSTs. These activities are performed on site with the TSE in Level II CM status. Level II Corrective Maintenance will be referred to as CM in this document.

• Level III Depot Maintenance – This consist of unscheduled corrective maintenance required by shipping the TSE to a depot facility for repair. Level III Depot Maintenance will be referred to as DM in this document.

4. Maintenance Activity Collation Process

Each TSE corrective maintenance activity must progress through a repair cycle from the initial notification of a failure to return of the TSE to operational status. The process identified in this document relies on the maintenance elements identified, collected and measured during each repair cycle conducted to return the TSE to full mission capability, the proper recoding of those maintenance action elements in the TSE Database, and the proper reporting of those maintenance elements for performance metrics development.

4.1 Maintenance Elements

Each corrective maintenance activity begins with the identification of a TSE maintenance requirement. It is important to identify the following maintenance elements in each repair cycle to properly document the corrective maintenance actions:

• TSE failure criticality

• TSE failure relevancy

• Maintenance action time points along the repair cycle

• Maintenance action low-level metrics derived from the maintenance action time points

All maintenance elements identified in this document, as well as other required maintenance activity information identified in contract Contract Data Requirements Lists (CDRL) and associated Data Item Descriptions (DID), must be collected and recorded for each corrective maintenance activity in the TSE Database. Other type information would include, but not limited to, the failure mode indicators and parts replacement information as explained in contracts.

4.2 TSE Failure Criticality

The criticality of a maintenance activity must be validated by the Contractor as soon as possible to ensure the prompt return of TSE to full mission capability. The critically of each TSE corrective maintenance activity must be documented in the TSE Database. The definitions below are used to determine whether a TSE failure is identified as a critical failure or a non-critical failure.

• Critical - Failures where TSE is unable to perform its required mission and requires immediate corrective maintenance to restore the TSE to full operational condition. This normally results in downtime from the time the failure occurs until the repair is complete.

Events that TSA Security Officers (TSO) can correct via system resets, simple removal of obstructions, etc., are not categorized as critical failures during the TSA Operations & Maintenance (O&M) phase.

• Non-Critical - Failures where TSE needs corrective maintenance but still operates effectively (can perform its mission) until the minor repair is completed.

CM and DM maintenance activities can be either critical or non-critical depending on when the TSE experiences downtime. Criticality of a CM maintenance activity affects how other maintenance elements must be measured and documented in the TSE Database.

4.3 TSE Failure Relevancy

For each TSE failure the relevancy of the failure must also be determined and documented in the TSE Database. The definitions below are used to determine whether a TSE failure and its associated repair cycle is recorded as a relevant failure or a non-relevant failure.

• Relevant - Failures attributable to TSE’s inherent performance reliability. In this case, the inherent reliability of the TSE is directly related to the failure.

• Non-Relevant - Failures verified to be caused by an unintentional or intentional external occurrence in the operational environment not related to the inherent performance reliability of TSE that causes the TSE to fail as either a critical or non-critical failure.

This could include such things as baggage processing irregularities or operator mishaps that physically damage the TSE. In this case, the inherent reliability of the TSE is unrelated to the failure.

CM and DM maintenance activities can be either relevant or non-relevant depending on the cause of the failure. Relevancy of a corrective maintenance activity also affects how other maintenance elements must be measured and documented in the TSE Database.

4.4 Maintenance Action Time Points

TSA relies on the collection of specific maintenance action time points (T-Values) along a corrective maintenance repair cycle to track its progress. Table 1 identifies and explains the 11 T-Values that must be recorded, as required, for each repair cycle to document CM and DM maintenance actions. These T-Values must identify the date and time of each maintenance action and be provided in real clock times in the TSE location’s local time based on a 24-hour day, seven-day week.

Table 1 - Measurement Time Points (T-Values)

ID Description

T1 Call Center Contacted: Time when the Dispatch Contact Facility is initially contacted after a TSE failure occurs.

T2 Technician Dispatched: Time when a FST is contacted to perform the repair.

ID Description

T3 Technician Starts Diagnostics: Time when the FST begins to diagnose the TSE problem.

T4 Technician Diagnostics Complete: Time when the FST completes the diagnosis and is ready to request any part(s) from the supply chain or start the repair if parts are not required.

T5 Parts Begin Shipment: Time when the supply chain indicates that the part(s) are en-route to the FST.

T6

Parts Delivered to Location: Time when the part(s) are delivered locally. Note that this is not necessarily when the part(s) are in the hands of the FST but rather when they are delivered from the supply chain locally; there may be additional time to get the part(s) to the FST to begin the repair.

T7 Technician Begins Repair: Time when the FST is at the TSE location and ready to begin the repair (has all necessary parts and tools).

T8 Technician Begins Testing: Time when the repair has been completed and the FST is ready to verify completion of the repair.

T9 TSE Ready for Signoff: Time when the FST has successfully completed the verification testing and the TSE is ready for the TSA signoff.

T10 TSA Signoff: Time when TSA signs off on the TSE repair and the TSE is ready to be returned to service.

T11 TSE Ready for Setup at the Location: Time when the TSE has returned from the depot or storage or completed depot repair at the site (TSE was in DM status).

4.5 Maintenance Action Low-Level Metrics

The T-Values above provide the start and end times that are used to derive low-level metrics (M- Values) that measure the duration of discrete maintenance actions along the repair cycle. The M- Values are derived as the difference between two specific T-Values, resulting in M-Values that must be recorded as applicable for a repair cycle. Table 2 shows the 12 M-Values that would be derived from the above T-Values in a straight linear progression. An explanation of each M- Value development process is identified below Table 2. Note in Table 2 that some T-Values and M-Values can be applicable when a TSE is in either a CM or DM repair cycle. Section 4.6 explains known deviations in the development of M-Values when a repair cycle is not in a straight linear progression.

Table 2 - Low-Level Metrics Summary

Maintenance Path

Low-Level Metric Start Time End Time

ID Description ID Description ID Description

CM & DM M1

Initial Call Center Interaction Time T1 Call Center Contacted T2 Technician Dispatched

CM & DM M2

Technician Dispatch Time T2 Technician Dispatched T3 Technician Starts

Diagnostics

Maintenance Path

Low-Level Metric Start Time End Time

ID Description ID Description ID Description

CM & DM M3

Technician Diagnosis Time T3 Technician Starts

Diagnostics T4 Technician Diagnostics Complete

CM M4 Part(s) Request Time T4 Technician Diagnostics Complete T5 Parts Begin Shipment

CM M5

Part(s) Fulfillment Time T5 Parts Begin Shipment T6 Parts Delivered to

Location

CM M6 Local Logistics Time T6 Parts Delivered to Location T7 Technician Begins

Repair

CM M7 Technician Repair Time T7 Technician Begins Repair T8 Technician Begins

Testing

CM &DM M8

Technician Testing Time T8 Technician Begins

Testing T9 TSE Ready for Signoff

CM & DM M9 TSA Signoff Time T9 TSE Ready for Signoff T10 TSA Signoff

CM M10 Time to Repair T3 Technician Starts

Diagnostics T4 Technician Diagnostics Complete

T7 Technician Begins Repair T9 TSE Ready for Signoff

DM M11

Depot Turn Around Time (DTAT) T4 Technician Diagnostics

Complete T11 TSE Ready for Setup at the Location

DM M12

Uncrate/Setup at Location Time T11 TSE Ready for Setup at the Location T8 Technician Begins Testing

• Initial Call Center Interaction Time - M1. This is the metric that measures how long it takes the Dispatch Contact Facility to gather the initial information about the TSE problem(s) and assign a FST to conduct the repair. It is measured as the time between when the Dispatch Contact Facility is initially contacted after the failure occurs (T1) and the time the FST is contacted to perform the work (T2).

• Technician Dispatch Time - M2. This is the metric that measures how long it takes for the FST to respond to the TSE to diagnose and repair the TSE. It is measured as the time between when the FST is initially contacted by the Dispatch Contact Facility to perform the work (T2) and the time that the FST arrives at the failed TSE and begins the diagnosis of the problem (T3).

• Technician Diagnosis Time - M3. This is the metric that measures how long it takes the

FST to diagnose the problem(s) with the TSE. It is measured as the time between when the FST begins the diagnosis of the problem (T3) and when the FST completes the diagnosis and is ready to request any part(s) from the supply chain (T4) or is ready to begin repair if no parts are required (T7). See Section 4.6 for this explanation.

• Part(s) Request Time - M4. This is the metric that measures how long it takes to request the part(s) from the supply chain required for the repair of the TSE. It is measured as the time between when the FST completes the diagnosis and is ready to request any part(s) from the supply chain when parts are required (T4) until the time the supply chain identifies that the part(s) are en-route to the FST (T5).

• Part(s) Fulfillment Time - M5. This is the metric that measures how long it takes to deliver (or fulfill) from the supply chain the part(s) required for the repair of the TSE. It is measured as the time between when the supply chain identifies that the part(s) are en-route to the FST (T5) until the time the part(s) are delivered locally (T6). Note that this is not necessarily when the part(s) are in the hands of the FST but rather when they are delivered locally from the supply chain; there may be additional time to get the part(s) to the FST at the TSE location to start repairs that would be included in M6.

• Local Logistics Time - M6. This metric measures any delays incurred where parts have been delivered locally but additional time is required to get the part(s) into the hands of the FST to start the repair of the TSE. It is measured as the time between when part(s) are delivered locally (T6) until they are delivered to the FST at the TSE location and the FST is ready to begin the repair (T7).

• Technician Repair Time - M7. This metric measures the repair time for a TSE. This is defined as the time between when the FST has available all the necessary tools and parts and begins the disassembly of the TSE for repair to when the TSE is reassembled and is ready for verification testing of a successful repair. It is measured as the time between when the FST has all necessary part(s), if required, and tools at the TSE and is ready to begin the repair (T7) until the time the TSE is repaired and ready for verification tests (T8).

• Technician Testing Time - M8. This metric measures the verification testing time for the TSE. It is measured as the time between when the TSE is ready for verification tests (T8) until the time the verification testing is successfully completed and the TSE is ready for the TSA signoff process (T9). At this time (T9) the FST notifies the Dispatch Contact Facility that the TSE is ready to be put into service. If identified in the contract, the FST will also notify the airport’s Coordination Center by e-mail to an address designated by the airport Coordination Center that the repairs are complete and that the TSE is ready to be put into service.

• TSA Signoff Time - M9. This metric measures the TSA signoff time for the TSE and is the last step before the TSE is placed back into service. It is measured as the time between when the verification testing is successfully completed by the FST and the TSE is ready for the TSA signoff process (T9) until the TSE repair is signed off by TSA and is ready to be returned to service (T10).

• Time to Repair - M10. This metric measures the actual repair time for a TSE, including the time required for diagnostics and testing. It is measured as the sum of M3 (T3 to T4), M7 (T7 to T8), and M8 (T8 to T9), which ends when the TSE is reassembled and is ready for TSA signoff (T9). See Section 4.6 for deviation to development of M7.

• Depot Turn Around Time - M11. This is the metric that measures how long a TSE remains away from its designated location for repair at the depot or is put in depot repair status at the site to complete a repair. It is computed as the time between when the diagnosis has been made and the DM decision made (T4) until the time TSE is ready for setup at its designated location after the completion of depot repair (T11). These measurements will be by hours or portions thereof based on a 24 hour day, seven day week.

• Uncrate/Setup at Location Time - M12. This is the metric that measures how long it takes the TSE to be prepared for re-installation at the airport’s designated location after it has returned from the depot or storage, or has completed depot repair at the site. It is measured as the time between when the TSE has returned from the depot or storage or completed depot repair at the site (T11) until the time the TSE is ready for testing after re-installation at the installed location (T8).

4.6 Maintenance Action Repair Cycles

Each corrective maintenance activity involves a repair cycle that starts from initial notification until the repair is completed successfully. T-Values must be recorded and applicable low-level metrics (M-Values) derived from the T-values of each maintenance action to quantify the repair cycle and to ensure consistent and accurate performance metric development. Some maintenance time points and associated low-level metrics will depend on whether the repair is being accomplished as a CM or DM repair. Figure 1 shows an example of two paths (CM or DM) that TSE can progress through in a repair cycle in a straight linear progression. It identifies the maintenance time points (T-Values, i.e. T1, T2, etc.) and low level metrics (M-Values, i.e.

(M1, M2, etc.) along a repair cycle, and illustrates the type data points and low level metrics that will be included in the TSE Database.

Figure 1 - Repair Cycle Paths

M1 – Init CC Interaction

M11 – Depot Turnaround Time

M3 – Tech Diagnosis

M2-Tech Dispatch

M7 – Tech Repair

M6 – Local Logistics

M5 – Parts Fulfillment

M4 _ Parts Request

M9 – TSA

Signoff

M8 – Tech Test

T1 T2 T3 T4

T4 T5 T6 T7 T8

T8 T9 T10

T4 T8

Corrective Maintenance

(CM)

Depot Maintenance

(DM)

T11

M12 – Uncrate /

Setup

Table 4 shows the standard low-level metrics (M-Values) that will be measured from the T- Values for each type failure in a repair cycle linear progression. The dark shaded blocks with no

M-Values on the Table indicate that the associated low level metric should not be applicable to that type failure in its repair cycle.

Table 3 -Type Failures

Type Failure Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8 M9

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8 M9

CM Critical Non- Relevant M1 M2 M3 M4 M5 M6 M7

M8 M9

CM Non- Critical

Non- Relevant M1 M2 M3 M4 M5 M6 M7

M8 M9

DM Critical Relevant M1 M2 M3 M11 M12 M8 M9

DM Non- Critical Relevant M1 M2 M3

M11 M12 M8 M9

DM Critical Non- Relevant M1 M2 M3

M11 M12 M8 M9

DM Non- Critical

Non- Relevant M1 M2 M3

M11 M12 M8 M9

Normally, a repair is achieved by progressing through the T-Values and low-level metrics in a straight linear progression as shown in Table 3. However, below are known deviations from the normal linear progression that should be identified and measured accordingly.

No Parts Replacement: The repair cycle identified in Figure 1 assumes the need for spare parts to complete the repair. In a CM where parts are not required to complete the repair the T-Values associated with parts replacement would not be recorded, some low level metrics would not be derived, and other low level metrics would be ended and started with different T-Values.

Following is the progression of a repair cycles without a parts replacement requirement:

• T1 through T4 recorded, and M1, M2 and M3 derived.

• T5, T6 and T7 not recorded, and M4, M5 and M6 not derived.

• T8 and T9 recorded, M7 derived (from T4 to T8), and M8 and M9 derived normally.

Recursions: As a repair cycle progresses, there may be a need to repeat some T-Values and or a need to change the type of repair cycle. This is referred to as recursion in the TSA maintenance collection process and can occur in two forms as identified below. Each recursion event and the applicable maintenance time points and derived low-level metrics must be captured in the TSE Database for a corrective maintenance activity.

• The first and most common form of recursion occurs when events result in the repetition of maintenance actions in a repair cycle and consequently the repetition of some T-

Values and derived M-Values. An example would be the delivery of defective or incorrect parts, which would cause the repeat of M4 and M5, or failure of FST Testing (M8), which would require the repeat of some portion of the repair cycle. An example of a repair cycle that includes this type of recursion is represented by the black line in Table 4 on the CM(1) row, with the repair timeline path going back and forth, starting at M1 and eventually ending at M9. In this case the repair cycle would stay the same but repeat the necessary T-Values and derived M-Values and document them in the TSE Database.

• The second and less common form of recursion occurs when a repair starts as CM

(repaired in place) and is switched to DM because of the complexity of the needed repair.

This is represented by the red line in Table 4 with the repair timeline path starting on the CM(2) row in M1 and eventually ending in the DM row at M9 because the repair required depot maintenance. In this case the maintenance cycle would be changed to DM.

Table 4- Timelines Recursion

Type Repair Activity Low-Level Metrics

CM (1) M1 M2 M3 M4 M5 M6 M7 M8 M9

CM(2) M1 M2 M3 M4 M5 M6 M7 M8 M9

DM M1 M2 M3 M11 M12 M8 M9

Time to Repair (M10) Development: As shown in Table 2, in a straight linear progression M10 consist of the sum of (from T3 to T4) and (from T7 to T9). As noted above, if parts are not required there is no T7. Therefore, when parts are not required in a repair cycle the maintenance repair action will be the sum of (from T3 to T4) and (from T4 to T9).

It is very importance that all applicable T-Values are recorded and M-Values derived for the different type repair cycles and their deviations. These T-Values and M-Values must also be accurately captured in the TSE Database for each corrective maintenance activity in accordance with the TSE CDRL and DID. Therefore, the Contractor should identify any other deviations that would preclude the documentation of T-Values and M-Values as identified above, and should make sure the TSE Database clearly identifies the T-Values used to derive an M-Value.

As previously mentioned, it is also important that these T-Values and derived M-Values identify the actual date and time of each maintenance action and be provided in real clock times in the TSE location’s local time based on a 24-hour day, seven-day week.

4.7 Low Level Metrics Counting

TSA uses the low level metrics to develop all performance metrics (high level metrics) required to evaluate the performance of TSE and to evaluate Contractor maintenance support under TSA contracts. This requires consistency in the counting of the low-level metrics in the development of each type high level metric. While each high level metric should specify which low level metrics to count, this Section describes the normal process TSA will use to develop the time and failure counts from a repair cycle. Any deviations to this process will be identified in the specific high level metric calculations or the TSA contract. Others noted by the Contractor should be identified to TSA to ensure a resolution and an accurate time and count.

For high level metric development, TSA only considers a system’s downtime (when TSE is not in a condition to perform its required mission) during an airport’s operational hours, which will be provided in the TSA contracts. Therefore, the counting of low level metric times will be based on the TSE operating time and TSE operational downtime as defined below, unless specifically identified in the particular metric.

• TSE Operating Time – The period of time a system is available to perform its required mission during the operational hours of a facility. During the Operations & Maintenance (O&M) life cycle phase, TSE operating time will be equal to the airport’s operational hours (period of time within 24-hour day that an airport is operational) unless the TSE experiences any operational downtime.

• TSE Operational Downtime – The period of time during an airport’s operational hours that TSE is not in a condition to perform its required mission. This includes all times that a system is not available due to corrective maintenance during the airport’s operational hours. The determination of the operational downtime is made after all the low-level metrics are recorded against the 24-hour day and measured against the airport’s operational hours.

Based on the above determination, low-level metrics times that accrue TSE downtime and occur within an airport’s operational hours will be counted in an operational downtime calculation;

however, TSE downtime that occurs during an airport’s non-operational hours would not be counted in an operational downtime calculation. Figure 2 shows an example of a repair cycle that has operational downtime recorded from T1 for a critical CM maintenance activity that starts during an airport’s operational hours, does not require spare parts for the repair, and completes the repair after the airport’s operational hours. The M-Value times will be counted as follows:

• Times associated with M1, M2, M3, and M7, or portion thereof, were conducted during the airport’s operational hours, and will be counted as operational downtime.

• Time associated with M8 will not be included in the calculation of operational downtime for that maintenance activity since it was performed after the airport’s operational hours.

• Although M9 must be derived and recorded, M9 times will not be included in any high level metric calculations since the TSE should be operational at T9 awaiting TSA acceptance.

• This corrective maintenance activity would generate one failure or occurrence for the metric calculations if there is operational downtime >0 for the failure.

If a repair cycle and M-Values were to continue into an airport’s next operational hours within the same reporting period, that operational downtime would also be counted in the same operational downtime calculations. (Note that M7 in this example is from T4 to T8 since there are no parts replacement, see Section 4.6 for the explanation).

Figure 2 - Operational Downtime Determination

All high level metrics will be calculated for the period and quantity of TSE specified by TSA, should include maintenance actions conducted during that reporting period, and should only include installed operational units in the calculations. The normal TSA reporting period is monthly through the last day of each calendar month unless identified otherwise in the TSA contract, and can be for an individual TSE unit, or grouped in accordance with the contract requirements.

Unless a specific high level metric identifies otherwise, only the portion of the repair cycle time completed during a reporting period will be counted in the metric calculations for a reporting period. Figure 3 shows an example of a critical maintenance repair cycle that continued into another reporting period, and into the next day’s airport operational hours. The M-Value times will be counted as follows:

• Times associated with (M1, M2, and M3) >0 or portion thereof, were conducted during the airport’s operational hours and will be counted as operational downtime for a high level metric calculation in reporting period 1. It will also be counted as one failure for reporting period 1 since operational downtime is > 0.

• Times associated with M4 and M5, or portion thereof, were conducted during the airport’s non-operational hours and would not be counted in the high level metric calculations.

• Times associated with M6, M7, and M8 were conducted during the airport’s operational hours in reporting period 2 and will be counted as operational downtime for high level metric calculation for reporting period 2. It will also be counted as one failure for reporting period 2 since operational downtime is > 0.

M1 M2 M3 M7 M8 M9

Airport Operational Hours

Record in TSE Database

TSE Operational Downtime 1 Failure

T1 T2 T3 T8T1 T2 T3 T4 T9 T10

• M9 will not be included in any high level metric calculations since the TSE should be operational at T9 awaiting TSA acceptance.

Figure 3 - Reporting Period Determination

The high level metric will also identify when critical or non-critical, and relevant or non-relevant failures will be included in the count. This identification and the reporting period will determine the number of failures and times to include in the metric calculations.

5. Operational RMA Metrics

TSA will use the RMA metrics identified in this Section to evaluate the operational performance of the TSE during their operational life cycle. Therefore, for the following TSE RMA metrics only failures documented as relevant failures will be used in the calculation of the TSE operational downtime and failure count unless identified otherwise in a contract. This ensures the failure and associated downtime is relevant to the TSE’s inherent performance. The operational downtime will be calculated by summing for each high level metric the applicable individual low-level metrics, as identified in the metric, for only relevant failures that occur during an airport’s operational hours. No averaging will be used to calculate the downtimes.

The first day of the designated reporting period will always start at 12:00 AM and end at 11:59 PM on the last day of the reporting period.

Each TSE RMA metric identified in Sections 5.1 through 5.3 identifies the applicable low level metrics to count in a particular type failure and repair cycle. Table 5 provides an example of how this will be identified. While Table 5 shows all the low level metrics that should be measured in a particular repair cycle, only the low-level metrics identified in lightly shaded blocks with diagonal lines should be counted for that metric. The dark shaded blocks on the Table with no M-Values indicate that the associated low level metric should not be applicable to

M1 M2 M3 M4 M5 M6 M7 M8 M9

Record in TSE Database

Reporting Period 1 Reporting Period 2

Airport Operational Hours

TSE Operational DT 1 Failure

Airport Operational Hours

TSE Operational DT 1 Failure

T1 T2 T3 T8T1 T2 T3 T7 T9 T10T4 T5 T6 that type repair cycle. Also note that the table does not show M9 which should be measured but not counted for these TSE RMA metrics.

Table 5 - Example Low Level Count

Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

DM Critical Relevant M1 M2 M3 M11 M12 M8

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

DM Non- Critical Relevant M1 M2 M3

M11 M12 M8

5.1 TSE Reliability Metrics

5.1.1 Mean Time Between Critical Failures

Mean Time Between Critical Failure (MTBCF) will evaluate the reliability of TSE in relation to all critical relevant failures. MTBCF is the average operational time TSE is available to perform their mission between critical relevant failures. MTBCF will be calculated for a single TSE or grouping of TSE in accordance with the contract RMA reporting requirements as follow:

1. Multiply daily airport operational hours as defined in contract by the number of TSE units in the TSE calculation population (one or more units) and the number of days in the reporting period;

2. Subtract the sum of the operational downtime associated with all critical relevant failures from the TSE calculation population (one or more units) during the reporting period;

3. Divide the difference by the total number of critical relevant failures with operational downtime > 0 (e.g., a critical relevant failure that fails and is repaired during the airport’s non-operational hours accrued no operational down time so is not included in the total number of failures) from the TSE calculation population (one or more units) during the reporting period;

MTBCF will be calculated for a specific reporting period (e.g., one month) and TSE combination (one or more units) based on the following graphical displays of the formula:

Individual TSE unit:

MTBCF =

(Airport Operational Hours x Number of Days) – (Sum of TSE Critical

Relevant Operational Downtime) Total Number of Critical Relevant Failures with Operational Downtime >0

Two or more TSE units:

MTBCF =

(Airport Operational Hours x Number TSE Units x Number of Days) –

(Sum of TSE Critical Relevant Operational Downtime) Total Number of Critical Relevant Failures with Operational Downtime >0

Where:

Airport Operational Hours = Period of time within 24-hour day that an airport is operational (airport operational hours).

• TSE Critical Relevant Operational Downtime = Period of time during which a TSE is not in a condition to perform its mission due to a critical relevant failure during an airport’s operational hours.

• Total Number of Critical Relevant Failures = Critical relevant failures that incur operational downtime and are counted in the metric calculations (where each critical relevant failure’s operational downtime > 0). If a critical relevant failure’s low-level metric times occur during the airport’s non-operational hours and incur no operational downtime, then this is not counted as a failure in the MTBCF operational downtime calculations.

For the type failures listed below, the low-level metrics (M-Values) identified in lightly shaded blocks with diagonal lines are counted toward the TSE operational downtime for MTBCF when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the MTBCF operational downtime. The operational downtime should be based on low level metrics that are corrected for airport operational hours. If a maintenance repair cycle continues into two airport operational hours within the same reporting period, the operational downtime time within each of the airport operational hours will be summed together and considered one failure if operational downtime > 0. When a maintenance repair cycle continues into a new reporting period, the operational down times and failure count will be calculated in the new reporting period as defined in Section 4.7.

Table 6 - MTBCF Count

Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

DM Critical Relevant M1 M2 M3

5.1.2 Mean Uptime

Mean Uptime (MUT) will evaluate the reliability of TSE in relation to all critical relevant and non-critical relevant failures. MUT is the average operational time TSE is available to perform their mission between all relevant failures (critical or non-critical). MUT is equivalent to Mean Time Between Failures (MTBF). MUT will be calculated for a single TSE or grouping of TSE in accordance with the RMA reporting requirements as follow:

units in the TSE calculation population (one or more units) and the number of days in the reporting period;

2. Subtract the sum of the operational downtime associated with all critical relevant and non-critical relevant failures from the TSE calculation population (one or more units) during the reporting period;

3. Divide the difference by the total number of critical relevant and non-critical relevant failures with operational downtime > 0 (e.g., a relevant failure that fails and is repaired during the airport’s non-operational hours accrued no operational down time so is not included in the total number of failures) from the TSE calculation population (one or more units) during the reporting period;

MUT will be calculated for a specific reporting period (e.g., one month) and TSE combination (one or more units) based on the following graphical displays of the formula.

MUT =

(Airport Operational Hours x Number of Days) – (Sum of all TSE Relevant

Operational Downtime) Total Number of Relevant Failures with Operational Downtime > 0

MUT =

(Airport Operational Hours x Number TSE Units x Number of Days) – (Sum of all TSE Relevant Operational Downtime

• Airport Operational Hours = The period of time within 24-hour day that an airport is operational (airport operational hours). A TSE is required to be available to perform its required mission during an airport’s operational hours.

• TSE Relevant Operational Downtime = The period of time during which a TSE is not in a condition to perform its mission due to any relevant failure (critical and non-critical) during an airport’s operational hours.

• Total Number of Relevant Failures = Relevant failures that incur operational downtime and are counted in the metric calculations (where each relevant failure’s operational downtime > 0). If a relevant failure’s low-level metric times occur during the airport’s non-operational hours and incur no operational downtime, then this is not counted as a failure in the MUT operational downtime calculations.

blocks with diagonal lines are counted toward the TSE operational downtime for MUT when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the

MUT operational downtime calculations. The operational downtime should be based on low level metrics that are corrected for airport operational hours. If a maintenance repair cycle continues into two airport operational hours within the same reporting period, the operational downtime within each airport operational hours will be summed together and considered one failure if operational downtime > 0. When a maintenance repair cycle continues into a new

Table 7 - MUT Count

Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

DM Critical Relevant M1 M2 M3 M11 M12 M8

DM Non-

5.2 TSE Maintainability Metrics

5.2.1 Mean Downtime

Mean Downtime (MDT) will evaluate the operational downtime of TSE in relation to all critical relevant and non-critical relevant failures. MDT is the average TSE operational downtime associated with all critical relevant and non-critical relevant failures during an airport’s operational hours. MDT will be calculated for a single TSE or grouping of TSE in accordance with the RMA reporting requirements as follow:

1. Sum the TSE operational downtime associated with all critical relevant and non-critical relevant failures from the TSE calculation population (one or more units) during the reporting period;

2. Divide by the total number of critical relevant and non-critical relevant failures with operational downtime >0 (e.g., a relevant failure that fails and is repaired during the airport’s non-operational hours accrued no operational down time so is not included in the total number of failures) from the TSE calculation population (one or more units) during the reporting period.

MDT will be calculated for a specific reporting period (e.g., one month) and TSE combination (one or more units) based on the following graphical display of the formula.

One or more TSE units;

MDT = Sum of TSE Relevant Operational Downtime

• TSE Relevant Operational Downtime = The time a TSE is down for repair due to any relevant failure during an airport’s operational hours.

• Total Number of Relevant Failures = Relevant failures that incur operational downtime and are counted in the metric calculations (where each relevant failure’s operational downtime > 0). If a relevant failure’s low-level metric times occur during the airport’s blocks with diagonal lines are counted toward the TSE operational downtime for MDT when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the MDT operational downtime calculations. The operational downtime should be based on low level metrics that are corrected for airport operational hours. If a maintenance repair cycle continues into two airport operational hours within the same reporting period, the operational downtime within each airport operational hours will be summed together and considered one failure if operational downtime > 0. When a maintenance repair cycle continues into a new

Table 8 - MDT Count

Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

DM Critical Relevant M1 M2 M3 M11 M12 M8

DM Non-

5.2.2 Mean Time to Repair

Mean Time to Repair (MTTR) will evaluate the operational downtime required to diagnose and repair a TSE due to all critical relevant and non-critical relevant failures during an airport’s operational hours. MTTR will be calculated for a single TSE or grouping of TSE in accordance with the RMA reporting requirements as follow:

1. Sum the TSE operational downtime associated with Time to Repair (M10) for all critical relevant and non-critical relevant failures from the TSE calculation population (one or more units) during the reporting period;

2. Divide by the total number of critical relevant and non-critical relevant failures with operational downtime > 0 (e.g., a relevant failure that fails and is repaired during the airport’s non-operational hours accrued no operational down time for M3, M7, and M8 is not included in the total number of failures) from the TSE calculation population during the reporting period.

MTTR will be calculated for a specific reporting period (e.g., one month) and TSE combination (one or more units) based on the following graphical display of the formula.

One or more TSE units:

MTTR =

Sum of TSE Relevant Corrective Maintenance Operational Downtime

(M3 + M7 + M8 = M10) Total Number of Relevant Failures with Operational Downtime M10 > 0

• TSE Relevant Corrective Maintenance Operational Downtime = Duration of corrective maintenance actions associated with diagnosis, repair, and testing of TSE relevant failures during an airport’s operational hours.

• Total Number of Relevant Failures = Relevant failures that incur operational downtime and are counted in the metric calculations (where each relevant failure’s operational downtime > 0). If a relevant failure’s low-level metric times occur during the airport’s blocks with diagonal lines are counted toward the TSE operational downtime for MTTR when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the MTTR operational downtime calculations. The operational downtime should be based on low level metrics that are corrected for airport operational hours. If a maintenance repair cycle continues into two airport operational hours within the same reporting period, the operational downtimes within each airport operational hours will be summed together and considered one failure if operational downtime > 0. When a maintenance repair cycle continues into a new

Table 9 - MTTR Count

Type Failure Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

5.3 TSE Availability Metrics

5.3.1 Operational Availability

Operational Availability (Ao) will evaluate the operational availability of TSE in relation to all critical relevant and non-critical relevant failures. It is the percentage of time during an airport’s operational hours that TSE is available to perform its required mission during a specified reporting period. Ao will be calculated in accordance with the RMA reporting requirements as follow:

units in the contract TSE calculation population (one or more units) and the number of days in the reporting period;

2. Subtract the sum of the operational downtime for relevant critical and non-critical failures within the TSE calculation population (one or more units) during the reporting period;

3. Divide the difference by the daily airport operational hours multiplied by the number of TSE units within the calculation population (one or more units) and the number of days in the reporting period.

Ao will be calculated for a specific reporting period (e.g., one month) and TSE combination (one or more units) based on the following graphical display of the formula:

Ao = (Airport Operational Hours x Number of Days) – (Sum of TSE Relevant

Operational Downtime) (Airport Operational Hours x Number of Days)

Ao = (Airport Operational Hours x TSE Units x Number of Days) – (Sum of TSE

Relevant Operational Downtime) (Airport Operational Hours x TSE Units x Number of Days)

• Airport Operational Hours = Period of time within a 24-hour day that an airport is operational (airport operational hours). A TSE is required to be available to perform its required mission during an airport’s operational hours.

• TSE Relevant Operational Downtime = The time a TSE is down for repair due to any relevant failure during an airport’s operational hours.

blocks with diagonal lines are counted toward the TSE operational downtime for Ao when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the Ao operational downtime calculations. The operational downtime should be based on low level metrics that are corrected for airport operational hours. If a maintenance repair cycle continues into two or more airport operational hours within the same reporting period, the operational downtimes within each airport operational hours will be summed together if operational downtime > 0. When a maintenance repair cycle continues into a new reporting period, the operational downtimes in the second reporting period will be calculated in the new reporting period as defined in Section 4.7.

Table 10 - Ao Count

Low-Level Metrics

Level Criticality Relevancy CM Critical Relevant M1 M2 M3 M4 M5 M6 M7 M8

CM Non- Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

DM Critical Relevant M1 M2 M3 M11 M12 M8

DM Non-

6. Average Depot Turn Around Time

TSA considers Depot Turn Around Time (DTAT) as a top level performance measure. DTAT (M11) measures the time a TSE remains away from their designated location for repair at the depot or in depot status. The Average DTAT metric will evaluate the average time TSE remain away from their designated location for repair at the depot or in depot status for a reporting period. It will include all critical relevant, non-critical relevant, critical non-relevant, and non-critical non-relevant failures that have an M11 in their repair cycle. Each M11 is measured as the duration between the time when the diagnosis has been completed and the DM decision made (T4) until the TSE is ready for setup at its designated location after the completion of depot repair (T11). M11 will be measured in 24 clock hours, or portions thereof, rather than an airport’s operational hours. For Average DTAT metric calculations, a M11 time will be included in the reporting period when completed; the times will not be split between reporting periods. The Average DTAT metric will be calculated for a specific reporting period (e.g., one month) based on the following graphical display of the formula:

Average DTAT = Sum of Depot Turn Around Time Hours ( M11) Number of DM Failures with M11

• Depot Turn Around Time Hours = Number of hours, using a 24 hour day, seven day week, a TSE is away from its operational location for repair at a depot or in depot repair status at the site.

• Number of DM Failures = Number of DM failures with M11 and counted in the metric calculations.

For the type failures listed below the low-level metrics M11 times identified in lightly shaded blocks with diagonal lines are counted toward the Average DTAT metric using a 24 hour day, seven day week.

Table 11 - DTAT Metric Count

Type Failure Low-Level Metrics

Level Criticality Relevancy

DM Critical Relevant M1 M2 M3 M11 M12 M8 M9

DM Non- Critical Relevant M1 M2 M3

M11 M12 M8 M9

DM Critical Non- Relevant M1 M2 M3

M11 M12 M8 M9

DM Non- Critical

Non- Relevant M1 M2 M3

M11 M12 M8 M9

Appendix A – List of Definitions

Corrective Maintenance. All repair cycle activities performed by an FST as a result of a failure to restore the TSE to a full operational condition.

Critical Failure. Failures that cause the TSE to be non-operational and requires immediate action by the FST to restore the TSE to full operational capability.

Depot Maintenance. If the TSE cannot be repaired at the installed location, it will be sent to a depot for repair or put in depot status on site.

Downtime. The period of time during which a TSE is not in a condition to perform its required mission, this includes all times that the TSE is not available due to CM or DM. This can be either downtime during airport’s operational hours or downtime (needing repair) during non-operational hours.

Field Service Technician. A trained and certified technician authorized to service a TSE.

Non-Critical Failure. Non-critical failures are those where the TSE is still able to perform its function until the FST begins the repair, at which time the downtime for the repair begins. This will be resolved by either CM or DM, although the DM resolution is less likely for this type of failure.