Attachment D7 - TSA ILS SLA Performance Metrics.rtf

Integrated Logistics Support (ILS) Mar 2019 file_0.jpg file_1.wmf

Transportation Security Administration Integrated Logistics Support (ILS) Service Level Agreement Performance Metrics

Attachment D-7

March 2019

Table of Contents

DTAT SLA Performance Requirement… 81

Purpose

The purpose of this document is to define the process for developing the Contractor performance metrics specified in Part C, Section C.8.10, CLS Service Level Agreement (SLA) Performance Requirements, to evaluate compliance with the SLA performance requirements; and the process for developing the metrics required to determine disincentives specified in Part A, Section A.2.6, Disincentives, that support Contractor development of invoices in accordance with Part B, Section B.3.4.7, Development of Invoices. The maintenance elements used in the development of these metrics are provided in Part D, Attachment 3, Transportation Security Equipment (TSE) Terms and Definitions & Reliability, Maintainability & Availability (RMA) Metrics.

Scope

This document identifies the process the Contractor should use to develop the metrics to demonstrate compliance with the SLA performance requirements and substantiate the determinations made for invoice development. It provides the calculation instructions for each metric required by the contract.

Data Collection and Reporting Process

The process identified in this document relies on the maintenance elements collected from the corrective maintenance actions conducted to return TSE to full operational capability. These maintenance elements will be collected in accordance with the process identified in Part D, Attachment 3, and will be documented in the TSE Database in accordance with the Contract Data Requirements List (CDRL) A004, TSE Database. This includes the time points (T-Values) and low-level metrics (M-Values) measured for each time points.

Sections 4 through 6 below identify the process for calculating the SLA performance metrics defined in Part C, Section C.8.10, and for calculating metrics to support disincentives that may be applied to invoices as defined in Part B, Section B.3.4.7. These metrics will be calculated as specified in each section and will be reported to TSA as part of the TSE RMA Metrics report in accordance with Contract Data Items List (CDRL) A018, TSE RMA Metrics report, as required. The TSE RMA Metrics Report must clearly identify the low level metrics and times for each used in the calculation of a metric.

Operational Availability (Ao)

The Government seeks to minimize TSE’s downtime associated with relevant failures. The Government will measure overall Contractor performance using Operational Availability (Ao) as defined in Part C, Section C.8.10.1. Ao will evaluate the operational availability of TSE in relation to All relevant critical failures and all relevant non-critical failures (includes corrective maintenance, depot maintenance and telephone customer support work orders). It is the percentage of time during an airport’s operational hours that TSE is available to perform its required mission during a specified reporting period. For Ao, TSA only considers a TSE’s downtime when it is not in a condition to perform its required mission during an airport’s operational hours. The counting of low-level metrics times for Ao will be based on the TSE operating time and TSE operational downtime as defined below.

Part C, Section C.8.10.1, Operational Availability, establishes the Ao performance level the Contractor must meet to satisfy the overall SLA performance requirement. Part B, Section B.3.4.7.1, Operational Availability Disincentive, identifies the performance level that can generate disincentives for non-compliance with the overall SLA performance requirement. Below is the process for developing the Ao metrics requirement.

Ao SLA Performance Requirement

For determining compliance with the Ao SLA performance requirement established in Part C, Section C.8.10.1, the Contractor shall calculate Ao monthly for each technology (e.g., all AIT models, all ETD models, etc.) at each airport using the airport’s operational hours as the required system operating time, and should only include installed operational units in the calculations.

The operational downtime will be calculated by summing the individual low-level metric times of each applicable failure that incurs operational downtime > 0. No averaging will be used over a reporting period.

To evaluate SLA compliance, Ao will be calculated monthly at each airport for each technology as follows:

Multiply daily airport operational hours as defined in Part D, Attachment 1, Airport Data, by the number of TSE units in a technology at an airport and the number of days in the reporting month period. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time.

Subtract the sum of the operational downtime for all relevant critical (Priority 1 (P1)) and all relevant non-critical (Priority 2 (P2)) failures for the same TSE units used in Step 1 for a technology at an airport during the reporting period.

Divide the difference by the total operating hours calculated in Step 1 (daily airport operational hours multiplied by the same number of TSE units at an airport in a technology and the number of days in the reporting period).

Invoice Development

For determining if Contractor performance meets the threshold for applying a performance disincentive to an invoice in accordance with Part B, Section B.3.4.7.1, Operational Availability Disincentive, the Contractor shall apply the same Ao calculations developed for the monthly Ao SLA performance requirement. For disincentive determination, Ao will be calculated monthly for each technology (e.g., all AT models, all ETD models, etc.) at each airport using the airport’s operational hours as the required system operating time, and should only include installed operational units in the calculations.

To determine if disincentives should be applied to an invoice, Ao will be calculated monthly at each airport for each technology as follows:

Multiply daily airport operational hours as defined in Part D, Attachment 1, by the number of TSE units in a technology at an airport and the number of days in the reporting month. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time.

Subtract the sum of the operational downtime for all relevant critical and all relevant non- critical relevant failures for the same TSE units used in Step 1 for a technology at an airport during the reporting period (Open-Ticket accounting method is applied). The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (Open-Ticket Accounting method is where all tickets’ (either open or closed status) down time are counted in the calculation.)

Divide the difference by the total operating hours calculated in Step 1 (daily airport operational hours multiplied by the same number of TSE units at an airport in a technology and the number of days in the reporting period).

Below is the Ao formula that will be used to develop the SLA performance metric and the disincentive threshold metric:

(Airport Operational Hours x TSE Units x Number of Days) –

Where:

Ao = (Sum of TSE all relevant critical (P1) and relevant non-critical (P2) failures Operational Downtime) file_2.wmf

(Airport Operational Hours x TSE Units x Number of Days)

Airport Operational Hours = Period of time within a 24-hour day that an airport is operational (airport operational hours).

TSE Relevant Operational Downtime = The time a TSE is down for repair due to any relevant failure during an airport’s operational hours

For the failure types listed below in Table 1, the low-level metrics (M-Values) identified in non-shaded blocks are counted toward the TSE operational downtime for Ao when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the Ao operational downtime calculations. If a maintenance repair cycle continues into two or more airport operational hours within the same reporting period, the operational downtimes within each airport operational hours will be summed together if operational downtime > 0. When a maintenance repair cycle continues into a new reporting period, the operational downtimes in the second reporting period will be calculated in the second reporting period as defined in Part D, Attachment 3, Section 4.7.2, Open Ticket Accounting. The lightly shaded blocks with M-Values are measured but not counted in the Ao operational downtime calculations. The dark shaded blocks with no M-Values on the Table indicate that the associated low-level metric should not be applicable to that failure type in its repair cycle as defined in Part D, Attachment 3.

Table 1 - Ao Low-Level Metrics Count Failure Type Low-Level Metrics Level Criticality Relevancy

CM

Critical Relevant M1 M2 M3 M4 M5 M6 M7

M8

CM

Non-Critical Relevant M1 M2 M3 M4 M5

DM

Critical Relevant M1 M2 M3

M11 M12 M8

TCS

Critical Relevant M1

M13

TCS

Non-Critical Relevant M1

M13

Average Parts Availability Time (PAT)

The Government seeks to minimize system downtime associated with parts availability for all failures. As defined and measured in accordance with Part D, Attachment 3, this includes the low level metrics M4 (Part(s) Request Time) and M5 (Part(s) Fulfillment Time, using airport operational hours. TSA has developed an Average Parts Availability Time (PAT) metric to evaluate compliance. This metric calculates the average low-level metrics (M4 + M5 > downtimes for all critical failures.

Part C, Section C.8.10.2, Parts Availability Time Supporting Performance Metric, establishes the Contractor SLA performance requirements for average parts availability time. Below is the process for developing the Average PAT metrics requirement.

PAT SLA Performance Requirement

For determining compliance with the Average PAT SLA performance requirement established in Part C, Section C.8.10.2, the Contractor shall calculate monthly average downtimes associated with all parts availability (M4 + M5 > 0) relating to all critical failures for the reporting month.

The downtime for the month will be calculated by summing the individual low-level metric times of each applicable M4 and M5 that incurs downtime > 0. No averaging over a reporting period (i.e. if quarterly reporting, no averaging over 3 months). Each month will report average PAT.

To evaluate SLA compliance, the Average PAT metric will be calculated monthly for all parts availability activities (M4 + M5 > 0) as follows:

Sum the individual TSE downtimes of each low-level metric (M4 + M5 > 0) from all critical failures (relevant and non-relevant) completed during the reporting month using airport operational hours. Only (M4 + M5 > 0) activities completed in the month will be included using the Closed-Ticket Accounting method. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (Closed ticket accounting is where only tickets that are closed during the reporting period counts. For example, if a ticket is open in period 1 and has not closed until the period 2, this ticket’s M4 and M5 will count in period 2).

Divide the sum by the total number of occurrences (work orders) during the reporting month that have an (M4 + M5 > 0) and accrue positive downtime during the month.

The Average PAT metric will be calculated as explained above based on the following formula:

Average PAT = Sum of Parts Availability Times (M4 + M5) file_3.wmf

Number of Critical Failures with Parts Availability Times where M4+M5 > 0

For the failure types listed below in Table 2, the low-level metrics (M-Values) identified in non-shaded blocks are counted toward the downtime associated with PAT. For the Average PAT metric calculations, a (M4 + M5 > 0) time will be included in the reporting period when completed as defined in Part D, Attachment 3, Section 4.7.1, Closed-Ticket Accounting; the times in an (M4 + M5 > 0) occurrence will not be split between reporting periods. The lightly shaded blocks with M-Values are measured but not counted in the PAT metrics calculations. The dark shaded blocks with no M-Values on the Table indicate that the associated low-level metric should not be applicable to that failure types in its repair cycle as defined in Part D, Attachment 3.

Table 2 - PAT Low-Level Metrics Count Failure Type Low-Level Metrics Level Criticality Relevancy

CM

Critical Relevant M1 M2 M3 M4 M5

Critical Non-Relevant M1 M2 M3 M4 M5

Average Technician Response Time (TRT)

The Government seeks to minimize system downtime associated with Field Service Technician (FST) response time to all critical failures. As defined and measured in accordance with Part D, Attachment 3, this includes low level metrics, M1 (Initial Call Center Interaction Time) and M2 (Technician Dispatch Time), using airport operational hours. TSA has developed an Average Technician Response Time (TRT) metric to evaluate compliance. This metric calculates the average low-level metrics (M1 + M2 > 0) downtimes for all critical failures.

Part C, Section C.8.10.3, Technician Response Time Supporting Performance Metric, establishes the Contractor SLA performance requirements for average FST response time. Below is the process for developing the Average TRT metric requirement.

TRT SLA Performance Requirement

For determining compliance with the Average TRT SLA performance requirement established in Part C, Section C.8.10.3, the Contractor shall calculate monthly average downtimes associated with all FST response times (M1 + M2 > 0) relating to all failures for the reporting month. The downtime for the month will be calculated by summing the individual low-level metric times of each applicable M1 and M2 that incurs downtime > 0. No averaging will be used over a reporting period (i.e. if quarterly reporting, no averaging over 3 months). Each month will report average TRT.

To evaluate SLA compliance, the Average TRT metric will be calculated monthly for all FST response activities (M1 + M2 > 0) as follows:

Sum the individual downtimes of each low-level metric (M1 + M2 > 0) from all critical failures (relevant and non-relevant) completed during the reporting month using airport operating hours. Only (M1 + M2 > 0) activities completed in the reporting month will be included using the Closed-Ticket Accounting method. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (Closed ticket accounting is where only tickets that are closed during the reporting period counts. For example, if a ticket is open in period 1 and has not closed until the period 2, this ticket’s M1 and M2 will count in period 2).

Divide the sum by the total number of occurrences during the past month that have an (M1 + M2 > 0) and accrue positive downtime during the month.

The Average TRT metric will be calculated as explained above based on the following formula:

Sum of M1 + M2 Average TRT = file_4.wmf

Number of Critical Failures where M1 + M2 >0

For the failure types listed below in Table 3, the low-level metrics M1 and M2 times identified in non- shaded blocks are counted toward the downtime associated with TRT. For the Average TRT metric calculations, a (M1 + M2 > 0) time will be included in the reporting period when completed as defined in Part D, Attachment 3, Section 4.7.1, Closed-Ticket Accounting; the downtimes (M1 + M2 > 0) occurrence will NOT be split between reporting periods. The lightly shaded blocks with M-Values are measured but not counted in the TRT metrics calculations. The dark shaded blocks with no M-Values on the table indicate that the associated low-level metric should not be applicable to that failure types in its repair cycle as defined in Part D, Attachment 3.

Table 3 – TRT Low-Level Metrics Count Failure Type Low-Level Metrics Level Criticality Relevancy

CM

Critical Relevant M1 M2 M3 M4 M5

Critical Non-Relevant M1 M2 M3 M4 M5

DM

Critical Relevant M1

M8

DM

Critical Non-Relevant M1

M8

Average Non-Critical Maintenance Actions

The Government seeks to minimize the length of time a non-critical maintenance action, as defined in Attachment 3, is unresolved. As defined and measured in accordance with Attachment 3, of Part D, this includes low level metrics, M1 to M8, of all non-critical maintenance actions. This metric calculates the average low-level metrics (M1 to M8) of downtime for all CM non-critical failures. The counting of low-level metrics for this calculation will be based on the 24-clock hours.

Part C, Section C.8.10.5, Non-Critical Maintenance Action Completion Supporting Performance Metric, establishes the Contractor SLA performance requirement.

SLA Performance Requirement:

For determining compliance with the Average Non-Critical Maintenance Action Time SLA performance requirement, the Contractor shall calculate monthly average downtimes associated with all CM non-critical (relevant and non-relevant) for the past month. The downtime for the month will be calculated by summing the individual low-level metric times of each applicable failure.

To evaluate SLA compliance, the Average Non-Critical Maintenance Actions metric will be calculated monthly as follows:

The Average Non-Critical Maintenance Action Time metric will be calculated as explained above based on the following graphical display of the formula:

Average Non-Critical (Priority 2) Failure = Sum of Non-Critical Failures (M1 to M8) downtime

Number of Non-Critical Failures

Where Non-Critical (Priority 2 (P2)) Failure = Failures where a TSE needs corrective maintenance but still operates at full mission capability until the minor repair is completed.

For the failure types listed below in Table 4, the low-level metrics (M-Values) identified in non-shaded blocks are counted toward the Non-Critical Maintenance actions. For the Average Non-Critical Maintenance Action Time metric calculations, a (M1 to M8) time will be included in the reporting period when completed; the times in an (M1 to M8) occurrence will not be split between reporting periods. The dark shaded blocks with no M-Values on the Table indicate that the associated low-level metric should not be applicable to that type failure in its repair cycle as defined in Attachment 3 of Part D.

Table 4 – Non-Critical Maintenance Actions Type Failure Low-Level Metrics Level Criticality Relevancy

CM

Non-Critical Relevant M1 M2 M3 M4 M5

Non-Critical Non-Relevant M1 M2 M3 M4 M5

Average Depot Turn Around Time (DTAT)

The Government seeks to minimize system downtime associated with Depot Turn Around Time (DTAT) for all depot maintenance actions. As defined and measured in accordance with Attachment D, of Part 3, this includes low level metrics M11 Depot Maintenance, using 24 clock-hours. TSA has developed an Average DTAT metric to evaluate compliance. This metric will evaluate the average time TSE remain away from their designated location for repair at the depot or in depot status for a reporting period. It will include all Depot Maintenance failures that have an (M11 >0) in their repair cycle. Each M11 is measured as the number of hours between the time when the diagnosis has been completed and the DM decision is made (T4) and the time when the TSE is ready for setup at its designated location after the completion of depot repair (T11). M11 will be measured in 24 clock hours rather than an airport’s operational hours. For the Average DTAT metric calculations, a M11 time will be included in the reporting period when completed using the Closed-Ticket Accounting method; the times will not be split between reporting periods.

Part C, Section C.8.10.4, Average Deport Turn Around Time (DTAT), establishes the Contractor SLA performance requirements for average DTAT and identifies the performance level threshold for the Contractor performance. Below is the process for developing the Average DTAT metrics requirement.

DTAT SLA Performance Requirement:

For determining compliance with the Average DTAT SLA performance requirement established in Part C, Section C.8.10.4, the Contractor shall calculate monthly the average downtimes associated with all DTAT (M11 > 0) completed during the reporting month. The Average DTAT for the month will be calculated by summing the individual low-level metric times of each applicable M11 > 0. No averaging will be used over a reporting period (i.e. if quarterly reporting, no averaging over 3 months). Each month will report average DTAT.

To evaluate SLA compliance, the Average DTAT metric will be calculated monthly for all Depot Repair activities (M11 > 0) as follows:

Sum the individual times of each low-level metric (M11 > 0) from all depot maintenance occurrences completed during the reporting month. Only (M11 > 0) activities completed in the month will be included using the Closed-Ticket Accounting method. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (Closed ticket accounting is where only tickets that are closed during the reporting period counts. For example, if a ticket is open in period 1 and has not closed until the period 2, this ticket’s M11 will count in period 2).

Divide the sum by the total number of (M11) occurrences completed during the past month that have an (M11 > 0).

The Average DTAT metric will be calculated as explained above based on the following formula:

Average DTAT = Sum of Depot Turn Around Times (M11 > 0) Number of DM Failures (where M11 > 0)

For the failure types listed below in Table 5, the low-level metrics M11 times identified in non-shaded blocks are counted toward the average DTAT using a 24-clock hours. The lightly shaded blocks with M-Values are measured but not counted in the DTAT downtime calculations. The dark shaded blocks with no M-Values indicate that the associated low-level metric should not be applicable to that type failure in its repair cycle as defined in Attachment 3 of Part D.

Table 5 – DTAT Low-Level Metrics Count Failure Type Low-Level Metrics Level Criticality Relevancy

DM

Critical Relevant M1

M8

DM

Critical Non-Relevant M1