Attachment_7_-_ILS_SLA_Performance_Metrics.docx

Transportation Security Administration

Integrated Logistics Support (ILS)

Service Level Agreement

Performance Metrics

May 2017

Table of Contents

Purpose The purpose of this document is to define the process for developing the Contractor performance metrics specified in Part C, Section C.8.10, CLS Service Level Agreement (SLA) Performance Requirements, to evaluate compliance with the SLA performance requirements; and the process for developing the metrics required to determine disincentives or incentives specified in Part A, Section A.1.2.8, Disincentives and Incentives, that support Contractor development of invoices in accordance with Part B, Section B.3.4.7, Development of Invoices. The maintenance elements used in the development of these metrics are provided in Part D, Attachment 3, Transportation Security Equipment (TSE) Terms and Definitions & Reliability, Maintainability & Availability (RMA) Metrics.

Scope This document identifies the process the Contractor should use to develop the metrics to demonstrate compliance with the SLA performance requirements and substantiate the determinations made for invoice development. It provides the calculation instructions for each metric required by the contract.

Data Collection and Reporting Process The process identified in this document relies on the maintenance elements collected from the corrective maintenance actions conducted to return TSE to full operational capability. These maintenance elements will be collected in accordance with the process identified in Part D, Attachment 3, and will be documented in the TSE Database in accordance with the Contract Data Requirements List (CDRL) A004, TSE Database. This includes the time points (T-Values) and low-level metrics (M-Values) measured for each time points.

Sections 4 through 6 below identify the process for calculating the SLA performance metrics defined in Part C, Section C.8.10, and for calculating metrics to support disincentives and incentives that may be applied to invoices as defined in Part A, Section A.1.2.8. These metrics will be calculated as specified in each section and will be reported to TSA as part of the TSE RMA Metrics report in accordance with Contract Data Items List (CDRL) A018, TSE RMA Metrics report, as required. The TSE RMA Metrics Report must clearly identify the low level metrics and times for each used in the calculation of a metric.

Operational Availability (Ao) The Government will measure overall Contractor performance using Operational Availability (Ao) as defined in Part D, Attachment 3. Ao will evaluate the operational availability of TSE in relation to All relevant and non-relevant CRITICAL failures and all relevant non-critical relevant failures (includes corrective maintenance, depot maintenance, and telephone customer support work orders). It is the percentage of time during an airport’s operational hours that TSE is available to perform its required mission during a specified reporting period. For Ao, TSA only considers a TSE’s downtime when it is not in a condition to perform its required mission during an airport’s operational hours. The counting of low-level metrics times for Ao will be based on the TSE operating time and TSE operational downtime as defined below.

TSE Operating Time – The period of time TSE is available to perform its required mission during the operational hours of a facility as specified in Part D, Attachment 1, Airport and Inventory Data. During the Operations & Maintenance (O&M) life cycle phase, TSE operating time will be equal to the airport’s operational hours (period of time within 24-hour day that an airport is operational) unless the TSE experiences any operational downtime.

TSE Operational Downtime – The period of time during an airport’s operational hours that TSE is not in a condition to perform its required mission. This includes all times that a system is not available due to applicable on-site Corrective Maintenance (CM), Depot Maintenance (DM), and Telephone Customer Support (TCS) Low-Level Metric times during an airport’s operational hours. The determination of the operational downtime is made after all the low-level metrics are recorded against the 24-hour day and measured against the airport’s operational hours.

Part C, Section C.8.10.1, Operational Availability, establishes the Ao performance level the Contractor must meet to satisfy the overall SLA performance requirement. Part A, Section A.1.2.8.1(a), Operational Availability Disincentive, identifies the performance level that can generate disincentives for non-compliance with the overall SLA performance requirement. Below is the process for developing the Ao metrics for these two requirements.

SLA Performance Requirement For determining compliance with the Ao SLA performance requirement established in Part C, Section C.8.10.1, the Contractor shall calculate Ao monthly for each technology (e.g., all AT models, all ETD models, etc.) at each airport using the airport’s operational hours as the required system operating time, and should only include installed operational units in the calculations. The operational downtime will be calculated by summing the individual low-level metric times of each applicable failure that incurs operational downtime > 0. No averaging will be used over a reporting period.

To evaluate SLA compliance, Ao will be calculated monthly at each airport for each technology as follows:

1. Multiply daily airport operational hours as defined in Part D, Attachment 1, Airport Data, by the number of TSE units in a technology at an airport and the number of days in the month’s reporting period. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time.

2. Subtract the sum of the operational downtime for all relevant and non-relevant critical (Priority 1 (P1)) and all relevant non-critical (Priority 2 (P2)) failures for the same TSE units used in Step 1 for a technology at an airport during the reporting period.

3. Divide the difference by the total operating hours calculated in Step 1 (daily airport operational hours multiplied by the same number of TSE units at an airport in a technology and the number of days in the reporting period).

Invoice Development For determining if Contractor performance meets the threshold for applying a performance disincentive to an invoice in accordance with Part A, Section A.1.2.8.1(a), Operational Availability Disincentive, the Contractor shall apply the same Ao calculations developed for the monthly Ao SLA performance requirement. For disincentive determination, Ao will be calculated monthly for each technology (e.g., all AT models, all ETD models, etc.) at each airport using the airport’s operational hours as the required system operating time, and should only include installed operational units in the calculations.

To determine if disincentives should be applied to an invoice, Ao will be calculated monthly at each airport for each technology as follows:

1. Multiply daily airport operational hours as defined in Part D, Attachment 1, by the number of TSE units in a technology at an airport and the number of days in the monthly reporting period. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time.

2. Subtract the sum of the operational downtime for all relevant and non-relevant critical and all relevant non-critical relevant failures for the same TSE units used in Step 1 for a technology at an airport during the reporting period (end of month accounting method is applied). The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (End of Month Accounting method is where all tickets’ (either open or closed status) down time either closed or opened are counted in the calculation.)

3. Divide the difference by the total operating hours calculated in Step 1 (daily airport operational hours multiplied by the same number of TSE units at an airport in a technology and the number of days in the reporting period).

Below is the Ao formula that will be used to develop the SLA performance metric and the disincentive threshold metric:

(Airport Operational Hours x TSE Units x Number of Days)

Where:

Airport Operational Hours = Period of time within a 24-hour day that an airport is operational (airport operational hours). A TSE is required to be available to perform its required mission during an airport’s operational hours.

TSE Relevant Operational Downtime = The time a TSE is down for repair due to any relevant failure during an airport’s operational hours For the failure types listed below, the low-level metrics (M-Values) identified in non-shaded blocks are counted toward the TSE operational downtime for Ao when the low-level metric times, or portion thereof, occur during the airport’s operational hours. The low-level metric times conducted during the airport’s non-operational hours are not counted in the Ao operational downtime calculations. If a maintenance repair cycle continues into two or more airport operational hours within the same reporting period, the operational downtimes within each airport operational hours will be summed together if operational downtime > 0. When a maintenance repair cycle continues into a new reporting period, the operational downtimes in the second reporting period will be calculated in the new reporting period as defined in Part D, Attachment 3, Section 4.7.2, Open Ticket Accounting. The dark shaded blocks with no MValues on the Table indicate that the associated low-level metric should not be applicable to that failure type in its repair cycle as defined in Part D, Attachment 3.

Table 1 - Ao Low-Level Metrics Count

M8

M8

M8

M8

M13

M13

M13

M13

Average Parts Availability Time (PAT) The Government seeks to minimize system downtime associated with parts availability for all failures. As defined and measured in accordance with Part D, Attachment 3, this includes the low level metrics M4 (Part(s) Request Time) and M5 (Part(s) Fulfillment Time), using 24 clock hours. TSA has developed an Average Parts Availability Time (PAT) metric to evaluate compliance. This metric calculates the average low-level metrics (M4 + M5 > 0) downtimes for all critical and non-critical failures. Two different PAT metrics will be calculated: one for all critical (P1) failures where M4+M5>0 and another for all non-critical (P2) failures where M4+M5>0.

Part C, Section C.8.10.2, Parts Availability Time Supporting Performance Metric, establishes the Contractor SLA performance requirements for average parts availability, and Part A, Section A.1.2.8.2(a), Parts Availability Incentive, identifies the performance level that can generate incentives for improvements in the Contractor performance. Below is the process for developing the Average PAT metrics for these two requirements.

SLA Performance Requirement For determining compliance with the Average PAT SLA performance requirement established in Part C, Section C.8.10.2, the Contractor shall calculate monthly average downtimes associated with all parts availability (M4 + M5 > 0) relating to all critical and non-critical failures for the past month. The downtime for the month will be calculated by summing the individual low-level metric times of each applicable M4 and M5 that incurs downtime > 0. No averaging over a reporting period (i.e. if quarterly reporting, no averaging over 3 months). Each month will report average PAT.

To evaluate SLA compliance, the Average PAT metric will be calculated monthly for all parts availability activities (M4 + M5 > 0) as follows:

1. Sum the individual TSE downtimes of each low-level metric (M4 + M5 > 0) from all applicable critical or non-critical failures (relevant and non-relevant) completed during the past month using 24 clock hours. Only (M4 + M5 > 0) activities completed in the month will be included using the closed ticket accounting method. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time. (Closed ticket accounting is where only tickets that are closed during the reporting period counts. For example, if a ticket is open in period 1 and has not closed until the period 2, this ticket will count in period 2).

2. Divide the sum by the total number of occurrences (work orders) during the past month that have an (M4 + M5 > 0) and accrue positive downtime during the month.

Invoice Development For determining if Contractor performance meets the threshold for applying a performance incentive to an invoice in accordance with Part A, Section A.1.2.8.2(a), Parts Availability Incentive, the Contractor shall calculate once over each quarter, starting in option year 1, the average downtimes associated with all parts availability (M4 + M5 > 0) relating to all critical and non-critical failures for the past quarter. The downtime for the quarter will be calculated by summing the individual low-level metric times of each applicable failure that incur operational downtime> 0. No averaging will be used over a reporting period.

To determine if an incentive should be added to the invoice, the Average PAT metric will be calculated monthly and invoiced quarterly for all parts availability activities (M4 + M5 > 0) as follows:

1. Sum the individual downtimes of each low-level metric (M4 + M5 > 0) from all applicable failures completed during the past quarter (closed ticket accounting method is applied). Only (M4 + M5 > 0) activities completed in the quarter will be included, and no averaging will be used over the reporting period. The reporting period starts the first day (at 12:00 AM) of the quarter and ends on the last day (at 11:59 PM) of the quarter based on local time.

2. Divide the sum by the total number of occurrences (work orders) during the past quarter that have an (M4 + M5 > 0) and accrue positive downtime during the quarter.

The Average PAT metric will be calculated as explained above based on the following formula:

Number of Failures (work orders) with Parts Availability Times where M4+M5 > 0

For the failure types listed below, the low-level metrics (M-Values) identified in non-shaded blocks are counted toward the downtime associated with PAT. For the Average PAT metric calculations, a (M4 + M5 > 0) time will be included in the reporting period when completed as defined in Part D, Attachment 3, Section 4.7.1, Closed Ticket Accounting; the times in an (M4 + M5 > 0) occurrence will not be split between reporting periods. The dark shaded blocks with no M-Values on the Table indicate that the associated low-level metric should not be applicable to that failure types in its repair cycle as defined in Part D, Attachment 3.

Table 2 - PAT Low-Level Metrics Count

M8

M8

M8

M8

Average Technician Response Time (TRT) The Government seeks to minimize system downtime associated with Field Service Technician (FST) response time to all failures. As defined and measured in accordance with Part D, Attachment 3, this includes low level metrics, M1 (Initial Call Center Interaction Time) and M2 (Technician Dispatch Time), using 24 clock hours. TSA has developed an Average Technician Response Time (TRT) metric to evaluate compliance. This metric calculates the average low-level metrics (M1 + M2 > 0) downtimes for all critical (P1) and non-critical (P2) failures. Two different TRT metrics will be calculated: one for all critical failures where M1+M2>0 and another for all non-critical failures where M1+M2>0.

Part C, Section C.8.10.3, Technician Response Time Supporting Performance Metric, establishes the Contractor SLA performance requirements for average FST response time. Part A, Section A.1.2.8.2(b), Technician Response Time Incentive, identifies the performance level that can generate incentives for improvements in the Contractor performance. Below is the process for developing the Average TRT metric for these two requirements.

SLA Performance Requirement For determining compliance with the Average TRT SLA performance requirement established in Part C, Section C.8.10.3, the Contractor shall calculate monthly average downtimes associated with all FST response times (M1 + M2 > 0) relating to all failures for the past month. The downtime for the month will be calculated by summing the individual low-level metric times of each applicable M1 and M2 that incurs downtime > 0. No averaging will be used over a reporting period.

To evaluate SLA compliance, the Average TRT metric will be calculated monthly for all FST response activities (M1 + M2 > 0) as follows:

1. Sum the individual downtimes of each low-level metric (M1 + M2 > 0) from all applicable failures completed during the past month using 24 clock hours. Only (M1 + M2 > 0) activities completed in the month will be included, and no averaging will be used over the reporting period. The reporting period starts the first day (at 12:00 AM) of the month and ends on the last day (at 11:59 PM) of the month based on local time.

2. Divide the sum by the total number of occurrences during the past month that have an (M1 + M2 > 0) and accrue positive downtime during the month.

The Average TRT metric will be calculated as explained above based on the following formula:

Number of Failures (work orders) where M1 + M2 >0

For the failure types listed below, the low-level metrics M2 times identified in non-shaded blocks are counted toward the downtime associated with TRT. For the Average TRT metric calculations, a (M1 + M2 > 0) time will be included in the reporting period when completed as defined in Part D, Attachment 3, Section 4.7.1, Closed Ticket Accounting; the times in a (M1 + M2 > 0) occurrence will not be split between reporting periods. The dark shaded blocks with no M-Values on the table indicate that the associated low-level metric should not be applicable to that failure types in its repair cycle as defined in Part D, Attachment 3.

Table 3 – TRT Low-Level Metrics Count

M8

M8

M8

M8

image1.png