D_-_ITIOSS__TO1_QASP_SLAs.pdf

ATTACHMENT D

IT Infrastructure Operations Support Services (ITIOSS)

Task Order 1

IT Infrastructure Operations Support Services (ITIOSS)

Operations and Maintenance (O&M)

Development Modernization and Enhancements (DM&E)

Quality Assurance Surveillance Plan (QASP)

7-1-19

Table of Contents

1. INTRODUCTION

2. ROLES AND RESPONSIBILITIES

3. PERFORMANCE REQUIREMENTS SUMMARY TABLE

4. INCIDENT ESCALATION SYSTEM

5. PRIORITIZING SUPPORT SERVICES BY IMPACT AND URGENCY

1. INTRODUCTION

This quality assurance surveillance plan (QASP) establishes procedures and guidelines

ITIOD will use to ensure the Contractor achieves the required performance standards or service levels on the task order. This QASP identifies performance objectives, defines the methodologies used to monitor and evaluate the Contractor’s performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results. The cornerstone of performance management is the ability to make decisions based on the analysis of performance data; this analysis yields information that indicates whether the contractor is achieving expected outcomes for the project.

The QASP will use a performance management approach, an approach that focuses on results rather than the contractor’s method of achieving them. This approach departs from previous quality assurance strategies, such as scrutinizing contractor processes and practices

(some process reviews are required by law and compelling business situations, such as safety and health). A performance-based approach provides the Contractor with the flexibility to continuously improve and innovate over the course of the contract as long as it achieves critical outcomes and meets the desired performance levels.

Note that this QASP establishes a baseline of acceptable performance. The Award Fee

Determination Plan (AFDP) includes the Target Quality Levels (TQL), or the level of performance expected to obtain an award fee. Refer to the Award Fee Calculation (Table 1) of the Award Fee Plan for weightings of the specific SLAs.

a. Methodologies to Monitor Performance

PBGC will monitor performance through stakeholder surveys, 100% review of monthly and quarterly deliverables and through the receipt of unscheduled feedback. PBGC will collect all performance data for review by the Performance Review Board (PRB) on a monthly basis.

Additionally, the PRB reviews all incidents in addition to the monthly Program Status Report and PBGC and the Contractor will jointly address issues or downward trends in customer satisfaction. PBGC will work with the Contractor to implement improvement initiatives that impact internal PBGC processes as well as to define improved processes, roles and forums for governing all service areas. In addition to surveys, each month, the COR will conduct a review of all contractor-generated deliverables and collect a log of negative feedback from PBGC end-users during performance. Note: the term “User” is inclusive of all end-users, customers, or events that cause an incident to be reported.

2. ROLES AND RESPONSIBILITIES

The Contracting Officer

The Contracting Officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the Contracting Officer's Representative (COR) and the Contractor. The CO will designate one full-time COR as the government authority for performance management. The number of additional representatives serving as technical inspectors depends on the complexity of the services measured, as well as the contractor’s performance, and must be identified and designated by the COR.

The Contracting Officer’s Representative

The Contracting Officer’s Representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR limitations are contained in the written appointment letter. The COR is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The

COR is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for completing QA monitoring forms used to document the inspection and valuation of the contractor’s work performance. Government surveillance may occur under the inspection of services clause for any service relating to the contract.

Contract Monitor

Contract Monitors are the representatives closest to technical work performed. Monitors are personnel who have responsibility for reviewing technical performance and/or interacting with the contractor on a regular basis. The responsibilities of the Contract Monitors are to:

• Provide detailed information monthly to the COR and the Award Fee Board Members in writing and or orally before the board on the contractor's performance

• May attend board meeting at the discretion of the AFEB chair

Performance Review Board (PRB)

• Refer to section 4.2 of the Award Fee Determination Plan for the roles and responsibilities of the PRB.

3. PERFORMANCE REQUIREMENTS SUMMARY TABLE

3.1 CPAF Performance Metrics

PWS

Refer

-ence

Service Level

Agreement

Metric Title

Service Level Agreement Metric

Description

AQL Surveillan ce Method

Frequency Additional Notes

6.2.2 Tier 2+ customer

survey covered by CPAF CLIN

This metric captures the overall customer satisfaction in addressing incidents escalated to

ITIOSS Tier 2 (CPAF services). When the incident is closed, an email is delivered which includes a link to a customer service survey.

Customers are invited but not required to participate in the survey. Surveys submitted by contract team members are accepted for service improvement reasons but are excluded from the SLA performance metric. Metric is percentage of responses with overall rating of 4 or 5 on 5-point scale.

90% 100%

Responses to

Survey

Monthly &

Quarterly QASP Section 5.4

5.1.1 ITIOD

stakeholders survey

This metric provides the COR an opportunity to evaluate the contractor’s performance during the period and thereby directly influence award fee determination.

The COR determines the final score based on survey inputs and other factors. Our self-assessment is based on the results of the stakeholder surveys.

Accept-able

100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.3.8 Oracle

Production

Availability

Oracle infrastructure availability performance data is provided by PBGC’s implementation of the Micro Focus (HP) monitoring suite, specifically the Business

Service Manager (BSM) component.

99.80% 100%

Inspection/

Reporting

Quarterly HP BSM

Performance

Monitoring

Tools- PWS

Section 6.5

6.4.1 LAN Latency LAN Latency performance data is provided

by PBGC’s implementation of the

Scrutinizer network monitoring tool. The

Scrutinizer tool is currently implemented at headquarters and measures performance between managed devices located throughout PBGC’s HQ infrastructure. This

SLA does not cover PBGC's WAN performance, access to Office 365, or other cloud services.

<=5ms 100%

Inspection/

Reporting

Quarterly Scrutinizer

Network monitoring

Tools- PWS

Section 6.6

6.4.1 LAN

Infrastructure

Availability

LAN Availability performance data are provided by PBGC’s implementation of the

Scrutinizer network monitoring tool. The

Scrutinizer tool measures availability performance of key managed network devices throughout PBGC’s HQ infrastructure.

99.80% 100%

Inspection/

Reporting

Quarterly Scrutinizer

Network monitoring

Tools- PWS

Section 6.7

6.6.2 Endpoint

Compliance

(Antivirus)

Anti-virus update performance data is provided by IBM Endpoint Manager (IEM) configured for daily assessment of endpoint antivirus definition compliance.

Compliance is defined as having the antivirus client installed with a definition file that is no more than 10 days old.

99% 100%

Inspection/

Reporting

Quarterly IIBM BigFix

Tool-PWS

Section 6.6

6.6.3 Critical

Vulnerabilities

Remediated

This SLA evaluates Critical vulnerability remediation timeliness; specifically, the percentage resolved in the 30 days following detection.

85.00% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.3 High

Vulnerabilities

Remediated

This SLA evaluates High vulnerability remediation timeliness; specifically, the percentage resolved in the 90 days following detection.

80% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.3 Medium

Vulnerabilities

Remediated

This SLA evaluates Critical vulnerability remediation timeliness; specifically, the percentage resolved in the 120 days following detection.

65.00% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.3 Tracking Old

Vulnerabilities

Percentage of government-approved remediation plan or risk acceptance established within 30 days of initial target remediation date for Critical and High vulnerabilities that were not remediated by their target remediation timeline (30 and 90 for critical and high, vulnerabilities respectively) and documented in the PVMG

Tracker SharePoint list.

90% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.3 Remediating

Old Critical

Vulnerabilities

Percentage of all open critical vulnerabilities remediated or risk accepted during the reporting period that were unresolved and past their initial target remediation date at the start of the reporting period.

50.00% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.3 Remediating

Old High

Vulnerabilities

Percentage of all open high vulnerabilities remediated or risk accepted during the reporting period that were unresolved and past their initial target remediation date at the start of the reporting period.

25% 100%

Inspection/

Reporting

Quarterly See Attachment

A for detail

6.6.1 Security

POA&M

Completion

Timeliness

PBGC has numerous Plans of Action and

Milestones (POA&Ms) developed by the

PBGC Security Program Management

Office (SPMO) to address outstanding security, compliance, and accreditation issues. Many POA&Ms will include tasks and milestones assigned to the ITIOSS contractor. SLA performance is measured as a percentage of approved and assigned milestones completed on or before their respective planned completion dates.

Because some factors influencing task completion may be outside the control of the contractor, individual performance adjustments may be made with COR (or designee) and contractor PM (or designee) approval. Specific Milestones to be measured will be documented and approved by the COR (or designee) and contractor

PM (or designee) as they are developed by the SPMO and resourced in view of other

ITIOSS contract requirements

75% 100%

Inspection/

Reporting

Effected

Quarters

Timelines TBD

6.8.2 Secure

Configuration

Baseline (SCB)

Remediation

Timelines

This metric measures the timely remediation when an individual configuration item (CI), e.g. workstation server, database, device, etc. with an approved security configuration baseline checklist falls below the minimal CI baseline compliance threshold defined in approved baseline documents and noted in the table below. PBGC’s custom developed

Continuous Automated Compliance

99% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

Monitoring (CACM) solution, will record a non-compliance exception and a notification to the Service Desk on the first

Monday that follows when an individual CI has fallen below the minimal CI baseline compliance threshold. Contractor is responsible for remediation within 30 days of the first detection by correcting the configuration issue, removing the CI from network, or documenting a risk acceptance.

6.8.2 SCB CI

(Configuration

Item)Aggregat e Compliance

This metric measures the overall Secure

Configuration Baseline (SCB) compliance for all configuration items (CIs), e.g.

workstation server, database, device, etc.

with an approved security configuration baseline checklist in alignment with

PBGC’s acceptable and target aggregate CI baseline compliance thresholds. PBGC’s custom developed Continuous Automated

Compliance Monitoring (CACM) solution, records daily aggregate compliance levels for each baseline and these are averaged over the period of performance to calculate the SLA score. Contractor is responsible for ensuring aggregate baseline scores are maintained above acceptable and ideally at target levels by correcting configuration issues, removing CIs from the network when they fall below the minimal CI baseline compliance threshold and cannot be corrected, and/or documenting a risk acceptance.

90% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.8.2 Secure

Configuration

Baseline (SCB)

Aggregate

Compliance

This metric measures the overall Secure

Configuration Baseline (SCB) compliance for all configuration items (CIs), e.g.

workstation server, database, device, etc.

with an approved security configuration baseline checklist in alignment with

PBGC’s acceptable and target aggregate CI baseline compliance thresholds. PBGC’s custom developed Continuous Automated

Compliance Monitoring (CACM) solution, records daily aggregate compliance levels for each baseline and these are averaged over the period of performance to calculate the SLA score. Contractor is responsible for ensuring aggregate baseline scores are maintained above acceptable and ideally at target levels by correcting configuration issues, removing CIs from the network when they fall below the minimal CI baseline compliance threshold and cannot be corrected, and/or documenting a risk acceptance.

95.6% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.3, 6.4, 6.5, 6.6, 6.7

Time to resolve incident

(priority 1) covered by

CPAF CLIN

Report data is collected from the

ServiceNow application. Scoring is based on the timeliness of priority 1 incident tickets being closed/resolved as recorded within ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates. Adjustments are requested for

85% 100%

Inspection/

Reporting

Quarterly QASP Section

5.4 tickets which are out of the scope of ITIOD

(service failures based on events outside of

ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

6.3, 6.4, 6.5, 6.6, 6.7

Time to resolve incident

(priority 2) covered by

CPAF CLIN

Report data is collected from the

ServiceNow application. Report data is collected from the ServiceNow application.

Scoring is based on the timeliness of priority 2 incident tickets being closed/resolved as recorded within

ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates.

Adjustments are requested for tickets which are out of the scope of ITIOD (service failures based on events outside of ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

85% 100%

Inspection/

Reporting

Quarterly QASP Section

5.4

6.3, 6.4, 6.5, 6.6, 6.7

Time to resolve incident

(priority 3) covered by

CPAF CLIN

Report data is collected from the

ServiceNow application. Report data is collected from the ServiceNow application.

Scoring is based on the timeliness of priority 3 incident tickets being closed/resolved as recorded within

ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates.

Adjustments are requested for tickets which are out of the scope of ITIOD (service failures based on events outside of ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

85% 100%

Inspection/

Reporting

Quarterly QASP Section

5.4

6.3, 6.4, 6.5, 6.6, 6.7

Time to resolve incident

(priority 4) covered by

CPAF CLIN

Report data is collected from the

ServiceNow application. Scoring is based on the timeliness of priority 4 incident tickets being closed/resolved as recorded within ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates. Adjustments are requested for tickets which are out of the scope of ITIOD

(service failures based on events outside of

ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

85% 100%

Inspection/

Reporting

Quarterly QASP Section

5.4

6.3, 6.4, 6.5, 6.6, 6.7

Infrastructure

Request

Response

ITIOD maintains a service catalog request and tracking system on SharePoint. This catalog, distinct from GetIT, provides access to common services such as new server builds, and software packaging requests. New server builds include only virtual servers (Windows and RedHat) and are not expected to be more than 20 per month. The SLA will be scored as ½ point for servers and ½ point for data fixes. The

AQL and TQL thresholds are based on the specific delivery deadlines defined in the infrastructure service catalog (2 business weeks for server builds, 2 business days for

90% 100%

Inspection/

Reporting

Quarterly See attachment A for detail data fixes). Builds assigned to non ITIOSS contractors were excluded

6.3, 6.4, 6.5, 6.6, 6.7

Time to

Address

Request for

Information(R

FI) covered by

CPAF CLIN

This SLA applies to IM tickets categorized as requests for information and is scored on the timeliness of RFI tickets being resolved/closed as recorded within

ServiceNow. The ticket prioritization is based on the status of the requestor (VIP or non-VIP)

85% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

Appe ndix

B

Deliv erabl es

CDRL quality and timeliness

Document quality control performance data are manually tracked based on timely delivery of accepted items controlled under the Contract Deliverable Requirement List

(CDRL)

98% 100%

Inspection/

Reporting

Quarterly PWS- Appendix

B - Deliverables

6.3 Backup

Success Rate

This SLA measures the backup success rate in all environments

98% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.6.2 Security

Incident

Response

This metric is calculated as the time from when a security event is declared a security incident to notification. The incident identification is documented in the Service

Desk ticket recording the suspicious security event, and ITIOD leadership is normally notified through email as an official communication.

98% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.5 Major Incident

Process

Adherence

This SLA measures accurate and efficient application of the Major Incident

Management Process. This SLA is scored through an after-action interview and survey with the ITIOD Federal Incident Manager

70% 100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.5.5 Major Incident

caused by

ITIOSS

Contractor (1st

Month of the

Quarter)

This SLA measures whether contractor is responsible for a Major Incident in the 1st month of a quarter. This is determined by the ITIOD Federal Incident Manager

None

(Zero)

100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.5.5 Major Incident

caused by

ITIOSS

Contractor (2nd

Month of the

Quarter)

This SLA measures whether contractor is responsible for a Major Incident in the 2nd month of a quarter. This is determined by the ITIOD Federal Incident Manager

None

(Zero)

100%

Inspection/

Reporting

Quarterly See attachment A for detail

6.5.5 Major Incident

caused by

ITIOSS

Contractor (3rd

Month of the

Quarter)

This SLA measures whether contractor is responsible for a Major Incident in the 3rd month of a quarter. This is determined by the ITIOD Federal Incident Manager

None

(Zero

100%

Inspection/

Reporting

Effected

Quarters

See attachment A for detail

6.5.9 Asset

Management

Annual

Inventory

Report

Measures the timely delivery of the final

Asset Management Inventory Report

15-Oct Report

Deliverable

Annual PWS-Section

6.5.9

6.5.9 Asset

Management

Annual

Inventory

Measures the percentage of scanned inventory relative to all IT assets contained within the Asset Management Database

96% 100%

Inspection/

Reporting

Annual PWS-Section

6.5.9

3.2 FFP Performance Metrics

FFP CLINs Services scope are covered by the PWS’ End-User Services Section (IT Service

Desk & Site Support), IT Service Catalog Support Section and Disaster Recovery/Continuity of

Operations Planning (COOP) and Testing Section

• Following KPI metrics shall be included in the contractor self-assessment but they excluded from Cost Plus Award Fee calculation table and the process.

• Some of the metrics are shared between FFP and CPAF CLINs since they are performed by resources under both CLINs. For example some of the escalated priority incidents are resolved both Site Support(FFP) or other platform teams.

PWS

Refer ence

Service Level

Agreement Metric

Title

Service Level Agreement Metric

Description

AQL Surveillance

Method

Frequency Additional Notes

6.2.1 Tier 1 customer survey This metric captures the overall customer experience performance of Tier-1 Service Desk staff in resolving issues for ITIOD customers. When the incident is closed, an email is delivered which includes a link to a customer service survey. Customers are invited but not required to participate in the survey. Surveys submitted by contract team members are accepted for service improvement reasons but are excluded from the SLA performance metric. Metric is percentage of responses with overall rating of 4 or 5 on 5-point scale.

90% 100%

Responses to Survey

Monthly &

Quarterly

QASP Section 5.4

6.2.2 Tier 2+ customer

survey covered by FFP

CLIN

This metric captures the overall customer satisfaction in addressing incidents escalated to ITIOSS Tier 2 (FFP services).

When the incident is closed, an email is delivered which includes a link to a customer service survey. Customers are invited but not required to participate in the survey. Surveys submitted by contract team members are accepted for service improvement reasons but are excluded from the SLA performance metric. Metric is percentage of responses with overall rating of 4 or 5 on 5-point scale.

90% 100% Responses to

Survey

Monthly & Quarterly

QASP Section 5.4

6.2.1

6.2.2

Time to resolve incident (priority 1) covered by FFP CLIN

Report data is collected from the

ServiceNow application. Scoring is based on the timeliness of priority 1 incident tickets being closed/resolved as recorded within ServiceNow.

Violations are reviewed both for process improvement and for the identification of exception candidates.

Adjustments are requested for tickets which are out of the scope of ITIOD

(service failures based on events outside of ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

90% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.5

6.2.1

6.2.2

Time to resolve incident (priority 2) covered by FFP CLIN

Report data is collected from the

ServiceNow application. Report data is collected from the ServiceNow application. Scoring is based on the timeliness of priority 2 incident tickets being closed/resolved as recorded within ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates. Adjustments are requested for tickets which are out of the scope of ITIOD (service failures based on events outside of ITIOD e.g.

failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

93% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.6

6.2.1

6.2.2

Time to resolve incident (priority 3) covered by FFP CLIN

Report data is collected from the

ServiceNow application. Report data is collected from the ServiceNow application. Scoring is based on the timeliness of priority 3 incident tickets being closed/resolved as recorded within ServiceNow. Violations are reviewed both for process improvement and for the identification of exception candidates. Adjustments are requested for tickets which are out of the scope of ITIOD (service failures based on events outside of ITIOD e.g.

failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

93% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.7

6.2.1

6.2.2

Time to resolve incident (priority 4) covered by FFP CLIN

Report data is collected from the

ServiceNow application. Scoring is based on the timeliness of priority 4 incident tickets being closed/resolved as recorded within ServiceNow.

Violations are reviewed both for process improvement and for the identification of exception candidates.

Adjustments are requested for tickets which are out of the scope of ITIOD

(service failures based on events outside of ITIOD e.g. failure of an outside organization’s web-site), or for tickets that are withdrawn by the customer.

94% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.8

6.2.1 Time to Answer This metric captures the average time for the IT Service Desk to answer an incoming call utilizing the call tracking data from the

PBGC phone system. Time periods of unusually high call volume may, at the discretion of ITIOSS COR or ITCOS

Division Manager or designee’s, be removed from the Average Speed to

Answer calculation.

<=40s

100% Inspection/R eporting

Monthly & Quarterly automatic call distributor (ACD) report

6.2.1 Abandon Rate This SLA is calculated as abandoned calls

divided by total inbound calls. Time periods of unusually high call volume may, at the discretion of ITIOSS COR, ITCOS Division Manager or a designee, be removed from SLA calculation

<=6%

100%

Inspection/R eporting

Monthly &

Quarterly automatic call distributor (ACD) report

6.2.1

6.2.2

Time to Address

Request for

Information covered by FFP CLIN

This SLA applies to IM tickets categorized as requests for information and is scored on the timeliness of RFI tickets being resolved/closed as recorded within ServiceNow. The ticket prioritization is based on the status of the requestor (VIP or non-VIP)

93% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.8

6.2.1 First Call

Resolution/Tier 1

Resolution

This tracks the number of phone tickets resolved by Service Desk out of the total number of phone tickets opened by Service

Desk. ServiceNow reporting provides the metric.

68% 100% Inspection/R eporting

Monthly & Quarterly

QASP Section 5.8

6.2.1 Time to Resolve

Service Desk Ticket

Tracks the number of phone tickets created and resolved by Service Desk within 2 hours or less out of the total number of phone tickets created and resolved by

Service Desk. ServiceNow reporting provides the metric.

96% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.8

6.2.1 Tier 1 Assignment

Quality

This SLA applies to the ticket escalation quality of service given to customers by the

Service Desk. Quality is assessed as the accuracy of ticket assignment to a resolver group. ServiceNow reporting provides the metric as a percentage of tickets resolved by the resolver queue initially assigned by the

Service Desk without additional re-assignment.

86% 100%

Inspection/R eporting

Monthly &

Quarterly

QASP Section 5.8

6.9.1 COOP Exercise

Failover

This SLA measures the percentage of failover tasks completed successfully during a COOP exercise

98% 100%

Inspection/

After Action Report(AAR

Twice a year

PWS 6.9.1

6.9.1 COOP Exercise

Failback

This SLA measures the percentage of failback tasks completed successfully during a COOP exercise

98% 100%

Inspection/ After Action

Report(AAR

Twice a year

PWS 6.9.1

6.9.1 COOP AAR Action

Item Completion

This metric is used to report progress toward addressing problems detected during

Continuity of Operations (COOP) exercises.

As an award fee performance metric, it is intended to encourage the contractor to manage resources to promptly remediate problems with the failover and failback continuity of operations procedures and/or infrastructure.

5% COOP

Corrective

Action Plan

Twice a year

PWS 6.9.1

Service Level Agreements (SLAs)

The SLAs represent the Government’s proposed performance measurement structure.

Offerors are strongly encouraged to recommend changes to the SLA that improve PBGC’s ability to achieve its program and service objectives. During transition (see § 5.2 of O&M and

DM&E Task Order), the Contractor will finalize and formally adopt contract SLAs. As part of that process, PBGC and the Contractor will analyze SLAs to ensure that they are fair and reasonable for the ensuing transfer of control. Both parties must review changes, and upon acceptance, the changes will be reflected by modification of the contract by the end of the transition period and prior to the Contractor taking full control of services. During this period, SLAs will be tracked, monitored, adjusted, and reported against targets as part of the monthly review by the Performance Review Board and may be adjusted through bilateral agreement.

The Contractor shall be evaluated on the proposed and accepted SLAs that apply specifically to transition activities during the first two months of award.1 The Contractor will be evaluated in accordance with contract SLAs beginning the month 3 of the contract. The first performance period under which the Contractor will be evaluated under the contract SLAs with therefore be four months. After that the contractor will be evaluated on a quarterly basis.

Remedy Plan for Underperformance

When failure to meet an AQL occurs two (2) or more consecutive months with the same metric, the Contractor shall propose a remedy plan. This plan shall be approved by the

Government and executed by the Contractor to bring the latter’s performance up to AQLs as defined by the SLAs.

4. INCIDENT ESCALATION SYSTEM

PBGC uses a tiered incident escalation system (Tiers 1, 2, 3 and 4) to classify the level of knowledge, skill, and access privileges typically required of IT support to resolve an incident.

All incidents are first addressed at the Tier 1 level (first-call resolution). In some circumstances, Tier 1 is insufficient to resolve an incident and will require escalation to a higher-level tier such as the platform teams for resolution.

Figure A – Tier Level Descriptions

Level Definition

Tier 1 An incident or problem is transferred to Specialist group(s) that may be part of the

Service Desk. Requestors may require desk-side support with Windows desktop functions, COTs/Custom applications, desktop repair or replacement, and network troubleshooting

Tier 2 An incident or problem is transferred to a Specialist group(s) that is part of End-User

Services or Data Center Operations. Requestors may require assistance with complex email, database, communications, and/or infrastructure issues.

Tier 3 An incident or problem is transferred to a Specialist group(s) that is part of a Business

Application Support Team. Requestors may require assistance with a custom developed Application

Tier 4 An incident or problem is transferred to a Specialist group(s) that is part of external vendor support. Requestors may require assistance with a complex hardware/software issue that is covered

1 See § 5.2 of Task Order 02 regarding requirements during transition and related provisions of the Award Fee Determination Plan.

5. PRIORITIZING SUPPORT SERVICES BY IMPACT AND URGENCY

Incidents are prioritized in accordance with impact and urgency ratings assigned to the end-user’s incident upon contacting the Service Desk. Each incident is assigned a predefined triage level based on the Priority Levels, and priority level incidents are reported monthly to the

Performance Review Board. Priority levels are determined based the determination of the impact of the incident on PBGC as well as the urgency as determined by the table below.

Using the number that results from the above matrix, the incidents are prioritized in accordance with the following Service Priority Levels. The incidents are also assigned a time-to-resolve as well as an acceptable quality level (AQL). PBGC measures the time-to-resolve from initial contact, response, and final resolution of an incident. Time-to-resolve may end when the user has been returned to a state of normal operation but has not responded to three documented contact attempts to ascertain the user’s full satisfaction. All instances of user non-response will be escalated to federal staff for resolution.

Figure B – Service Priority Levels

Priority Prioritization Guidance Time-to-Resolve &

AQL/TQL

• An unplanned outage of a mission-essential production application or service (as documented by “COOP

Essential” or “COOP Critical” within mAppIT, excluding desktop applications)

• A primary GFE desktop, laptop or mobile application or infrastructure outage impacting a VIP’s ability to perform work. Identification of VIP tickets shall rely upon the

‘requested for’ field in SM9.

• A desktop application outage impacting 50 or more users.

3 business hours.

AQL=85%

TQL=90%.

2 • An unplanned outage of a non-critical production business application/service (as documented by “Non-Essential” within mAppIT, excluding desktop applications).

• A desktop application outage impacting 2-49 users or impacting a single user such that this user is completely unable to work in any capacity using IT systems because of the outage.

• A degradation in infrastructure or application performance or fault tolerance of a critical/essential production business application/service.

• A non-Production (CDE-T, CDE-I, ITC) service outage impacting many users and/or seriously imperiling a scheduled software release.

1 business day (12 business hours).

TQL=93%

3 • A primary GFE desktop, laptop, or mobile application or infrastructure outage impacting a nonVIP’s ability to perform work.

• A degradation in infrastructure or application performance or fault tolerance of a non-essential production business application/service. C) A non-Production (CDE-T, CDE-I, ITC) service/application outage impacting several users

2 business days (24 business

TQL=93%

4 • A degradation or outage of a secondary production desktop, laptop, or mobile peripheral (e.g. direct attached printer/scanner, a 2nd monitor, etc.)

• A non-Production (CDE-T, CDE-I, ITC) service/application outage impacting a single user

• A non-Production (CDE-T, CDE-I, ITC) service/application degradation.

4 business days (48 business

TQL=94%

ATTACHMENT A

SLA Detailed Descriptions:

CS-1 Tier 1 Customer Survey

This metric captures the overall customer experience performance of Tier-1 Service Desk staff in resolving issues for PBGC users. When the incident is closed, an email is delivered which includes a link to a customer service survey. Users are invited but not required to participate in the survey. Surveys submitted by ITIOSS contractor team members are accepted for service improvement reasons but are excluded from the SLA performance metric. The current

Acceptable Quality Level (AQL) is 70% and Target Quality Level (TQL) is 80% of the overall responses being “Good” (4) or “Excellent” (5).

CS-2 Tier 2+ Customer Survey

This metric captures the overall customer satisfaction in addressing incidents escalated to

ITIOSS Tier 2. When the incident is closed, an email is delivered which includes a link to a customer service survey. Customers are invited but not required to participate in the survey.

Surveys submitted by ITIOSS contract team members are accepted for service improvement reasons but are excluded from the SLA performance metric. The current Acceptable Quality

Level (AQL) is 70% and Target Quality Level (TQL) is 80% of the overall responses being

“Good” (4) or “Excellent” (5)

Below is an example of the CS-1 and CS-2 survey questionnaire:

CS-5 COR Opinion/ITIOD Stakeholders Survey

This metric provides the COR an opportunity to evaluate the contractor’s performance during and directly influence award fee determination. The COR determines the final score based on survey inputs and other factors. The result is calculated by the average of all responses being

“Unacceptable”, “Poor”, “Acceptable”, “Good” or “Excellent” (scores 1,2, 3, 4 or 5). The

Accepted Quality Level(AQL) is the average of all responses being “Acceptable” (average score of 3) and the target quality level (TQL) is the average of all responses being “Good” or

“Excellent” (scores 4 or 5).

Below is the survey that is sent to ITIOD stakeholders via a SharePoint page link on a monthly basis.

TR-10: Infrastructure Request Response

ITIOD maintains an Infrastructure service catalog request and tracking system on SharePoint.

This catalog, distinct from GetIT (PBGC enterprise service catalog), provides access to common services such as new server builds, and software packaging requests. Currently "Server: New

Server" and "Oracle: Data Fix" catalog are measured. New server builds include only virtual servers (Windows and RedHat) and are not expected to be more than 20 per month. The SLA will be scored as ½ point for servers and ½ point for data fixes. The AQL and TQL thresholds are based on the specific delivery deadlines defined in the infrastructure service catalog (2 business weeks for server builds, 2 business days for data fixes). Builds assigned to non ITIOSS contractors were excluded.

TR-13 Time to Address Request for Information(RFI)

RFI are the IM tickets (In ServiceNow) that categorized as request for information in

ServiceNow. TR-13 measures the timeliness in which RFI's are resolved. RFI's are primarily categorized as Priority 3 (VIP) or Priority 4(non-VIP) Tickets. P3 tickets must be resolved in 4 business days and P4 tickets must be resolved in 8 business days. The SLA percentage is calculated by taking the numbers of RFI's completed on time over the total number of RFIs

TR-14 First Call Resolution/Tier 1 Resolution

TR-14 measures how many phone tickets are being resolved by the Service Desk. It calculated by taking the number of phone tickets resolved by the service desk over the total number of phone tickets resolved.

TR-15 Time to Resolve Service Desk Ticket

TR-15 measures the resolution timeliness of phone tickets resolved by the service desk. The

TQL for resolution time for phone tickets resolved by the service desk two hours. The SLA is calculated by taking the number of on time resolved phone tickets over the total number of phone tickets by the service desk.

TR-16 Tier 1 Assignment Quality

TR-16 measures the amount reassignments a phone ticket has once it has been escalated to another team by the service desk. The TQL for this SLA is 0 reassignments. The SLA is calculated by taking the number of tickets with 0 reassignments over the total number of tickets.

SA-4 Backup Success Rate

SA-4 measures the daily backup of PROD activities. It is calculated by taking the number of successful backup jobs over the total number of backup jobs run.

AR-2 Security Incident Response

This metric is calculated as the time from when an event is declared an incident—after the identification phase of the Incident Handling process—until the Enterprise Cybersecurity

Division (ECD) is notified that a security incident has occurred. The incident identification is documented in the Service Desk ticket recording the suspicious security event, and ECD is normally notified through email as an official communication. This SLA is calculated by taking the number of incidents responded within one hour divided by the total security incidents during the period.

AR-7b Major Incident Process Adherence

This SLA provides an incentive for the accurate and efficient application of the Major Incident

Management Process. The SLA will take the form of an after-action interview or survey with the

ITIOD Federal Incident Manager to characterize performance based on several categories explained in the SLA report/checklist below. AR-7b measures the quality of response to a Major

Incident. Major Incidents are scored using the Major Incident Scorecard that totals 51 points. The

SLA score is calculated by taking the number of points received on the scorecard by the federal incident manager over the total number of points available.

Major Incident Checklist/Scorecard (example score)

AR-7 Major Incident

Scorecard

Incident:

Incident date:

Caused by

ITIOSS Contractor? (

AR-7c,d,e) Yes/No Monthly=1/3

Quarterly=1

Description:

Scorecard Item

Contract or

Assessm ent

Federal Incident Manager Assessm ent

Associated Points

Points Recei ved

Criteria

Time determine whether incident is deemed “major”

Score of 1 to 5 based on subjective criteria

Assessed on whether determination of major incident was made within an appropriate timeframe using either the “Known Major

Incident” List or SME determination

Timely notification of

Federal Incident

Manager

Score of 1 to 5 based on subjective criteria

Assessed on whether FIM notified and provided clear, actionable information within an appropriate timeframe

Time to create IM ticket

Score of 1 to 5 based on subjective criteria

Assessed on whether ticket was generated at start of Incident Process

Timely notification of

PBGC

stakeholders

Score of 1 to 5 based on subjective criteria

Assessed on whether a SOR is created and

Advisory (optional) is sent out within an appropriate timeframe;

Assesses use of out-of-band communications in cases where primary communications are unlikely to be effective (e.g. email outage)

Effectiveness of troubleshooting

, and determination o f work around if applicable

Score of 1 to 5 based on subjective criteria

Assessed based on the effectiveness of the

O&M team’s troubleshooting efforts and utilization of all resources including vendors

Effectiveness and timeliness of ongoing communication s to FIM and Stakeholders throughout incident

Score of 1 to 5 based on subjective criteria

Assessed based on the effectiveness/timelines s of ongoing communications to FIM and Stakeholders throughout the incident and at resolution

(Number 6 in Major Incident

Process Workflow)

Time to restore service, or implement workaround

Score of 1 to 5 based on subjective criteria

Assessed based on time to resolve the incident or implement the workaround

Documentation of incident diagnosis and resolution changes in approved emergency

RFC

Score of 1 to 5 based on subjective criteria

Assessed on accurate diagnosis of the incident, and whether the Emergency RFC

Procedures were followed if applicable

Effectiveness of end-user communication

Score of 1 to 5 based on subjective criteria

Assessed based on the effectiveness of our end-user communications

Timeliness of

AAR

0 or 1 point 1

Assessed based on whether the AAR is delivered within one week of when it was requested

Quality of AAR

Score of 1 to 5 based on subjective criteria

Assessed based on the quality of the AAR including an analysis of the root cause of the incident, if applicable

Summary Score

Total Points Earned

Total Points Possible

Percentage

Secure Configuration Baseline (SCB) SLAs

Secure Configuration Baseline (SCB) SLAs

1. AR-8 SCB Non-Compliant Individual CI Remediation Timeliness:

This metric measures the timely remediation when an individual configuration item (CI), e.g.

workstation server, database, device, etc. with an approved security configuration baseline checklist falls below the minimal CI baseline compliance threshold defined in approved baseline documents and noted in the table below. PBGC’s custom developed Continuous

Automated Compliance Monitoring (CACM) solution, will record a non-compliance exception and a notification to the Service Desk on the first Monday that follows when an individual CI has fallen below the minimal CI baseline compliance threshold. Contractor is responsible for remediation within 30 days of the first detection by correcting the configuration issue, removing the CI from network, or documenting a risk acceptance. The current Acceptable Quality Level

(AQL) is remediation of 99% and Targeted Quality Level (TQL) is remediation of 100% of all detected non-compliance exceptions initially detected by CACM during the POP end - 30 days.

Specifics about remediation are as follows:

1. Correcting the configuration issue and bringing the individual CI abovethe minimalCI baseline compliance threshold

2. Submit Risk Acceptance to ISSO after obtaining concurrence from federal POC that the specific CI and setting is a candidate for Risk Acceptance

3. Remove the device from network (O&M staff is empowered to remove it from the network when they have provided 3 notices within 30 days to the owner of the CI and they have not responded to the requests)

When an issue is resolved, CACM will automatically record the exception has been resolved including the date resolution was detected. Individual CIs that have not been on the network for more than 30 days are not scored. Reports will be generated from CACM on a monthly basis for quarter-to-date and the final quarterly report will be used to calculate the SLA score. Below are the fields that will be included on the report:

1. ComputerName

2. BaselineName

3. FirstReportDate

4. TargetResolutionDate

5. ResolutionStatus

6. ResolutionDetectionDate

7. InAdvanceofTargetResolutionDate

The SLA score will be calculated as follows:

Count of InAdvanceofTargetResolutionDate (yes) / (Count of InAdvanceofTargetResolutionDate (yes) + Count of InAdvanceofTargetResolutionDate (no))

Overall SLA weight is 2 points (~4% of all SLAs). Overall earned SLA score = SLA weight

(2pts) * Total earned score

BaselineName MinimumIndividualCIThreshold

PBGC WIN10 FY-2016 90

PBGC Customized Win2012

MS

PBGC RHEL6 FY-2016 90

PBGC RHEL7 FY-2017 90

PBGC Customized Win2008R2

MS

PBGC Customized Win2008

R2 DC

PBGC SOL11 FY-2017 80

PBGC Customized WIN2016 90

Sample Report and calculation:

Computer Name

BaselineN ame

FirstReport Date

TargetResoluti onDate

ResolutionS tatus

ResolutionDetecti onDate

InAdvanceofTargetReso lutionDate

WinServer

PBGC

Customize d Win2012

MS

03/02/201

04/01/2019 Resolved 03/05/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/02/201

04/01/2019 Resolved 04/07/2019 No

WinLaptop

PBGC

WIN10

FY-2016

03/02/201

04/01/2019 Resolved 03/07/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/02/201

04/01/2019 Resolved 03/07/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/02/201

04/01/2019 Resolved 04/07/2019 No

WinLaptop

PBGC

WIN10

FY-2016

03/02/201

04/01/2019 Resolved 03/07/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/05/201

04/04/2019 Resolved 03/13/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/05/201

04/04/2019 Resolved 03/13/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/05/201

04/04/2019 Resolved 03/06/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/08/201

04/07/2019 Resolved 03/25/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/08/201

04/07/2019 Resolved 03/11/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/08/201

04/07/2019 Resolved 03/13/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/08/201

04/07/2019 Resolved 03/13/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/11/201

04/10/2019 Resolved 03/22/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/11/201

04/10/2019 Resolved 03/12/2019 Yes

RHELServe r1

PBGC

RHEL7 FY-

03/21/201

04/20/2019 Resolved 03/23/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/21/201

04/20/2019 Resolved 03/23/2019 Yes

WinLaptop

PBGC

WIN10

FY-2016

03/21/201

04/20/2019 Resolved 03/25/2019 Yes

Count of InAdvanceofTargetResolutionDate (yes) / (Count of InAdvanceofTargetResolutionDate (yes) + Count of InAdvanceofTargetResolutionDate (no))

=> 16 / (18 +0) (88.9%).

100.0% would then be multiplied by the AR-8 SCB Non-Compliant Individual CI Remediation

Timeliness SLA weight: 88.9% x 2 = 1.78 points.

2. AR-9 SCB Aggregate Compliance SLA:

This metric measures the overall Secure Configuration Baseline (SCB) compliance for all configuration items (CIs), e.g. workstation server, database, device, etc. with an approved security configuration baseline checklist in alignment with PBGC’s acceptable and target aggregate CI baseline compliance thresholds as outlined in the table that follows. This table outlines the relative weights for each baseline, which generally gives baselines with more Cis more weight, as well as the AQL and TQL levels for each baseline. PBGC’s custom developed

Continuous Automated Compliance Monitoring (CACM) solution, records daily aggregate compliance levels for each baseline and these are averaged over the period of performance to calculate the SLA score. Additionally, a non-compliance exception is recorded in CACM when any aggregate baseline score has fallen below PBGC’s acceptable aggregate CI baseline compliance threshold to provide additional visibility. Contractor is responsible for ensuring aggregate baseline scores are maintained above acceptable and ideally at target levels by correcting configuration issues, removing CIs from the network when they fall below the minimal CI baseline compliance threshold and cannot be corrected, and/or documenting a risk acceptance.

Baseline

Name

AcceptableAg gregateThres hold

TargetAggr egateThresh old

SLA

Weig ht

SLA_AQL_A

ggregateThres hold

SLA_TQL_A

ggregateThres hold

PBGC

WIN10

FY-2016

95 99 4 95 98.75

PBGC

Customi zed Win2

012 MS

98 99 3 98 98.75

PBGC

RHEL6

FY-2016

96 99 3 96 98.75

PBGC

RHEL7

FY-2017

96 99 2 96 98.75

PBGC

Customi zed

Win2008

R2 MS

96 99 2 96 98.75

PBGC

Customi zed

Win2008

R2 DC

96 99 1 96 98.75

PBGC

SOL11

FY-2017

90 99 1 90 98.75

PBGC

Customi zed

WIN201

98 99 1 98 98.75

Reports will be generated from CACM on a monthly basis for quarter-to-date and the final quarterly report will be used to calculate the SLA score. Below are the fields that will be included on the report:

1. BaselineName

2. SLAWeight

3. SLA_AQL_AggregateThreshold

4. SLA_TQL_AggregateThreshold

5. AverageAggregateBaselineScore

6. WeightedBaselineSLAScore

This is calculated by multiplying the Baseline’s SLAWeight times:

1 if the AverageAggregateBaselineScore >= SLA_TQL_AggregateThreshold

0 if the AverageAggregateBaselineScore >= SLA_AQL_AggregateThreshold < SLA_TQL_Aggrega teThreshold

-1 if the AverageAggregateBaselineScore < SLA_AQL_AggregateThreshold

The SLA score will be calculated as follows:

Sum of WeightedBaselineSLAScore / Sum of SLAWeight

Overall SLA weight is 2 points (~4% of all SLAs). Overall earned SLA score = SLA weight

(2pts) * Total earned score

Sample Report and calculation:

Baseli neNa me

SLA

Weig ht

SLA_AQL_Ag gregateThresh old

SLA_TQL_Ag gregateThresh old

AverageAggre gateBaselineSc ore

WeightedBa selineSLASc ore

PBGC

WIN1

0 FY-

4 95 98.75 98.9 4

PBGC

Custo

3 98 98.75 97.6 -3 mized

Win20

MS

PBGC

RHEL

6 FY-

3 96 98.75 99.0 3

PBGC

RHEL

7 FY-

2 96 98.75 97.4 0

PBGC

Custo mized

Win20

08R2

MS

2 96 98.75 99.0 2

PBGC

Custo mized

Win20

08 R2

DC

1 96 98.75 100.0 1

PBGC

SOL1

1 FY-

1 90 98.75 99.0 1

PBGC

Custo mized

WIN2

1 98 98.75 98.6 0

Sum of WeightedBaselineSLAScore / Sum of SLAWeight => 8 / 17 (47.1%).

47.1% would then be multiplied by the AR-9 SCB Aggregate Compliance SLA weight: 47.1% x

2 = .942 points.

Patch and Vulnerability Management Process (PVMG)

Metric Title Metric Description

Critical

Vulnerabilities

Remediated

Critical vulnerabilities…