RFI Atch 4 - QASP_DRAFT.pdf

ESS Next QASP Page 1 of 56 Pages

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

1. Contract or Task Order Title:

ESS Next

2. Purpose:

QASP Task numbers are aligned with PWS paragraph numbers.

• PWS Section 3: Task 3.1 – Program Management

• PWS Section 3: Task 3.2 – The METIC

• PWS Section 3: Task 3.3 – Data Center as a Service (DCaas)

• PWS Section 3: Task 3.4 – Infrastructure as a Service (IaaS)

• PWS Section 3: Task 3.5 – Desktop as a Service (DaaS)

• PWS Section 3: Task 3.6 – Platform as a Service (PaaS)

• PWS Section 3: Task 3.7 – Software as a Service (SaaS)

• PWS Section 3: Task 3.8 – Containers as a Service (CaaS)

• PWS Section 3: Task 3.9 – General Services (applicable to all other services)

• PWS Section 3: Task 3.10 – Cybersecurity & Policy Adherence (applicable to all other services)

3. Roles and Responsibilities:

The following personnel shall oversee and coordinate surveillance activities:

Program/Project Manager (PM) – The PM provides program oversight and supports the Contracting Officer Representative’s (COR’s) performance assessment activities. The assigned PM for this effort is:

Contracting Officer (KO) – The KO shall ensure performance of all necessary actions for effective con-tracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Finally, the KO shall determine the final assessment of the con-tractor’s performance. The assigned Contracting Officer for this effort is:

Contracting Officer’s Representative (COR) – The COR is responsible for providing continuous tech-nical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/sur-veillance process. The COR shall keep a Quality Assurance file that accurately documents the contrac-tor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance prob-lems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor per-formance. The COR is not empowered to make any contractual commitments or to authorize any con-tractual change on the Government’s behalf. The assigned COR for this effort is:

4. Primary Method of Surveillance:

A mix of random sampling, 100% inspection, and periodic surveillance will be utilized to assess delivera-bles and outcomes under this effort. Inspection of deliverables will include reviews and assessments by the Government.

Page 2 of 56 Pages

Surveillance Matrix

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The Acceptable Quality Level (AQL) performance Standard Table should be used to form the foundation of the COR’s inspection checklist.

5. Performance Standards:

Performance Objective

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Performance Obj #1 Provide an on-demand intermediate, digital, secure storage for test evidence and files from applications in the Lab (or in cloud) so testers can upload to test tool at later date

PWS

3.2

PRS# 1

Availability: 24x7x365

Incident Response: Work be-gins within 5 business days for Priority 4 incidents, 1 business day for Priority 3 in-cidents, 1hr for Priority 2 inci-dents, 30 minutes for Priority 1 incidents

Imaging, Security Updates, and Patching: 24hrs of client approval for critical, 72hrs for important, 30 days for moder-ate, 60 days for low priority updates on laptops and infra-structure in data centers and cloud environments

Restoration: Restore functionality of systems within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for priority 3 incidents, 4hrs for priority 1 incidents

• Availability: 99.9% of time

• Incident Response:

90% of the time for Priority 4 incidents;

90% of the time for Priority 3 incidents;

95% of the time for Priority 2 incidents;

95% of the time for Priority 1 incidents

• Imaging, Security Updates, and Patching: 99% of the time for critical, 99% of time for important, 95% of the time for moderate, 95% of the time for low priority updates

Restoration: 90% of the time for Priority 4 inci-dents; 90% of the time for Priority 3 incidents;

95% of the time for Prior-ity 2 incidents; 95% of the time for Priority 1 in-cidents

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates.

Performance Obj. #2 PWS 3.2

PRS #2

Availability: 24x7x365

• Availability: 99.9% of

Random Sampling:

Page 3 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Provide network capa-bility to allow users to connect to .com and .mil in the METIC and DDC and connect to Wi-Fi in METIC

Incident Response: Work be-gins within 5 business days for Priority 4 incidents, 1 business day for Priority 3 in-cidents, 1hr for Priority 2 inci-dents, 30 minutes for Priority 1 incidents

• Restoration: Restore functionality of systems within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for priority 3 incidents, 4hrs for priority 1 incidents

Priority 4 incidents;

90% of the time for Priority 3 incidents;

95% of the time for Priority 2 incidents;

• Restoration: 90% of the time for Priority 4 incidents; 90% of the time for Priority 3 incidents; 95% of the time for Priority 2 incidents; 95% of the time for Priority 1 incidents

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #3 Allow users easy and timely remote access to servers, storage, and networking resources through a Wide Area Network (WAN) or similar communication structure

PWS

3.3

PRS #3

Availability: 24x7x365

Incident Response: Work be-gins within 5 business days for Priority 4 incidents, 1 business day for Priority 3 in-cidents, 1hr for Priority 2 inci-dents, 30 minutes for Priority 1 incidents

• Time to provision: <=1 day

Priority 4 incidents;

90% of the time for Priority 3 incidents;

95% of the time for Priority 2 incidents;

• Time to provision:

95% of the time monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #4 Develop a “platform management suite” with standardized suite of tools (tools are standardized via gov-ernance process) to provide an avenue to product managed ser-vices

PWS

3.6

PRS #4

Availability: 24x7x365

Incident Response: Work be-gins within 5 business days for Priority 4 incidents, 1 business day for Priority 3 in-cidents, 1hr for Priority 2 inci-dents, 30 minutes for Priority 1 incidents

AQL performance is achieved when:

• Availability: 100% of the time

Priority 4 incidents;

90% of the time for Priority 3 incidents;

95% of the time for

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 4 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Restoration: Restore function-ality of systems within 10 business days for Priority 4 in-cidents, 3 business days for Priority 3 incidents, 8hrs for priority 3 incidents, 4hrs for priority 1 incidents

Priority 2 incidents;

incidents; 90% of the time for Priority 3 incidents; 95% of the time for Priority 2 incidents; 95% of the time for Priority 1 incidents

Performance Obj. #5 Maintain the ATO for TI.mil pre-prod envi-ronments and the site accreditation for the METIC & DDC data centers

PRS #5

• ATO Compliance:

Maintain TI.mil, METIC, & DDC compliance with annual ATO re-certifications

• ATO compliance is achieved 100% of the time

Periodic Surveillance:

Compare required reports submitted against reports required for ATO re-certi-fication

Performance Obj. #6 Develop, update, store, and maintain architec-ture documentation that describes the various “as a service” topology and services

3.1

PRS #6

Architecture Diagrams: Up-dates completed within 30 days of changes implemented

• Architecture Updates:

Documentation updated monthly

• Architecture Updates achieved 90% of the monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates.

Performance Obj. #7 Manage chip/devices catalog, remote man-agement, and monitor-ing/alerting tools

PRS #7

Catalog Accuracy: Catalog is accurate

• Catalog Updates: Orders for changes to catalogue received by noon should be updated same day

• Catalog Accuracy:

100% of the time

• Catalog Updates:

Performance Obj. #8 Partner with the PEO DHMS CM team to re-view and manage Change Requests sub-mitted to the PEO CCB

3.9

PRS #8

Change Request (CR) Process Time: CRs processed within 10 days of request

• Change Request Report:

CR process report provided every month

• CR Process Time:

90% of the time

• CR Report: 90% of monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 5 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Performance Obj. #9 Support DHA imaging, security updates, and patching on the infra-structure and laptops in METIC, DDC data centers, and in cloud

3.5

PRS #9

DHA Image Installation: Lap-tops and Infrastructure in METIC, DDC, Cloud will have DHA image installed and maintained within 3 business days of receipt

• Imaging, Security Updates, and Patching: Installation of updated image, security updates, and patching will be within 24hrs of client approval for critical, 72hrs for important, 30 days for moderate, 60 days for low priority updates on laptops and infrastructure in METIC, DDC, and Cloud

• DHA Image Installation: 90% of the time

• Imaging, Security Updates, and Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and 95% of the time for low priority updates

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #10 Provide support for specialized projects, in-cluding facilitating POCs for new tools/technologies.

Specialized projects in-clude:

• Tech Refresh - re-placement of hardware devices (laptops, medi-cal devices)

• Major OS upgrades (desktops and servers)

PRS #10

DHA Image Installation: Lap-tops and infrastructure in data centers and cloud environ-ments will have DHA image installed and maintained within 3 business days of re-ceipt

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Time to provision: Within agreed target time

• DHA Image Installation: 90% of

Patching: performance standard achieved 99% of the time for critical, 99% of the time for critical, 95% of the time for moderate, and 95% of the time for low priority updates

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #11 Verify, manage, and track physical security for designated data centers

3.3

PRS #11

• Facility Response: 5 minutes to respond to security issues and alarms in the facility

• Facility Response:

Vendor provides physical access logs, physical sur-veillance video, and docu-mented issues as re-quested; COR reviews and validates

Page 6 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Performance Obj. #12 Maintain and track desktop licensing for VDIs

PWS

3.5

PRS #12

• Effective License Position

Reporting: 100% accurate

• Effective License Position Reporting:

Random Sampling monthly metrics into cen-tralized reporting dash-board by leveraging in-cluding calculations and supporting data; COR re-views and validates.

Performance Obj. #13 Provide a seamless self-service mechanism for scheduling events in the METIC and doc-ument/distribute SOP for stakeholders on the process

PRS #13

Event Request: schedule and deliver events in METIC and DDC within 48 hours of re-quest (including weekend sup-port)

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1 hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8 hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents

• Event Request: 90% of the time

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #14 Conduct Cyber Secu-rity Asset Scans

PWS

3.10;

3.1.8

PRS #14

• Frequency: Conduct

Cybersecurity Asset Scans XX per month

• Frequency: 95% of the

TBD

Performance Obj. #15 Maintain property book to track and receive all hardware and software assets in the METIC and the DDC

PWS

3.2

PRS #15

• Inventory Accuracy:

Inventory records should reflect actual stock

• Inventory Accuracy:

98% of the time

Manual spot check of physical inventory

Page 7 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-views and validates

Performance Obj. #16 Develop, manage, and support “serverless computing” by provid-ing scalable containers to allow developers to rapidly deploy applica-tions

Develop, manage, and support “serverless computing” by provid-ing scalable containers to allow developers to rapidly deploy applica-tions

PWS

3.8

PRS #16

Implementation Time: imple-mentation provision time within 5 hours for a low prior-ity request, 3 hours for a me-dium priority request, and 1 hour for a high priority re-quest

• Availability: 24x7x365 days a year

• Implementation Time:

monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #17 Support and manage requirement analysis for pre-production en-vironments to host Ku-bernetes Platform for container services

PWS

3.8

PRS #17

Kubernetes Request Require-ment Analysis: completed <= 5 business days from request

• Kubernetes Request Requirement Analysis:

monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #18 Provide operational support and optimize DoD Network enclaves' (e.g., Med-COI, NIPERNET) network connection for Garrison and .mil for DHMS program capabilities such as OpMed

PRS #18

Implementation Time: imple-ment and/or optimize network connection within 12 weeks for low priority requests, 8 weeks for medium priority re-quests, and 4 weeks for high priority requests

Latency: < 100ms during the first year of contract perfor-mance, < 90ms during the sec-ond year of contract perfor-mance, < 80ms thereafter

Availability: 24x7x365 days a year

• Implementation: 95%

• Latency: 95% of the

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 8 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents

• Restoration: 90% of the time for Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1

Performance Obj. #19 Provide access to secret network/vault for DHMS and DoD health programs conducting high side testing using SIPRNET in secure fa-cilities

PRS #19

Implementation Time: imple-ment and/or optimize network connection within 12 weeks for low priority requests, 8 weeks for medium priority re-quests, and 4 weeks for high priority requests

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root

• Implementation: 95%

• Network Availability:

99.9% of the time

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 9 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incident the time for Priority 1 incidents

Performance Obj. #20 Provide an on-demand dashboard and/or re-porting capability for software and hardware assets in on-prem and cloud

PRS #20

Inventory Accuracy: inventory records should reflect actual stock for hardware

Effective License Position:

100% accurate

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1 hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8 hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents

98% of the time

• Effective License Position: 100% of the

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #21 Provide automated ca-pability to receive, track, and inventory equipment, devices, and test assets in the METIC and DDC

PRS #21

Inventory Accuracy: inventory records should reflect actual stock with an accuracy of >=98%

Tracking Updates: updates to the tracking system should be

• Tracking Updates:

Manual spot check of physical inventory

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging

Page 10 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance reflected within 30 minutes of any status change

Availability: M-F, 0700-1900

EST

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents

• Incident Response:

90% of the time for Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #22 Conduct gap analysis to determine how to implement access to se-cret network/vault for DHMS and DoD health programs conducting high side testing using

SIPRNET

PWS

3.9

PRS #22

Implementation Time: imple-ment and/or optimize network connection within 12 weeks for low priority requests, 8 weeks for medium priority re-quests, and 4 weeks for high priority requests

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30

• Implementation Time:

• Network Availability:

99.9% of the time

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 11 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance minutes for Priority 1 inci-dents

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Performance Obj. #23 Provide .mil, .com, and Med-COI network ac-cess and connectivity to test enclaves and ex-ternal connections

3.4

PRS Table #23 Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1 hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: performance standard achieved 99% of the time for critical, 99% of the time for critical, 95% of the time for moderate, and 95% of incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of

Random Sampling:

Provide .mil, .com, and Med-COI network access and connectivity to test enclaves and external connections

Page 12 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incident the time for Priority 1 incidents

Performance Obj. #24 Develop and provide a centralized platform (via PaaS) to enable standardized DevSecOps frame-work, standardized pro-cesses, associated standardized tooling, and templatized CI/CD pipelines to automate development and test-ing in the cloud

PWS

3.6

PRS #24

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents

AQL performance is achieved when:

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Performance Obj. #25 Develop and enable a self-service provision-ing portal (via IaC) with standardized infra-structure offerings (i.e., servers, storage, and network resources) ac-cessible via service cat-alog

PWS

3.4

PRS #25

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents,

• Imagine, Security

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 13 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents

• Time to Provision: <1 day

Patching: 99% of the time for critical, 99% of the time for critical, 95% of the time for moderate, and 95% of incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of

Performance Obj. #26 Provide a hybrid model for IaaS services by leveraging on-prem “private cloud” and “public cloud” services via automation (IaC)

3.4

PRS #26

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for critical, 95% of the time for moderate, and 95% of

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 14 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents

• Restoration: 90% of the time for Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of

Performance Obj. #27 Monitor physical secu-rity of infrastructure in data center 24x7x365 in real-time using a building management system

PRS #27

• Log Access: Provide physical access logs and physical surveillance video within 1 week of request

• Log Access: 100% of the time

Random Sampling:

Vendor provides physical access logs and physical surveillance video upon request; COR reviews and validates

Performance Obj. #28 Provide locker for stor-age of securable sup-plies (iPhones, CACs, iPads, etc.) in the Labs

PRS #28

Locker Space: Locker space is available

• Storage Compliance:

locker space is compliant with access control restrictions of the facility

• Locker Space: 100%

• Storage Compliance:

Physcial inspection

Performance Obj. #29 Provide capability to remotely connect Vir-tual Desktop Infrastruc-ture (VDI) to DHA net-work within the Lab to remove the need for AVHE and reduce la-tency

PRS #29

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 15 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incidents important, 95% of the time for moderate, and low priority updates incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Performance Obj. #30 Provide remote Virtual Desktop Infrastructure (VDI) for PMOs to test applications within the METIC (inlcuding FFGAL), connected to DoD network enclaves (e.g., MED-COI, NIPRNET, SIPRNET)

PRS #30

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems,

• Availability: 100% of

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and 95% of the time for low priority updates

• Restoration: 90% of the time for Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 16 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incident time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Performance Obj. #31 Provide and maintain network access to VDI, AVHE, NIPRNET, MedCOI, and external partner connections as PMOs would need B2B for partner test environ-ments

PRS #31

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incident

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 17 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Performance Obj. #32 Host, support, and pro-vide ease of use auto-mated managed soft-ware for any/all tools/technologies through an automated self-service portal

PWS

3.9

PRS #32

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents

• Time to Provision: <5 days

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #33 Perform ongoing opti-mization, improve-ment, upgrades, admin-istration, and operation of networks and sys-tems to support DCaaS, CaaS, DaaS, IaaS (e.g., VMWare resource

PWS

3.9

PRS #33

Latency: < 100ms during the first year of contract perfor-mance, < 90ms during the sec-ond year of contract perfor-mance, < 80ms thereafter

Availability: M-F, 0700-1900

EST

Priority 4 incidents, Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Page 18 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance pools, storage), PaaS, and SaaS capabilities

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 incident

90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #34 Provide fully managed tools (standardized, packaged, license con-trolled, and automated tools) to allow stake-holders to conduct de-velopment, integration, testing, and other auto-mated tasks

PRS #34

Latency: < 100ms during the first year of contract perfor-mance, < 90ms during the sec-ond year of contract perfor-mance, < 80ms thereafter

Availability: M-F, 0700-1900

EST

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-

Page 19 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Imaging, Security Updates, and Patching: Installation of updated image, security up-dates, and patching will be within 24 hours of client ap-proval for critical, 72hrs for important, 30 days for moder-ate, and 60 days for low prior-ity updates on laptops and in-frastructure in data centers and cloud environments

• Restoration: restore the functionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for

Patching: 99% of the time for critical, 99% of the time for important, 95% of the time for moderate, and incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #35 Develop industry standard set of tools/technologies at the core of PaaS offer-ing for version control, issue tracking, security tracking, and central-ized log analysis

PRS #35

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents,

• Restoration:

performance standard achieved 90% of the time for Priority 4 incidents, 90% of the time for Priority 3

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 20 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents incidents, 95% of the time for Priority 2 incidents, and 95% of the time for Priority 1 incidents

Performance Obj. #36 Enable self-service por-tal for DHMS custom-ers to access and utilize SaaS Offerings via au-tomated service catalog products (incl. any plug-ins)

3.7

PRS #36

Latency: < 100ms

Availability: 24x7x365 days a year

Incident Response: incident is routed to a supplier's queue and work begins within 5 business days for Priority 4 in-cidents, 1 business day for Priority 3 incidents, 1hr for Priority 2 incidents, and 30 minutes for Priority 1 inci-dents

Restoration: restore the func-tionality of systems, through addressing the root cause or implementing a workaround within 10 business days for Priority 4 incidents, 3 business days for Priority 3 incidents, 8hrs for Priority 2 incidents, and 4 hours for Priority 1 inci-dents

Time to provision: within agreed target time

Priority 4 incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, incidents, 90% of the time for Priority 3 incidents, 95% of the time for Priority 2 incidents, and 95% of

Random Sampling:

Monitor time it takes for a transaction to make a round trip from the DHMS network boundary to the vendor provided system and back to the DHMS network boundary

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #37 Provide METIC availa-bility, including testing infrastructure facility space, equipment floor space, and office floor space, from 0700 - 1900 M-F EST in the District of Columbia, Maryland, and Virginia region with options to schedule overtime and for weekend and surge

PWS

Sec-tion 7

PRS #37

Operating Times: Data Center open 0700 - 1900, M-F EST, with weekend support pro-vided on an ad hoc basis

• Operating Times: 95% of the time

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 21 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance support as required. In-cludes what is currently hosted as well as the capability to be ex-panded for additional requirements. This in-cludes infrastructure, physical medical de-vices, peripherals (printers, laptops, stor-age areas), and facili-ties provisioned for consumption as well as

METIC.

*This is under the as-sumption that the DDC will operate 24x7x365 with remote capability to METIC

Performance Obj. #38 Support data center fa-cilities, including in-stallation and mainte-nance of UPS power, cooling, and rack ser-vices, as required and approved

PWS

3.3

PRS #38

• Response Time: Respond within 24 hours of request

• 95% of the time Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates

Performance Obj. #39 Provide data migration services for data hosted in various tools and in-frastructure in METIC, DDC, TI.com, TI.mil, & cloud to the enter-prise shared services tools (i.e., SDLC tool data)

PWS

3.9

PRS #39

Migration Time: Within agreed target time

• Migration Time: 95% monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-porting data; COR re-views and validates.

Performance Obj. #40 Develop and manage “utilization reports” for the below metrics:

1. % of available Com-pute utilized

2. % of available Memory utilized

3. % of available Stor-age utilized

PWS

3.1

PRS #40

• Reporting: provide monthly reports within 10 business days of the end of the month

• Reporting: 100% of the time

Random Sampling:

Vendor self-reports monthly metrics into cen-tralized reporting dash-board by leveraging DHMS ITSM tool includ-ing calculations and sup-

Page 22 of 56 Pages

PWS

Ref.

Performance Standards - Performance Requirements Summary

(PRS)

Acceptable Quality Level (AQL)

Method of Surveillance

4. % of available Net-work utilized

5. # active user ac-counts

6. # of METIC visit re-quests

7. METIC space utili-zation

8. # of licenses utilized for all software tools/technologies

9. # of software and hardware assets pro-cured/provisioned

10. Cost usage reports in cloud

11. # of containers pro-visioned Performance Obj. #41 Establish security poli-cies and procedures in accordance with DoD

RMF

PWS

3.10;

3.1.8

PRS #41

• RMF Documentation

Complete: submit documentation which meets DoD RMF requirements within 6 months of contract start;

update policies & procedures annually thereafter

• RMF Documentation Complete: 100% of

Compare DoD RMF re-quirements versus vendor developed security poli-cies and procedures

Performance Obj. #42 Perform NIST RMF-based Cybersecurity activities on the TI and provide centralized cyber compliance re-ports at certain fre-quency, in compliance with SOC 2 Type 1 and Type 2 requirements

3.10;

3.1.8

PWS #42

RMF Compliance: compli-ance with DoD RMF report-ing requirements, or waiver granted

• SOC 2 Compliance: pass

3rd party audit

• RMF Compliance:

• SOC 2 Compliance:

Compare required reports submitted against reports required on a monthly and an…