Revision 2.0 - Attachment J-2 Performance Work Statement.pdf

U.S. Department of Homeland Security

Office of the Chief Information Officer

ATTACHMENT J-2: Performance Work Statement for

Data Center and Cloud Optimization

February 2021

AMENDMENT 0005

Revision 2.0

This page intentionally left blank

70RTAC21R00000006 - Amendment 0005 1 Attachment J-2

1 General The Department of Homeland Security (DHS) Data Center and Cloud Optimization (DCCO) acquisition plans to acquire information technology services to manage and operate the DHS Hybrid Computing Environment (HCE). The HCE is a collection of enterprise computing resources including a data center, colocation sites, and commercial and private cloud services. In addition, DHS will acquire associated professional services to automate, optimize, and modernize the HCE.

1.1 Background

The main source of enterprise computing resource for DHS has been Data Center 1 (DC1), a Government Owned Contractor Operated data center located at the National Aeronautics and Space Administration (NASA) Stennis Space Center in Mississippi. NASA owns and manages the data center facility. DHS is one among several tenants within this facility. DHS leases approximately 36,000 square feet of raised floor space in the data center. NASA provides power management, raised floor space, physical security, office space, environmental control, and fire suppression to support DHS IT needs.

In addition to DC1, DHS will add Infrastructure as a Service (IaaS) offered by Cloud Service Providers (CSPs) and colocation facilities to its portfolio of enterprise resources. The Contractor shall operate, optimize, and automate the HCE as an integrated entity. The addition of colocation and CSP environments will extend the capacity, scalability, and redundancy of DHS enterprise computing capacity by providing additional physical, virtual, and cloud resources.

As of September 2020, the operating environment for DC1 can be characterized by the following:

• 14 DHS Components hosting systems in DC1

• 109 Hosted customer Federal Information Security Modernization Act (FISMA) systems

• 2009 Physical servers; 1660 virtual machines (VM)

• Installed networking devices:174 Firewalls; 74 Load balancers; 46 Routers; 1,183 Switches

• 601 Racks populated with DHS equipment, 975 racks installed, total rack capacity 1049

• 765 Terabytes (TB) of provisioned storage, 432 TB utilized

• 7,267 Devices installed supporting Component-owned systems

• 1,425 Devices installed supporting data center management and operations

• Monthly workload statistics, January through June 2020 o 2,694 Service desk tickets generated, ranging from 2,467 low to 2,940 high o 264 Change requests processed, ranging from 225 low to 288 high o 4,062 Incidents managed, ranging from 2,488 low to 7,395 high

70RTAC21R00000006 - Amendment 0005 2

Three unclassified General Support Systems (GSS) operate in DC1 that are essential for the management and operation of the current DC1 and the future HCE:

1. Data Center 1 (DC1) is the GSS providing the underlying hardware infrastructure for the data center consisting of hardware switches, routers, network devices, firewalls, load balancers, and WAN optimization appliances and respective operating systems for the data centers.

2. Data Center 1 Management Zone (DC1 MGTZN) is the GSS providing the support mechanism (management, monitoring, and storage) for DC1 systems consisting of the appliances, hardware (e.g., Storage Area Network (SAN) storage, storage hardware, encryption devices, tape libraries, tape backups, servers, blades, network devices, monitoring appliances), software, and applicable reporting mechanisms and enterprise solutions dedicated to supporting the availability of DC1 Component systems and services.

3. Infrastructure as a Service - DC1 (IaaS-DC1) is the GSS providing compute IaaS private cloud for production and development activities for DHS on-premises systems and applications. The support system provides virtualized networking, storage, and compute capacity and excludes platform as a service (PaaS). The system consists of a fabric manager, computing system chassis, blade servers, hypervisors, Cisco Unified Computing System (UCS) managers, and management tools.

The current HCE contains physical compute servers organized into five zones in DC1 and colocation facilities. The servers include but are not limited to x86 blades from HP, Dell, and other vendors. Server operating systems include but are not limited to Windows, Red Hat Enterprise Linux (RHEL), Community Enterprise Operating System Linux distribution (CentOS), and IBM Advanced Interactive eXecutive (AIX). Some of the servers are configured with hypervisors, including VMware and IBM Power Virtualization Center (VC), to provide VMs. Citrix products are used to support virtual desktop environments provided by the HCE.

The HCE also contains storage servers and SANs including but not limited to products from EMC and Hitachi.

This requirement focuses on information technology services required to support the DHS HCE including those DC1 support services not provided by NASA and infrastructure support services at colocation sites. It also includes the migration, operation and maintenance of systems and applications among the data center, colocation, and cloud environments that constitute the HCE.

For DC1, the DHS Federal Data Center Operations team coordinates with the NASA team for all facility support activities. DHS Federal staff act as the intermediary between NASA and the data center support services Contractor.

The purpose of this requirement is to acquire Contractor support for the operation, maintenance, automation, optimization, and modernization of the DHS HCE and to offer an efficient, responsive Information Technology (IT) hosting environment that serves as the foundation for continued computing operations in support of the DHS mission. The HCE shall include unclassified and classified IT infrastructure, applications, and data.

The Contractor shall provide data center-based hosting and IaaS obtained from both CSP commercial services and CSP services hosted in Government cloud approved environments. The

70RTAC21R00000006 - Amendment 0005 3

Contractor shall provide professional services to support application operation, migration, and Operations and Maintenance (O&M) Orders. The Contractor shall provide customer service including an introduction to HCE environments and services for DHS customers.

DHS intends to operate the HCE as a high priority 24x7x365 computing environment supporting critical mission and business needs for the Department. This includes, but is not limited to, maintaining Service Level Agreements (SLAs) and having the appropriately cleared and qualified staff and support services necessary to maintain operations.

The HCE will offer a more efficient, responsive IT hosting environment that serves as the foundation to ensure continued operations in support of the DHS mission.

1.2 Scope

The scope of this Performance Work Statement (PWS) is the provision of operations, hosting and professional services for the DHS HCE. The Contractor shall identify, provide and manage personnel, processes, tools, and reporting to facilitate transparency and decision‐making as well as to support the transition to the future state HCE. This requirement also includes the continuing support of current operations and services.

The Contractor’s responsibility within DC1 includes the maintenance, operation, enhancement and organization of the IT infrastructure technologies in the HCE. Power, environmental controls, and facility modifications come under the purview of NASA Shared Services Center (NSSC) National Center for Critical Information Processing and Storage (NCCIPS) management. Facility management for DC1 is out of scope in this PWS; provision of facility services in colocation environments is in scope.

The subject PWS will support a single award Indefinite-Delivery Indefinite-Quantity (IDIQ) Contract, which will be available for the use by only the DHS and its Components. All resulting requirements anticipated for execution under this IDIQ Contract shall be coordinated through the Office of the Chief Information Officer (OCIO) Directorate of Information Technology Operations (ITO) Enterprise Operations Division (EOD).

1.3 Objectives

The DCCO requirements will be the core support services needed to drive a more efficient, responsive hybrid IT environment that serves as the foundation for the management and integration of on-premises, colocation, and cloud-based environments.

These support services must optimize and ensure continued DC1 operations while implementing and managing the future state HCE in support of the DHS mission and, where appropriate, the migration of infrastructure and applications within the HCE (e.g., from DC1 to CSP environments).

The objective of DHS’s DCCO support service is to expedite the transformation of IT capabilities from an asset‐based model to a service‐based, customer‐centric IT business model;

provide transparent operational expenditures; and reduce both capital expenditures and time‐to‐ delivery for new capabilities. To accelerate this transformation, DHS will continue existing initiatives associated with cloud optimization efforts as well as leverage new approaches. The business objectives for this requirement include:

70RTAC21R00000006 - Amendment 0005 4

1. Improved Total Cost of Ownership (TCO) and Cost Transparency - Continuously improve the operation of the HCE and provide detailed operational and financial metrics to guide improvements.

2. Modernization and optimization - Operations within the HCE should reflect state-of-the-art, optimized, automated, modern processes that rely upon automatic processes, to the maximum extent practicable, with the minimum required human intervention.

3. Responsiveness ‐ Maximize the customer experience, improve trust, reduce the time to deliver, and enhance operational performance.

4. Scalability ‐ Elastically scale hosting environments and supporting services quickly to meet mission operations and surge capabilities.

5. Improved Reliability and Availability ‐ Monitor and improve Mean‐Time‐Between‐ Failures and Mean‐Time‐To‐Repair metrics.

6. Secure HCE Environment ‐ Monitor and evaluate systems, capabilities, interfaces, applications and data transactions to assess and mitigate cybersecurity threats.

7. Continual Operational Assessment and Improvement ‐ Continuously collect and analyze performance metrics - both technically and administratively - to identify and implement optimizations that improve products and services, and processes reflected by those metrics - leading to continuing performance improvements as processes achieve greater levels of optimization.

8. Architectural approach - Identify objectives for further development and improvement of the HCE technical architecture that enhance interoperability, improve standards, reduce risks, improve security, and support the ability of DHS to take advantage of emerging technologies and capabilities.

9. Service Governance ‐ Enforce a strong governance mechanism to ensure consistent interpretation of policy, monitor DHS enterprise computing performance, adhere to DHS policy requirements and address stakeholder issues.

10. Simplified Management ‐ Improve communication through a data‐driven, transparent, management approach that allows for clear focus and attainment of management and technical goals.

1.4 Applicable Documents

See Appendix A for a full list of Applicable Documents.

1.5 Performance Requirements Summary

This PWS includes a Performance Requirements Summary (PRS) in Section 12. The PRS plays an integral role in the administration of the resulting contract. In addition to any applicable inspection clauses or other related terms and conditions contained in the contract, the PRS shall serve as a primary tool for inspection and acceptance of services as facilitated by the Contracting Officer’s Representative (COR). Evaluation of the Contractor’s overall performance shall be in accordance with the performance standards set forth in the PRS, and will be conducted by the

70RTAC21R00000006 - Amendment 0005 5

COR. The PRS constitutes a material aspect of the PWS and will not be changed or otherwise modified without prior written approval of the Contracting Officer.

1.6 Definitions

Incident Severity. Categorization of the level of negative impact that service outages have on the DHS enterprise.

a. Severity 1 - production systems or applications either not operating or unavailable causing end users to be unable to work in a single or multiple sites for one or more DHS Components.

b. Severity 2 - development environments unavailable with no workaround or alternate with intermittent or no functionality for multiple users in a single or multiple sites for one or more DHS Components; or production environments operating in failover or redundancy mode with intermittent or no functionality for multiple users in a single or multiple sites for one or more DHS Components.

c. Severity 3 - production systems or applications operating in failover or redundancy mode with no impact to multiple users in one or multiple sites for one or more Components; or testing, preproduction, or development environments operating in failover or redundancy mode with intermittent or no functionality for multiple users at one or multiple sites for a single Component.

Inventory. Inventory shall consist of a complete list of all equipment (i.e., hardware), applications (i.e., software), and resources within the HCE, in all physical, virtual, colocation and cloud environments. Software inventory parameters include but are not limited to applications, system, owner, software type, description, type of license (e.g., enterprise, single site), version, cost, date of acquisition, date of installation, and anticipated end of software assurance date.

2 Management Requirements and Tasks The Contractor shall support the requirements in this Section across the HCE. The support includes physical and virtual assets in the DC1 facility in Stennis, MS. It includes the addition of private cloud assets in DC1 and in colocation facilities. The support also includes the addition of commercial cloud and Government cloud approved environment assets from CSPs. All these assets shall be considered as part of the HCE.

2.1 DHS Information Technology Infrastructure Management

The Contractor shall manage DHS IT infrastructure. As required, the Contractor shall interface with DHS Data Center Operations Team to address any issues that require modification to general facility services. The Contractor shall be responsible for implementing operational best practices including the following:

• HCE Operations, Maintenance, and Management

• Network Communications LAN & WAN

• Governance

• Service Classification

• Hybrid Computing Environment Management Automation

70RTAC21R00000006 - Amendment 0005 6

• Policies and Procedures

• General Support Systems Management

• Customer Dashboard and Information Repository

• Acquisition

• Colocation Services

• Cloud Services Management

• Intermediary for Software Licenses

• Hardware and Software License Management System Architecture / Engineering

• Contract Transition

• Automation

• Evolving Technology Evaluation

2.1.1 HCE Operations, Maintenance, and Management

The Contractor shall operate, maintain, and manage the DHS HCE which includes DC1, colocation facility installations, and cloud resources. The Contractor shall operate the HCE on a 24x7x365 basis with at least 99.9% availability. Within the HCE the Contractor shall manage, maintain, and operate Component owned equipment, systems, and software as specified on Task Orders and the GSS described in Section 1.1. The HCE shall provide compute, storage, network, database, and security services for DHS and Component use. Storage services include file, object, block and archival storage. The Contractor shall work with the DHS Data Center Operations Team and COR for any facility management support.

The Contractor shall generate and maintain a Service Offering listing of all services provided within the HCE. The Contractor shall post the Service Offering listing in the Information Repository (See Section 2.1.8) and update the listing with changes as needed but not more than 30 days following actual changes to service offerings.

The Contractor shall meet with the COR and other Government Program representatives weekly to review performance and address operational and managerial issues.

2.1.2 Network Communications LAN & WAN

The Contractor shall maintain and operate Local Area Networks (LAN) within HCE facilities and environments that connect to the DHS Homeland Security Enterprise Network (HSEN) Wide Area Network (WAN) also known as OneNet. DHS provides all WAN connectivity points and equipment. The LAN shall provide dual ports to each hosted server for redundancy and an overall redundant architecture.

The Contractor shall monitor and administer network traffic throughout the HCE, forecast bandwidth requirements associated with service requests, and implement network modifications to accommodate demand. The Contractor shall design the network architecture to support networking requirements including provision for a Zero Trust model and maintain as-designed and as-installed architecture drawings and descriptions in the Information Repository. The Contractor shall also provide dashboard displays of real time network performance metrics and posts notices regarding unplanned network outages as well as any planned service disruptions.

70RTAC21R00000006 - Amendment 0005 7

The Contractor shall provide any access or network monitoring required by DHS management or programs and shall support any integration with enterprise monitoring requirements.

2.1.3 Governance

The Contractor shall operate under the umbrella of DHS governance and oversight processes that dictate interactions among DHS Headquarters, DHS Components, the Contractor Team, colocation providers, CSPs, and all other DHS contractors addressing interrelated requirements.

The Contractor shall adhere, follow, and support all DHS IT and Information Security Policy in governing change and configuration management.

The Contractor shall obtain Government approval for all solutions proposed to satisfy requirements and for all purchases required to fulfill those requirements.

2.1.4 Service Classification

The Contractor shall be capable of providing Level 1 services (see Section 3.8.1), Level 2 services (see Section 3.8.2), and Professional Services (see Section 4) at classified levels, Secret and Top Secret, at DC1 and at all current and future colocation facilities.

The Contractor shall provide services at the following classification levels across the HCE:

• Sensitive But Unclassified (SBU) Level. Including FedRAMP certified services from CSP commercial and CSP Government cloud approved environments assessed at FIPS 199 Low-Low-Low categorization, up to a High-High-High categorization, and variations of any FIPS 199 categorization.

• Secret Level. Services at the Secret level including services in Secret CSP environments.

• Top Secret (TS) / Sensitive Compartmented Information (SCI) Level. Services at the

TS/SCI level including services in TS/SCI CSP environments.

This includes services from public CSPs and includes both commercial and Government cloud approved environments.

The Contractor shall provide services support for Sensitive Compartmented Information Facilities (SCIFs) in HCE in compliance with DHS and Government Security Directives, as needed.

DHS will provide agency specific guidance on the protection of national security information post award.

2.1.5 Hybrid Computing Environment Management Automation

The Contractor shall automate processes throughout the HCE to maximize operational efficiency, included but not limited to:

- Alert Monitoring

- Circuit Monitoring

- Change Management

- Incident Management

- CSP-based Monitoring

70RTAC21R00000006 - Amendment 0005 8

- Configuration Management

- Infrastructure, Security, and Network Monitoring

- Hardware and Software Monitoring

- Analytics, Intelligence, and Reporting

2.1.6 Policies and Procedures

The Contractor shall create, update, and maintain Contractor policies and procedures that govern its operation of the HCE and store those policies and procedures in the Information Repository described in Section 2.1.8.

2.1.7 General Support Systems

The Contractor shall manage, operate, and maintain the existing equipment, software, and other assets that constitute the GSS described in Section 1.1 to manage the existing HCE environment.

The Contractor shall extend GSS management, operation, and maintenance capabilities beyond DC1 to all HCE resources, current and future. The Contractor shall provide dashboard access to GSS-generated reporting, metrics, and statistical data. The Contractor shall be responsible for populating various reports and interfacing GSS operational data to enterprise dashboards as requested.

The Government will provide the infrastructure currently in place supporting the three GSS listed in Section 1.1 as Government Furnished Equipment (GFE). The Contractor shall assume full responsibility for maintaining, enhancing, updating, and operating the hardware and software included within that infrastructure so that the Contractor meets the Government performance requirements. As equipment upgrades, technical refreshments, and replacements are required, the Contractor shall accomplish those upgrades, refreshments, and replacements with hardware and software purchased and owned by the Contractor. DHS intends to transition the three GSS systems described in Section 1.1 from Government Owned, Contractor Operated status to Contractor Owned, Contractor Operated during the period of performance of this contract. This transition will be executed through task order(s) under this IDIQ.

The Government will provide the list of GFE and any related information regarding the GSS listed in Section 1.1 as specified in Section 8.

The Contractor shall support and provide continuous availability of Workplace as a Service (WPaaS) private cloud service supported by the IaaS-DC1 GSS system described in Section 1.1.

WPaaS provides virtualized desktops that are accessible by all DHS Components subscribing to the service.

2.1.8 Customer Dashboard and Information Repository

The Contractor shall develop, establish, maintain, and operate one or more centralized, integrated dashboards and a centralized Information Repository with real-time, on-demand, 24x7x365 access by DHS and Components. The Contractor shall provide role-based access to authorized DHS users and groups, with privileges allowing viewing, data retrieval, and data download in formats (e.g., Microsoft EXCEL, PDF) suitable for analytical processing.

The Contractor shall use the dashboard to track and manage assets and services provisioned and consumed in the HCE, including DC1, colocation centers, and CSPs. The dashboards shall

70RTAC21R00000006 - Amendment 0005 9 enable DHS to manage, use, report, audit, and track assets and services. The dashboards shall include inventory and management of physical and virtual assets and cloud services, usage and performance metrics for all assets and services, security metrics such as cyber threats and vulnerabilities, and metrics for billing and accounting. The dashboards shall capture and report actual and anticipated expenditures, purchases from the catalogs established under this contract, and forecast future spending to support financial, budget, audit, and benchmarking activities. The dashboards shall also provide visibility into resource and asset tagging.

The Contractor shall implement a Financial Management solution based on ITIL Version 4 to provide cost-effective management of HCE resources in accordance with awarded Task Orders.

The Contractor shall provide a dashboard displaying all spending and purchasing under awarded Task Orders and forecasts for future spending.

The Contractor shall establish a centralized knowledge management capability, in conjunction with the dashboard and Information Repository, to share deliverables and related documentation about the HCE. The Contractor shall populate the Repository with information and documentation during the Transition-in Period. The Contractor shall maintain current versions of all documentation during the contract performance period in the Repository and shall provide final versions during the Transition-Out period. The Contractor shall provide DHS with access to the information throughout the period of performance.

The Contractor shall work with DHS to modify the Dashboard and the Information Repository as requested. The Contractor shall perform periodic quality assurance reviews on dashboard data to ensure accuracy and make necessary modifications. The Contractor shall ensure DHS can collect, analyze and synthesize the data (e.g., usage, security, performance) along with system metrics and SLA results on the services being provided. The Contractor shall also enable DHS to collect, analyze and synthesize cost data, including non-labor or professional services, to ensure accurate billing, configuration, and cost transparency. The Contractor shall enable dashboard data to be exported into formats, such as Microsoft Office and Adobe PDF for download.

The Contractor shall upload deliverables, work products, and related material to the Information Repository. These documents shall include environment descriptions and operations manuals, describing the HCE data center, colocation, and cloud IaaS environments, usage, processes and procedures, with updates when there are significant changes in those environments, processes or procedures. The Contractor shall provide separate addendums to these documents for customer unique configurations, processes, or procedures.

2.1.8.1 Real Time Resource Consumption and Cost Tracking

The dashboard shall enable DHS to use, track, report and audit all assets and services, including compute instances, storage volumes, and third-party products. The dashboard shall enable DHS to set and manage manual and automated approval of workloads, quotas, and thresholds for asset and service usage across the HCE for organizational and project-specific accounts. The dashboard shall enable DHS to provision, deploy, deprovision, and decommission assets, infrastructure, and services manually or automatically. The dashboard shall enable DHS to start, stop, terminate, and reboot virtual machines and services. The dashboard shall separate usage costs into billable groups for reporting purposes; and shall provide DHS with the ongoing capability to set thresholds and limits to usage, deployment, and provisioning of resources.

70RTAC21R00000006 - Amendment 0005 10

2.1.8.2 Catalog Purchase Tracking

The Contractor shall track all purchases made through the catalogs established under this contract for equipment & software, professional services, and cloud infrastructure as a service.

The Contractor shall record each catalog’s purchase transactions in a dashboard data set stored in a processable format available to the Government for review and download as part of the Monthly Financial report. The Contractor shall ensure that the purchase data set is updated monthly with notification to the Government upon completion.

2.1.8.3 Capacity Management

The Contractor shall monitor the capacity utilization of HCE resources and provide real-time dashboard displays and periodic reports to DHS regarding capacity utilization. The Contractor shall also develop future capacity requirement projections for each type of resource provided within the HCE. The contractor dashboard shall integrate with DHS enterprise-level systems and dashboards for data uploads, downloads, monitoring, and reporting.

2.1.9 Acquisition

Requirements may demand the acquisition of computing, communications, and data storage assets necessary to host and operate systems applications that DHS Components want to implement and operate. The Contractor shall assess the Task Order requirements specified by Components, determine whether resources are available within the HCE, and either allocate resources if available or procure additional resources as necessary to satisfy those requirements.

2.1.10 Colocation Services

The Contractor shall provide colocation services to DHS customers and shall support the process to obtain physical security authorization and Authority to Operate (ATO) at these colocation locations. Colocation services are defined as leased data center services that provide facility management, connectivity, cloud enablement, and security services. The Contractor shall provide the same operations, maintenance, and management services in colocation facilities as provided in DC1 including on-site hands-on services.

2.1.11 Cloud Services

The Contractor shall obtain CSP commercial cloud and Gov-Cloud IaaS and ancillary cloud services on behalf of DHS customers. The Contractor shall identify FedRAMP certified IaaS and ancillary cloud services offered by CSPs that meet DHS customers’ technical, security, and business objectives as defined in Task Orders and subsequently acquire, integrate, and manage those services as components of the HCE on behalf of the Government.

2.1.12 Intermediary for Software Licenses

The Contractor shall purchase software licenses on behalf of DHS, manage all software licenses, and provide visibility of software licenses within the HCE.

70RTAC21R00000006 - Amendment 0005 11

2.1.13 System Architecture / Engineering

The Contractor shall provide system architecture and engineering services in order to maintain and continuously improve the HCE. The Contractor shall document the HCE system architecture including DC1, colocation centers, CSP infrastructure, the interconnections among HCE elements, and HCE security provisions. The Contractor shall develop and maintain associated operations manuals to optimize operations by DHS and store those architectural descriptions and manuals in the Information Repository.

2.1.14 Contract Transition

The Contractor shall assume responsibility for the management and operation of the HCE from the existing contractor and shall ensure that all operations continue with minimal to no disruption during the contract transition.

2.1.14.1 Transition-In

The Contractor shall develop and implement an Incoming Transition Plan to accomplish the transition of operations from the current service provider to the DCCO Contractor within 120 days from Transition Task Order award without any disruption of service. At a minimum, the Transition shall include:

1. Establishing a Contractor Transition Team within 5 business days of Task Order award.

2. Conducting and completing a site survey and assessment of DC1 within 15 business days of Task Order award.

3. Creating and delivering a draft contract-level Transition-In Plan for assuming full operational and management control of the HCE to include an initial staffing plan, within 30 business days of Transition-In Task Order award. The Contractor shall deliver a final Transition Plan, to include the final staffing plan, within 90 business days of Transition- In Task Order award.

4. Developing and implementing one or more dashboards as part of the Transition-In Task Order to achieve the objectives described in section 2.1.8. The Contractor shall incorporate costs associated with on-going dashboard operation, maintenance, and enhancement within its overall management cost structure.

5. Conducting and documenting an inventory of all equipment (i.e., hardware) and applications (i.e., software), and resources within the HCE - in all physical, virtual, colocation and cloud environments within 60 days of Task Order award.

6. Providing continuous operation of WPaaS, a private cloud service provided by IaaS-DC1.

7. Establishing Task Orders under this new contract for continuing existing customer data center support and Task Orders for new customer support requirements in the HCE. Task Order transition will be addressed on an order by order basis. All Task Orders under the predecessor contract all expire with that contract and must be established under the new contract.

70RTAC21R00000006 - Amendment 0005 12

2.1.14.2 Transition-Out

The Contractor shall:

1. Create and deliver an Exit Transition Strategy Plan within 60 days of Transition-in Task Order Award.

2. Deliver final updates on all policies and procedures governing current operations within the HCE not less than 90 days prior to the end of the period of performance or last exercised option period.

3. Deliver a current, final outgoing inventory of all equipment (i.e., hardware) and applications (i.e., software), and resources within the HCE - in all physical, virtual, colocation and cloud environments, not less than 90 days prior to the end of the period of performance or last exercised option period.

2.1.14.3 Task Order Transition Exit

Shall be specified at the Task Order Level.

2.1.15 Evolving Technology Evaluation

The Contractor shall track, research, and evaluate evolving and emerging technologies that have the potential to improve HCE operations through lower operational cost or improved technical performance and recommend improvements to the HCE to the Government. Those recommendations accepted by the Government will be reflected in subsequent Task Orders. The Contractor may incorporate new or emerging technologies within the three GSS described in Section 1.1 at the Contractor’s discretion after DHS review and approval.

2.2 Security Services - HCE Level

The Contractor shall monitor the security of HCE components, and coordinate with DHS as appropriate, by performing the following work activities:

2.2.1 Security Operations Management

The Contractor shall provide security management services for the HCE and shall also provide specific security-related services in accordance with awarded Task Orders. The security services include maintenance of Zero Trust through identity and access management (role-based or attribute-based) with multifactor authentication and include encryption to protect data at rest and in transit. The contractor shall ensure any access requirements or security monitoring needed by DHS management or programs, or integration with enterprise monitoring requirements, is provided.

The Contractor shall implement the security controls required for DHS servers or applications and implement security mitigations as recommended by the Network Operations and Security Center (NOSC) or other enterprise-level authority. The Contractor shall conduct security testing to verify that the servers, applications, and networking and other equipment are protected against potential cyber threats.

Personnel security services are addressed in Section 5.5.2.

70RTAC21R00000006 - Amendment 0005 13

2.2.1.1 Vulnerability Assessments

The Contractor shall implement and deploy the tools, toolsets, and staff to support, operate, and maintain vulnerability assessment services in the HCE for systems and services delivered and operated by the Contractor. The Contractor shall develop a Vulnerability Assessment Plan and conduct routine, network-based vulnerability scans and assessments.

The Contractor shall track and collect threat and vulnerability data, report threats and vulnerabilities in the centralized dashboard, and implement mitigations as a defense from potential attacks.

2.2.1.2 Intrusion Detection and Prevention Systems (IDS/IPS)

The Contractor shall deploy, operate and maintain an Intrusion Detection System (IDS) to include host-based and network-based detection and an Intrusion Prevention System (IPS) to include host-based and network-based prevention for networks and systems resident within the

HCE.

2.2.1.3 Firewall Management

The Contractor shall operate and maintain firewall software and hardware components and implement and deploy the tools, toolsets, and staff to support, operate, manage and maintain firewalls for applications, systems, and services delivered and operated by the Contractor within the HCE.

2.2.1.4 Anti-Virus Management

The Contractor shall operate and maintain anti-virus protection for the HCE and ensure that signatures and databases are the latest approved and tested versions.

2.2.2 Equipment Access and Control

The Contractor shall:

1. Ensure that all delivery and removal of equipment within the data center facility is authorized by DHS personnel.

2. Ensure that Contractor-issued and owned electronic devices meet DHS configuration guidance, otherwise the equipment shall not be permitted in the data center.

3. Prevent personally owned electronic devices (e.g., laptops, portable storage media, cell phones, etc.) from being taken into restricted areas within the data center.

4. Scan all electronic devices (e.g., government-issued devices, and Contractor-furnished devices) to identify vulnerabilities, verify the existence of up-to-date virus definitions, and ensure compliance with DHS configuration and policy guidance.

5. Dispose of or destroy media containing sensitive information in accordance with National Institute of Standards and Technology (NIST) Special Publication 800-88 Revision 1, Guidelines for Media Sanitization and DHS Sensitive Systems Policy Directive 4300A.

Inventory tracking status and approval chain of custody shall be updated in the dashboard.

70RTAC21R00000006 - Amendment 0005 14

6. Allow the DHS-appointed Communications Security (COMSEC) custodian to execute the responsibilities as outlined in DHS National Security Systems Policy 4300B.200 (COMSEC) Version 4.0.

2.2.3 Authority to Operate

The Contractor shall obtain and maintain a formal ATO as described in the DHS System Authorization Guide for the three GSS described in Section 1.1.

2.2.4 Risk Management

The Contractor shall create a Risk Management Plan and conduct and document Risk Assessments (RA) for the GSS operated by the Contractor after completing a NIST 800-53 evaluation, Contingency Plan testing, and assisting the OCISO in conducting Security Tests and Evaluation (ST&E). Risk Assessments shall identify threats and vulnerabilities, assess the impacts of the threats, evaluate in-place countermeasures, and identify additional countermeasures necessary to ensure an acceptable level of security. Risk Assessments shall also address the cost and schedule of mitigation activities. The Contractor shall update Risk Assessments annually.

2.2.5 Common Controls

The Contractor shall:

1. Enter security controls for each GSS that the Contractor operates and for the infrastructure supporting DC1 private cloud service offerings and the HCE in the DHS Enterprise Cyber Risk Management tool, currently Xacta, and update those controls annually. The controls shall consist of controls provided by FISMA systems as well as physical controls inherited or provided by NCCIPS and by Contractor processes and services such as Change Management and anti-virus deployment.

2. Provide inheritable controls to be used by tenants for their security documentation for systems and applications that those tenants implement on the HCE and enter into the DHS Enterprise Cyber Risk Management tool. These tenant systems and applications will inherit the controls entered by the Contractor in the DHS Enterprise Cyber Risk Management tool for the private cloud infrastructure.

3. Provide additional security controls for systems and services delivered and operated by the Contractor.

2.2.6 Access Management

The Contractor shall provide DHS with full inquiry, retrieval, and data download user access to all approved service offerings. In addition to user access, the Contractor shall enable application access to the HCE by enterprise applications. This shall include allowing applications to use the Application Programming Interfaces (APIs), Management Consoles, and Software Development Kits (SDKs) available from the CSPs, to directly interact with CSP services.

70RTAC21R00000006 - Amendment 0005 15

3 Operations Support Requirements and Tasks

3.1 Operations Services

The Contractor shall provide HCE operations services on a 24x7x365 basis. The Contractor shall provide real-time system status to DHS customers and provide monthly service level reports. The Contractor shall develop and maintain Standard Operating Procedures (SOPs) governing infrastructure operations to align with relevant SLAs and shall provide access to the documentation following commercial and federal standards and best practices.

3.2 Service Desk

The Contractor shall maintain and operate a 24x7x365 Service Desk for infrastructure, applications, and services residing within the HCE. The Contractor shall respond to and manage incidents occurring within the HCE. The Contractor shall provide efficient management of incident response and maintain communication with DHS stakeholders during response and resolution and provide an escalation path for the resolution of complex issues. The Contractor shall:

1. Accept and process incoming service requests.

2. Track and manage service requests from receipt through closure, problem reports to closure, and provide statistics for service desk performance reporting and analysis.

3. Provide enterprise help desk support for GSS applications that the Contractor operates in on-premises locations, colocation environments, approved CSP platforms and environments, and internal private cloud infrastructure supported by the Contractor.

4. Provide service and incident management support to Components for systems, applications, and services hosted within the HCE.

5. Integrate solutions (e.g., service management software), processes, and procedures with overall DHS IT enterprise reporting solutions and incident management processes and functions as components of enterprise DHS incident management.

6. Provide documented root cause analysis on request with recommendations for remediation and provide initial and final analysis reports to DHS, load reports to the Information Repository, and provide role-based access to the root cause reports.

7. Track ongoing infrastructure operation metrics and Service Desk performance metrics in the dashboard, in the monthly Service Quality Review (SQR), and in the formal quarterly Service Level Agreement Level of Service Report.

3.3 Inventory Control and Asset Management

The Contractor shall use the dashboard to manage the inventory of all equipment (i.e., hardware) and applications (i.e., software), and resources within the HCE - in all physical, virtual, colocation and cloud environments in the HCE. The Contractor shall:

1. Maintain a Configuration Management Database (CMDB) with the complete HCE hardware and software asset inventory including IaaS assets (e.g. virtual machines

70RTAC21R00000006 - Amendment 0005 16 and storage services) accessible through the dashboard and provide integration with and access to enterprise CMDB solution(s).

2. Record receipt of all incoming hardware and software items within an asset tracking and inventory management system and record changes in location for each item as they occur.

3. Provide retrieval, view, and download access to all asset information indicating location and status of each inventory item as well as maintenance and licensing agreements and expiration dates for all inventory items. Download access shall enable the export of data in processable formats (e.g., Microsoft EXCEL, PDF) for offline analysis.

4. Deliver a complete, fully validated inventory of all equipment, applications and system software within the HCE not less frequently than at four-month intervals in both the dashboard and as a separate analytical data file.

3.4 Change Management

The Contractor shall control change within the HCE in compliance with the enterprise change management process to ensure the completeness and integrity of all changes implemented within the HCE. The Contractor shall follow established DHS processes and procedures for adequate change management control and implementation.

The change processes shall manage the following components that may change in fulfillment of Task Orders:

• Hardware

• Architecture

• System software

• Application software

• Cloud IaaS services and resources

All documentation and procedures associated with the operation, support, and maintenance of systems, equipment, services, and applications shall be stored in the Information Repository.

3.5 Customer Satisfaction

The Contractor shall solicit an assessment of Customer satisfaction from each Task Order owner.

Contractor shall design and implement surveys to capture and document customer satisfaction using Net Promoter Score to gauge Customer satisfaction. The Contractor shall develop and execute corrective action plans to remedy deficiencies identified through these surveys.

3.6 Quality Control

The Contractor shall develop and implement a Quality Management Program. The Contractor shall create and submit a Quality Management Plan (QMP) describing the standards, processes and procedures used to support the consistent delivery of high-quality, professional products and services provided in support of 24x7x365 HCE operation. The QMP shall be based on the Quality Assurance Surveillance Plan (QASP) provided by the +. The Contractor shall deliver the

70RTAC21R00000006 - Amendment 0005 17 initial QMP 60 days after Transition Task Order award and review and update the QMP at least annually.

The Contractor shall implement processes and procedures described in the QMP to meet and measure performance pursuant to the Government-provided SLAs. The Contractor shall not diminish the service offerings from the CSPs to a level lower than the published commercial cloud service, or standard SLAs (Acceptable Quality Levels [AQLs] in Section 12 correspond to SLAs), unless otherwise specified and approved by DHS.

3.6.1 Application Quality Control

The Contractor shall provide Application Quality Control services for applications hosted in the HCE in order to maintain application integrity. The Contractor shall manage Quality Control according to industry best practices and shall detect and report quality problems per contract SLAs and any SLAs particular to Task Orders.

3.6.2 Independent Verification and Validation Support

As directed by the COR, the Contractor shall provide cooperation and support to any Independent Verification and Validation (IV&V) project performed by or commissioned by DHS or the Department’s designated representative.

3.7 Availability Management

The Contractor shall ensure CSPs provide a minimum availability of 99.9% for each service (e.g., virtual machine, object storage, virtual private cloud) incorporated into the HCE, unless otherwise published by the CSP. The Contractor shall ensure each service provided to DHS, meets or exceeds, the commercially advertised level or published SLA. The Contractor shall ensure that services launched by DHS in multiple cloud zones or regions continue to operate and remain available when any of the CSP’s data centers are offline or unavailable. The Contractor shall ensure CSP services acquired for DHS have redundancy characteristics that ensure the capability for maintaining the minimum availability requirement.

3.8 Operations and Maintenance Services

The Contractor shall follow the industry recognized ITIL version 4 practices to provide lifecycle O&M support for the HCE. This includes installing and configuring hardware and system software on physical equipment, system software on cloud-based IaaS services, integrating, testing and securing physical and cloud resources, and conducting ongoing maintenance. The ongoing maintenance shall include asset management, change management, problem and incident management, continuity of operations, continuous monitoring, and decommissioning.

The Contractor shall apply a similar approach to providing lifecycle O&M support for customer applications in the HCE. The Contractor shall apply this lifecycle approach at the required classification levels: Unclassified, SBU, Secret, and Top Secret (TS), including Sensitive Compartmented Information (SCI).

70RTAC21R00000006 - Amendment 0005 18

The Contractor shall provide tiered service levels:

• Level 1 services apply to physical assets located in DC1 or in colocation centers -compute, storage, database, and networking devices.

• Level 2 services apply to all system software, including operating systems, database management systems, data backup systems and network software, in any HCE environment. System software also includes tools for managing and monitoring hardware and software components, problems and incidents, and resource management.

Level 2 services may apply to physical assets within the HCE, virtual assets within the HCE, or to assets placed in HCE cloud environments.

3.8.1 Basic Level Service (Level 1)

Basic Level Service (Level 1) is a hosting service offered for physical assets installed within the HCE. Level 1 services include hardware installation and maintenance and network monitoring for equipment. This is the minimum level of service that is provided for all physical assets residing within the HCE. The Contractor shall ensure that equipment is installed in DC1 or in colocation facilities as appropriate and that the equipment is brought up to an operational state and the Contractor shall also provide the services described in this section. Basic Level Service provides network connectivity from the servers to the WAN point of demarcation for all systems hosted in the environment.

3.8.1.1 Installation

The Contractor shall perform all tasks necessary to properly install equipment to ensure all items are installed in the state as required by Task Orders. The Contractor shall test and document equipment and ensure that the enterprise change management process is followed. The Contractor shall ensure proper communication with the DHS COR, HCE service desk, DHS Data Center Operations Team, and facility management (whether NASA for DC1 installations or colocation facility management for colocation installations), as appropriate.

3.8.1.2 Hardware Maintenance

For all equipment within the HCE the Contractor shall:

1. Monitor equipment.

2.