1.1.4_Statement of Work_Cloud Hosting BPA.docx

FEMA Cloud Hosting BPA Description of Requirement

1. FEMA Cloud Hosting BPA The Federal Emergency Management Agency (FEMA) Office of the Chief Information Officer has a continued requirement for a reseller to provide access to FedRAMP certified, true On-Demand commercial Cloud Service Providers (CSPs) for our Commercial Cloud Hosting Services (CCHS). This includes Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).

2. Background FEMA has built and is currently operating over 30 major, mission-critical systems that are being hosted in the Amazon Web Services (AWS) commercial cloud. These applications are deployed in AWS and designed to leverage programmatic provisioning capabilities offered by the AWS commercial cloud.

This requirement supports the delivery of FEMA systems via Continuous Integration/Continuous Delivery (CI/CD) DevOps methodologies for building quality software that is developed, built, tested, and implemented into production quickly and in rapid succession using the CI/CD pipeline.

FEMA programs and systems supported by this Description of Requirement (DOR) serve both public and private stakeholders and can range broadly in terms of size, complexity, and importance. In order to support this architecture in CSPs, FEMA requires a true on-demand secure, scalable, flexible, automated and cost effective cloud computing environment that provides a developer access to Web Graphical User Interface (GUI) Console and Application Programming Interfaces (APIs) to programmatically provision all hosting services, hardened images, formation templates, auto scaling of infrastructure, large scalable databases as a service, large scalable storage as a service, development services, monitoring services, containerization services, automated configuration control services, and automated scalable networks services that provide for rapid provisioning and de-provisioning of entire system environments in minutes.

3. Scope FEMA requires reseller services to access multiple FedRAMP certified true on-demand CSPs. The scope of this Blanket Purchase Agreement (BPA) includes the ability to access new CSPs as they become FedRAMP certified and available through the reseller. The access provided by this BPA shall support FEMA’s CI/CD DevOps Information Technology (IT) systems development model and provide SaaS options. The types of cloud services required by this BPA are IaaS, PaaS, and SaaS (see definitions section for additional details). The BPA is for reseller services only to provide access to and a payment mechanism for consumption based true On-demand Commercial Cloud service providers. Any services are administrative only as described in the DOR for providing the unfettered access and assisting in any service outages or issues with the CSPs. Development, migration, or managed service provider services are not part of this procurement and will not be considered if proposed.

This BPA shall include access to the AWS CSP to continue to host existing FEMA systems that are currently deployed in AWS (approximately 95% of the total requirement). This BPA shall also include access to the Microsoft Azure Cloud Platform and services, and to the Google Cloud Platform (GCP) and services. This BPA may also include access to other FISMA Moderate and FISMA High FedRAMP certified, true on- demand IaaS / PaaS CSP(s) that meets the salient characteristics listed below.

Additionally, this BPA shall include access to several different SaaS options, including at least one SaaS option that meets the salient characteristics listed for each of the SaaS offerings identified in Section 4.2. The scope of this BPA encompasses the ability to add additional SaaS requirements as needed.

The decision about where to place new programs, systems and IT capabilities being implemented by the agency, and additional legacy information systems migrating to cloud-based IT support models, will be made by the government based on an analysis of the technical architecture and cost effectiveness of each CSP, depending on the system.

4. Systems Requirements All services shall be provided by a consumption based True On-demand Commercial Cloud Service Provider with the capability to provide real-time hardware resources to support the Governments virtual infrastructure without any need to estimate or procure those hardware resources. The hardware resources underlying the virtual infrastructure shall become available in real time at the launch of the virtual instance and the virtual instances shall be billed in increments no larger than one hour. There shall be no requirement for any interaction between the Contractor or CSP and the FEMA DevOps teams in order to configure any of the infrastructure and platform cloud services and the hardware to operate those resource shall be instantly available to support the performance specifications for that service. For example, there shall be no ticketing system to a third party to configure the infrastructure. The DevOps teams shall have direct access to the GUI and API to launch and configure resource instances, and these resource instances shall be available within 5 minutes after the DevOps Teams execute script API calls, via automated infrastructure code that executes API calls, or initiating the configuration from the GUI console.

a) Scalability – Each CSP shall be both horizontally and vertically scalable to two thousand virtual instances in less than 10 minutes. It shall support hundreds of petabytes of storage, a minimum of six terabytes of storage for databases, and be able to scale across two or more data centers as a selectable option in the API or web console. The CSP shall provide a serverless compute offering. The CSP must provide a Point of Presence at a co-location facility where a connection of 10GB or greater can be connected to the FEMA network without a partner connection. The CSP must provide Relational databases as service for multiple database vendors. The CSP support multiple operating systems including RHEL and Windows 2019. CSP must provide a Kubernetes platform as a service. The Kubernetes service must support RHEL worker nodes.

b) Reliability – Each CSP shall provide redundancy across multiple data centers, geographically dispersed from each other, to prevent natural and man-made disasters from impacting operational status. All services shall be scaled over multiple data centers and provide for real-time automatic failover without interruption to the service the systems are supporting, should a disaster occur. Launching resources in multiple data centers shall be a selectable option within the API and web console.

c) Availability – Each CSP shall provide a minimum availability level of 99.9% for each of their services unless otherwise stated for a particular service. And each service shall meet their commercially advertised Service Level Agreements (SLA’s). Resources launched in multiple data centers shall continue to operate and be available when one of the data centers is offline for 100% uptime redundancy.

d) Direct Access to the Console and APIs – Each CSP shall provide for direct and unfettered access to the cloud management and configuration console and APIs by Government personnel to control all the resources and configuration within the cloud.

e) CloudCheckr (or equal) Integration – All IaaS and PaaS clouds must be integrated with the CloudCheckr (or equal) cloud reporting tool – identified in Section 4.2 below – as this is how the

Government manages its cloud environments for cost, configuration management, and security compliance.

f) Technical Support Requests – FEMA shall have the ability to submit technical support requests to each CSP and interface directly with each CSP’s support team as applicable without restriction or approval by the Contractor.

4.1 IaaS and PaaS True On-Demand Commercial Cloud Services – In order to support FEMA’s objective and architectures and provide cloud diversity, the contractor shall provide the CSPs named in Section 3 of this DOR for the IaaS and PaaS offerings. The offeror may provide more offerings. Each offering shall include all of the following services:

4.1.1 Virtual Compute Service (VCS) – Virtual compute resources shall be configurable in real-time. In addition, VCSs must:

a) Be able to instantiate and resize on demand without the need to rebuild the virtual instance,

b) Provide scalable, on demand sizing to support additional load or reduction in load via programmatic scripting through an API.

c) Allow the automated configuration of memory, CPU, instance storage, and the boot partition size that is optimal for the Government’s choice of operating system and application. The VCS shall support the latest versions of operating systems such as Red Hat Linux, Ubuntu Linux, CentOS Linux, and Microsoft Windows Server 2012 or newer. These shall be updated over the life of the BPA as new operating system technologies emerge and are supported by each CSP.

4.1.2 Scaling Automation Service (SAS) – Each CSP shall provide a Virtual Computer scaling service that provides for programmatic launching and shutdown of compute instances based on load of the instances in the compute pool. Load shall be based on an increase or decrease in processor or memory utilization of a period of time to trigger the launch or shutdown action.

4.1.3 Database Management Services (DBMS) – Each CSP shall provide a minimum of three database services that do not require any management or administration on the part of the customer (FEMA) for the underlying DBMS. It shall provide for automatic backups that can be configured by the user to save the data in cloud storage for data recovery purposes in the event of a system failure. All database configuration commands and options shall be available via the GUI or API

4.1.3.1 Relational Database Services (RDS) – Each CSP shall provide a highly available relational database service for large scalable databases without FEMA having to manage the servers or DBMS. This service must be geographically redundant over more than 500 miles. The database creation, modification, deletion and management on this service must be available via API’s and scale on demand to 20000 IOPS and 6 TB. The RDS shall provide a minimum of two of the following databases: Oracle, MySQL, PostgreSQL, and Microsoft SQL Server.

4.1.3.2 NoSQL (NSDS) – Each CSP shall provide a fast and flexible NoSQL database service for applications that need consistent, single-digit millisecond latency at any scale. It shall be a fully managed cloud database that supports both document and key-value store models. It shall provide a flexible data model, reliable performance, and automatic scaling of throughput capacity, providing response times from milliseconds to microseconds up to millions of requests per second.

4.1.3.3 Data Warehouse Service (DWS) – Each CSP shall provide a fully managed, petabyte-scale data warehouse service in the cloud, which allows for a100GB of data to stability and the ability to scale to a petabyte or more. This service shall allow the user to run Business Intelligence tools against the data source to use the data to acquire new insights for the business. Complex query datasets shall be returned within minutes using SQL based tools to analyze the data.

4.1.3.4 Data Migration Service (DMS) – Each CSP shall provide a database migration service to securely move data from on premise databases and file storage to the CSP. The on premise database shall remain fully operational during the migration, minimizing downtime to applications that rely on the database. The Database Migration Service shall migrate data to and from widely used commercial and open-source databases. It shall also migrate from the same database platform as well as convert from one database platform to another. For example, from Oracle to Oracle, or from Microsoft SQL Server to MySQL.

4.1.4 Storage – All storage shall be highly available and replicated automatically across multiple data centers to prevent outages or loss of data.

4.1.4.1 Scalable Block Storage (SBS) – Each CSP shall provide persistent high speed, low latency block storage for the virtual compute instances that remain even when the virtual compute instance is turned off or decommissioned. The storage shall be scalable up within 15 minutes of an API call or administration console change.

4.1.4.2 Highly Scalable Network Storage Service (HSNSS) – Each CSP shall provide access to reliable, fast, and inexpensive network data storage infrastructure. It shall provide web-scale computing by allowing storage and retrieve any amount of data, at any time, from within CSP or anywhere on the web. It shall store data objects redundantly on multiple devices across multiple facilities and allows concurrent read or write access to these data objects by many separate clients or application threads. The redundant data stored shall be usable to recover reliably and quickly from instance or application failures. This storage shall also allow for the storage and recovery of machine images, snapshots, and data backups programmatically. It shall provide for programmatic recovery from these snapshots and backups in the case of a system failure. The data stores shall be logically organized in buckets and be accessible to the systems that are providing permission to that data. The data shall have HTTPS URL pointers that address the data by the logical buckets in which it is stored.

4.1.4.3 Highly Scalable (Glacial) Network Storage Service (GNSS) – Each CSP shall provide slower speed network storage for data that is less frequently accessed. This storage has the same redundancy requirements as the Highly Scalable Network Storage, but with higher latency disk to provide for more cost effective storage of long term, infrequently accessed data.

4.1.4.4 Storage Gateway Service (SGS) – Each CSP shall provide a storage access gateway that allows on premise computing equipment to access data bucket in the HSNSS and the GNSS.

4.1.4.5 Auto-Scaling File Storage (AFS) – Each CSP shall provide for automatic scaling File Storage that allows the expansion and contraction of the file system as data is added to and removed from the File Store. The file store shall be mountable by the Virtual Compute Instances and provide file storage as needed based on the storage that is needed. The AFS shall provide the following capabilities:

a) The AFS shall support Network File System (NFS) standards allowing multiple VCS instances to access AFS file system at the same time, allowing the AFS to provide a common data source for workloads and applications running on more than one VC instance.

b) The AFS shall provide the capability to be mounted by on-premise resources allowing them to migrate data sets to AFS, enable cloud bursting scenarios, or backup on-premise data to AFS.

4.1.4.6 Storage Import and Export Services – Each CSP shall provide a FIPS 140-2 compliant petabyte-scale physical data transport service that uses secure appliances to transfer large amounts of data into and out of the cloud. This service shall alleviate the common challenges with large-scale data transfers such as high network costs, long transfer times, data synchronization and secure strong encryption and transport.

4.1.5 Server Containerization Services (SCS) – Each CSP shall provide a highly scalable, high performance, container management service that supports a standard container format such as Docker, Kubernetes, or Pivotal Cloud Foundry and allows applications to easily run on a CSP managed cluster of Virtual Computer instances.

a) This service shall eliminate the need to install, operate, and scale the cluster management infrastructure.

b) The service shall, via the API, be able to launch and stop container enabled applications, query the complete state of the cluster, and access many features like security groups, Virtual Load Balancing, SBS volumes, and IAM roles. The system teams shall be able to schedule the placement of containers across the cluster service based on resource needs and availability requirements.

c) This service shall support third-party schedulers to meet business or application specific requirements.

4.1.6 Network Services – Each CSP shall provide the following network services:

4.1.6.1 Private Virtual Cloud Network (PVCN) – Each CSP shall provide a Private Virtual Cloud Network segment that is a logically isolated section of the CSP where resources can be launched in a virtual network that FEMA defines. FEMA shall have complete control over the virtual networking environment, including selection of its own IP address range, creation of subnets, and configuration of route tables and network gateways. The PVCN shall support both IPv4 and IPv6. The service shall support a minimum 100 PVCN’s per account. The PVCN shall support multiple layers of security:

a) The PVCN shall support a minimum of 1000 Security/Firewalling rules per PVCN to control traffic flow.

b) The PVCN shall support a minimum of 1000 Access Control Rules to control traffic flow.

c) The PVCN shall support a hub and spoke networking model to allow traffic flow inside the CSP to a central point that can be connected back to the DHS data centers or an external MTIPS service via an encrypted Virtual Private Network (VPN) connection between the DHS datacenters to act as an extension of these datacenters.

4.1.6.2 Virtual Load Balancing (VLB) – Each CSP shall provide an auto-scalable virtual load balancing capability that automatically distributes incoming application traffic across multiple targets, such as VCS instances or SCS containers. It shall handle the varying load of application traffic locally or across data centers. VLB shall offer three types of load balancers that all feature the high availability, automatic scaling, and robust security necessary to make the applications fault tolerant.

a) Application Load Balancing – An Application Load Balancer shall support load balancing of HTTP and HTTPS traffic and provide advanced request routing targeted at the delivery of modern application architectures, including micro services and containers, operating at the individual request level (Layer 7).

b) Network Load Balancing – Network Load Balancer shall provide load balancing of Transport Control Protocol (TCP) traffic where extreme performance is required. It shall operate at the connection level (Layer 4) and handle millions of requests per second while maintaining ultra-low latencies.

c) Classic Load Balancing – Classic Load Balancer shall provide basic load balancing across multiple VCS instances and operates at both the request level and connection level.

d) All Virtual Load balancing services shall adhere to the following performance characteristics:

1. High Availability – Automatically distribute incoming traffic across multiple targets in multiple data centers and only to healthy targets that can process the traffic.

2. Security – Provide robust security features, including integrated certificate management and SSL termination and decryption. Centrally manage SSL settings and offload CPU intensive workloads from the applications.

3. Auto Scalability – Virtual Load Balancing shall be capable of handling rapid changes in network traffic patterns. Additionally, integration with auto scaling capabilities shall ensure sufficient application capacity to meet varying levels of application load without requiring manual intervention.

4. Flexibility – Virtual Load Balancing shall provide the capability to use IP addresses to route requests to application targets. This shall provide the capability using application load balancing to run multiple applications on the same virtual compute instance using the same network port to also simplify inter-application communication in micro services based architecture.

5. Monitoring – Virtual Load Balancing shall provide monitoring of applications and their performance in real time with metrics, logging, and request tracing exportable to Splunk to improve visibility into the behavior of your applications, uncovering issues and identifying performance bottlenecks in your application stack at the granularity of an individual request.

6. Hybrid Load Balancing – Virtual Load Balancing shall provide the ability to load balance across the CSP and on-premise resources using the same load balancer.

7. DNS Services (DNSS) – Each CSP shall provide a DNS service that integrates with the customers on premise DNS service to provide programmatic add, update, and deletion of DNS records via GUI or API.

4.1.6.3 Direct Connection Service (DCS) – The ability to bypass the public Internet and provide a direct connection to the commercial cloud service provider from the DHS Data Center over a secure scalable on-demand cloud interconnection, providing a secure connection over a private link that improves performance, reduces costs, increases security, and ensures consistent throughput. This connection shall support network speeds up to 10Gbps. This service shall be charged by the amount of data throughput consumed. The service shall support the following criteria:

a) Ability to move large volumes of data in and out of the cloud faster and more securely than over a public internet connection.

b) Ability to achieve predictable network performance and deliver a consistent end user experience.

c) Ability to provide flexible bandwidth options below one Gbps enabling bursts up to two times the committed access rate at no additional cost.

d) Ability to handle workloads that require occasional bandwidth bursts, such as backups, allowing a benefit from higher performance when bandwidth needs spike and from reduced connection costs when bandwidth needs are low.

e) Ability to provide infrastructure components such as Compute, Storage, Database Services, Monitoring, Network, Cloud Automation, and middleware services via programmatic interfaces.

4.1.7 Development Platforms – Each CSP shall provide PaaS development platforms that provide for loading code and running a workload through a managed CI/CD pipeline and all infrastructure is handled by the development and infrastructure code. This service shall support, at a minimum, .Net, Java, Ruby, Node JS, and PHP development platforms.

4.1.8 Artificial Intelligence Platforms – Each CSP shall provide a PaaS capability that supports machine learning and artificial intelligence based on algorithms that can parse data, find patterns and learn from that data make determinations/predictions about the world based on the data, and update settings without relying on a human to change the rules-based programming.

4.1.9 Cloud Service Monitoring (CSM) – Each CSP shall provide a monitoring service for cloud resources and the applications run in the cloud. This service shall collect and track metrics, collect and monitor log files, set alarms, and automatically react to changes in the cloud resources. This service shall monitor all forms of cloud service resources (e.g. virtual compute, storage, network, database, containerization, etc.), application logs, and provide that data via an automated interface to external logging tools such as Splunk.

4.1.10 Content Delivery Services (CDS) – The commercial cloud service shall provide a global CDS that securely provides the following capabilities:

a) Cached static data content – secured delivery of cached static data content across the internet outside of the direct CSP network connection to provide for fast delivery of content to the end user.

b) Video Streaming – secured delivery of streaming video content over the internet to the end user. This service shall support streaming for video, pre-recorded files and live events with sustained, high throughput to support high definition video. On-demand streaming, shall support multi-bitrate adaptive streaming in Microsoft Smooth, HLS, HDS, or MPEG-DASH formats to any device.

c) Security against Distributed Denial of Service (DDoS) – All CDS services shall provide security against DDoS attacks and prevent attacks that alter or affect the content that is being provided by this service.

d) Secure Delivery methods – The CDS shall deliver content in a secure manner compliant with PCI, DSS, HIPAA, and ISO to ensure secure delivery of the most sensitive data. It shall provide secure delivery against DDoS attacks. Secure APIs for applications shall be delivered via SSL/TLS, and advanced SSL features are available automatically.

e) Commercial Third Party Key Support – The service shall be able to utilize commercially available third party certificates from major provider to secure content. This shall include for example: Symantec, Comodo, Entrust, Verizon, Digicert, etc.

4.1.11 Key Management Service (KMS) – Each CSP shall provide a fully managed secure key management service where the CSP handles the availability, physical security, and hardware maintenance of the underlying infrastructure. This service shall store keys in hardware security modules (HSMs). This service shall provide the following:

a) Support for import of internal US Treasury or commercially available encryption certificates keys into the HSMs.

b) Each CSP shall patch, and maintain HSMs and key management software. FEMA shall be able to provision new vaults and keys in minutes and centrally manage keys, secrets, and policies.

c) The applications and FEMA developers shall be granted permissions to the keys, but shall never have direct access to the keys. FEMA developers shall have access to manage Dev and Test Keys, but once transitioned to production, production security and operations personnel shall be the only ones with access to production keys.

d) The production key stores shall support SSL/TLS certificates and automatically renew certificates with public certificate authorities.

e) Keys shall be created, imported, and rotated via commands from the API.

f) All Key management events shall be logged and provided via the CMS to an external security tool, in this instance the FEMA Splunk tool.

4.1.12 Server-less Compute Service (SCS) – The SCS shall provide compute services to execute code that does not require the customer to first provision virtual servers or container services to execute code. The compute instances instantiated by the CSP to execute the code shall be continuously scalable for parallel code execution. The service shall be on-demand based on the compute resources used for the period of time the code execution consumes.

4.1.13 Identity and Access Management Service (IAMS) – Each CSP shall provide an IAMS that is fully configurable by customer that integrates with customer three factor authentication systems. This service shall support the Government three factor authentication standard.

4.1.14 Cloud Service Catalog (CSC) – Each CSP shall provide a CSC to create and manage catalogs of IT services that are approved for use in the cloud. These IT services can include everything from virtual machine images, servers, software, and databases to complete multi-tier application architectures. The CSC shall provide for centrally managed, commonly deployed IT services to achieve consistent governance and meet compliance requirements, while enabling users to quickly deploy approved IT services.

4.1.15 Cloud Formation Service (CFS) – Each CSP shall provide a cloud formation capability via its API calls which can build an entire system computing environment from script. This shall include all networking components, routing, firewalling, security, keys, certificates, users, roles, virtual computing instances, containers, micro-services, storage, databases, load balancers, messaging services, monitoring, and content delivery to establish an entire environment.

4.1.16 Messaging Services – Each CSP shall deliver the following messaging services:

a) Message Queueing Service (MQS) – Each CSP shall provide a fully managed MQS that easily enables the decoupling and scaling of micro-services, distributed systems, and server- less applications. This service shall support the building of applications from individual components that each performs a discrete function which improves scalability and reliability and is fault tolerant. This service shall allow the sending, storage, and receipt of messages between software components at any volume, without losing messages or requiring other services to always be available. The queues shall be manageable via the API just like any other service in the cloud. The queueing services shall provide the following two choices for methods of delivery:

1. Best Effort – The Best Effort method of deliver shall provide for maximum throughput, best-effort ordering, and at-least-once delivery.

2. First In First Out (FIFO) – The FIFO method shall guarantee that messages are processed exactly once, in the exact order that they are sent.

4.1.17 Publish/Subscribe Notification – Each CSP shall provide a fully managed pub/sub messaging service that makes it easy to decouple and scale micro-services, distributed systems, and server- less applications. This service shall support communications to the messaging queues, desktop architectures, and mobile devices including Android and Apple iOS. This service shall support event driven computing workflows based on topics and triggers.

4.2 Software as a Service – The contractor shall provide access to at least one offering of each of the following FedRAMP certified SaaS services:

4.2.1 Customer Management Services – The reseller shall provide a SaaS Customer Relationship Management service that has the following capabilities:

a) A dedicated document library for creating, storing, and retrieving documentation on each customer, standards for call protocols, and the customer’s application status with FEMA.

b) Role based user access that integrates with the DHS Active Directory service.

c) Workflow automation for handling the calls that is customizable by FEMA personnel to match the workflow patterns of the Agency.

d) Integration with the DHS Exchange Email system to manage communications within the agency.

e) API calls to allow for automation of work patterns.

f) Standard and Customizable reports to get status on calls such as, call volume, call patterns, call hang-ups, call wait states, call resolutions, and call volumes by topic.

4.2.2 Service Desk Management Services – The reseller shall provide a SaaS Service Desk Management service that supports Asset Management, customizable ticketing workflows, knowledge base management, smartphone apps for the user, web based user portal, customizable, user role based permissions that integrate with the DHS Active Directory Services, and integration with the Microsoft Exchange email system.

4.2.3 Unified Communications Services – The reseller shall provide a SaaS unified communications service that is cloud based and meets government-level security standard to allow organizations to collaborate with internal government and external commercial entities from any device, mobile, computer, or desk phone. This service shall include the capability to delivering mobility, voice, video conferencing, web, chat, presence, audio conferencing, voicemail, textual chat.

4.2.4 Video Conferencing Services – The reseller shall provide access to Video Conferencing SaaS that allows for using the call connection hardware completely hosted in the cloud. It shall be capable of supporting between 2-1,000 participant connections supporting small to large Town Hall type conferencing capabilities. This service shall also include shared whiteboard, computer application and screen sharing, calls, and text chat.

4.2.5 Mobile Device Management Services – The reseller shall provide a SaaS to deploy, configure, secure, manage and support smartphones, tablets, laptops and other devices across multiple mobile applications and operating systems. It shall support line-of-business devices the Government can implement in a containerized solution or as a comprehensive solution based on device type, use case and user role in the organization.

4.2.6 Desktop Virtualization Services – The reseller shall provide the SaaS capability to deliver virtual desktops to the entire FEMA organization which is approximately 25,000 desktops. The desktops shall support Government security standards, provide per user profiles, support Microsoft Windows Desktop and Linux Operating systems, and be capable of running any application that can run on these operating systems on a physical desktop.

4.2.7 Office Application Services – The reseller shall provide the Microsoft Office Suite as SaaS to support the FEMA enterprise deployment of the Office Suite but in an easier to manage deployment in a FedRAMP High secured cloud.

4.2.8 Application Performance Monitoring Service –The reseller shall provide an application monitoring service that can support 100% predictive and holistic monitoring of the FEMA IT application infrastructure for FEMA business application processing. This includes detection and diagnosis application performance problems to maintain an expected level of service, translation of network performance metrics into business meaning, and translation of data and metrics on different aspects of the application. It also includes actionable monitoring results for the management of performance and availability of FEMA’s applications that diagnose application performance problems and identify actions that can be taken to resolve the issue.

4.3 Software as a Service – The Contractor shall provide access to at least one offering of each of the following services. If the services are not FedRAMP authorized at time of offering, authorization status and roadmap to authorization shall be provided at time of offering, as appropriate.

4.3.1 CloudCheckr (Brand Name or Equal) – The reseller shall provide CloudCheckr or equal as SaaS to support FEMA cloud hosting management, including collection, analysis, and synthesis of FEMA usage data for all services provided by the cloud. The Contractor must provide a cost management solution either on-prem or SaaS that can aggregate costs of the major CSPs. The solution used for cost management must provide the following characteristics: support of multi- account views, recommendations for cost savings, Reserve Instance or Savings Plan recommendations across a multi-account environment for applicable CSPs, supports ability to create spending plans and alerts, supports single-sign (SAML), provides monitoring functions for server performance, and provides information on the status of compliance with industry benchmarks such as CIS Level I and Level II.

4.3.3 Twilio (Brand Name) Third Party Messaging Service – FEMA requires Twilio, a three-tiered authentication mechanism to validate and verify an end user’s identity. The authentication shall be done through the (1.) username, (2.) password, and a (3.) security code transmitted to the user via text message, email, or voicemail.

5. Contract Administration The contractor shall directly provide contract administration in support of the IaaS / PaaS / SaaS services. Such professional services may include, but are not limited to:

a) Provisioning of accounts, including collection of FEMA user information; Management of the Master Payer Account, organization of the accounts in the console and reporting/analysis tools.

Regarding AWS, specifically, FEMA will utilize AWS Organizations to manage self-service items. The Contractor shall integrate with the FEMA organization structure as it relates to AWS.

b) Contractor shall provide the ability for FEMA to purchase Reserved Instances and Savings Plans in the Master Payer Account AND linked accounts.

c) Collection, analysis, and synthesis of Agency usage data for all services provided by a contractor, whether non-labor or professional, to ensure configuration and accurate billing through the CloudCheckr (or equal) tool, when compatible with a CSP, and a functionally similar tool with other CSP that are not compatible with CloudCheckr (or equal).

d) Contractor shall provide direct access to a dedicated payer account.

e) Contractor shall directly perform, without relying on third parties, the billing, reporting, and operational support as identified in this Description of Requirement.

6. Transition Phase

6.1 Phase-In Transition Period – The purpose of this phase is to permit a transition that is seamless to the Government. It is expected that all existing FEMA Cloud Hosted Services will be moved to the control of the new service provider no later than 60 calendar days after the start of the order, without interruption to FEMA production, and behind the scenes of day-to-day production. Upon award of the first order, the Contractor shall proactively and directly work and cooperate with the Government and incumbent contractor(s) and cloud hosting service companies to move FEMA Existing Cloud Services fully onto the new award. The Contractor shall also provide the Government with weekly written status reports of accounts until all have transitioned.

6.2 Phase-Out Transition Period – Not less than 60 days prior to BPA expiration or cancellation, the Contractor shall provide a Transition Plan for transitioning work from the current BPA and orders to a follow-on contract. For planning purposes, the Contractor shall assume a phase-out period of up to 60 days prior to the end of the BPA term. During this phase-out period, the Contractor shall assist the CO or their designee and the incoming Contractor to ensure a seamless transition with no adverse effect on performance. This Transition Plan shall include all requirements or other needed items for the seamless transition to occur. Including:

a) Coordination with Government representatives.

b) Review, evaluation and transition of current support services.

c) Disposition of the Agency’s current inventory of services and accounts to coordinate transition to a successor. This includes a complete description of services, location, data, etc. to facilitate the transition of the Agency’s services.

d) Budgetary estimates for cloud hosted services during the last 60 days of the BPA/order for the Government’s planning purposes, only.

In accordance with the Government-approved Transition Plan, the Contractor shall assist the Government in planning and implementing a complete transition from this BPA to a new vehicle and/or contractor. The transition support shall include formal coordination with Government staff and successor staff and management.

7. Acronyms and Definitions

7.1 API – Application Programming Interface

7.2 CCHS – Commercial Cloud Hosting Services – A FEMA program within the Enterprise Cloud Services branch that manages the services provided by the Commercial Cloud Service Providers to the organization.

7.3 CSP – Cloud Service Provider – a commercial entity that provides for virtualized IaaS and PaaS resources which are launched and managed via a programmatic interface to support fully automated CI/CD pipelines and system environments. It is a consumption based on-demand cloud hosting model that is billed in increments no larger than one hour.

7.4 CI/CD Pipeline – Continuous Integration/Continuous Delivery Pipeline – an automated method of software and infrastructure integration, testing, and deployment that is accomplished through programmatic interfaces. The pipeline usually includes several system environments including development, functional testing, unit testing, performance testing, pre-production, and production. The programmatic scripting is based on triggers and testing tools to validate the new code and move the code through the pipeline from one environment to the next until it has passed all the tests and is deployed to production.

7.5 DHS – Department of Homeland Security

7.6 GUI – Graphical User Interface

7.7 IaaS – Infrastructure as a Service – a virtualized set or resources that are programmatically instantiated and managed by the customer from the operating system through to the application layers.

7.8 IOPS – This is the input/output operations per second used to measure disk performance.

7.9 PaaS – Platform as a Service – a virtualized set or resources that are programmatically instantiated and managed by the customer from the operating system through to the application layers.

7.10 SaaS – Software as a Service – a virtualized set or resources that are programmatically instantiated and managed by the customer from the operating system through to the application layers.

7.11 Services - The term “services,” in the context of this document, does not mean services performed by personnel. It is service analogous to telephone, cable, electrical, or cell phone service. There are no personnel involved in these services other than commercial cloud hosting resellers provisioning cloud accounts and providing data reports to verify CloudCheckr (or equal) billing. It is a compute service function that allows FEMA to build and host system in a virtualized public cloud infrastructure environment.

7.12 True On-Demand Commercial Cloud – a Commercial Cloud that features completely virtualized infrastructure and services where the customer has direct unfettered programmatic access to launch and manage virtual resources on-demand in real-time. The underlying hardware resources are managed by the CSP and always have sufficient hardware to deliver the performance specified by the service offerings. It is a consumption-based model where each service is billed in increments no larger than one hour.

7.13 FEMA – United States Citizenship and Immigration Services