FY25 HSI FL LIMS_PWS (draft)V1.pdf

HSI Forensic Laboratory Information Management System (LIMS)

ICE HSI Forensic Laboratory Library Information Management System

(LIMS) Performance Work Statement

(PWS)

December 23, 2024

Version 1

U.S. Immigration and Customs Enforcement (ICE)

Office of the Chief Information Officer (OCIO)

HSI Forensic Laboratory Systems O&M Support Services

Table of Contents

1.0 Requirement Background

2.0 Technical Landscape

2.1 System Overview

2.2 Hosting Environment

3.0 Scope of Work

4.0 Tasks

4.1 Task One –Support and Contract Oversight

4.2 Task Two – Database Management

4.3 Task Three – Operational Support

4.3.1 Support the Production Environment

4.3.2 Performance Tuning

4.3.3 Configuration Management

4.4 Task Four – System Security/ISSO Support

4.5 Task Five – Queries

4.6 Task Six – Reporting

4.7 Task Seven – System Enhancements/Maintenance

4.7.1 Tier 1 Support

4.7.2 Tier 2 Support and Tier 3 Support

4.8 Task Eight – Establish and Support of Interfaces and Data Imports/Exports

5.0 Team Resources

5.1 Software Developer Systems Junior (Key Personnel):

6.0 Project Deliverables

6.1 Transition-In Activities

6.2 Transition-Out

6.3 System Lifecycle Management (SLM) Deliverables

6.4 Quarterly Progress Report

6.5 Certification and Accreditation (C&A) Documentation

6.6 Quality Assurance Surveillance Plan

6.7 Monthly Report

6.8 Deliverables Table

7.0 Performance Standards

7.1 Project Governance

8.0 Government Furnished Equipment and Information

8.1 Remote Access

8.2 Use of Government Equipment

9.0 Place of Performance

10.0 Closeout

11.0 Section 508 Requirements

The purpose of this procurement is to provide Operations and Maintenance (O&M) Systems Support Services for the Department of Homeland Security (DHS) /Immigration & Customs Enforcement (ICE) / Homeland Security Investigations (HSI) / Forensic Laboratory (FL) Library Information Management System (LIMS). The HSI Forensic Laboratory requires Information Technology (IT) support to maintain their applications and business processes in support of the HSI Forensic Laboratory mission.

1.0 Requirement Background

ICE is the principal investigative arm of DHS and the second largest investigative agency in the federal government. ICE's primary mission is to promote homeland security and public safety through the criminal and civil enforcement of federal laws governing border control, customs, trade, and immigration.

The HSI Forensic Laboratory is the only U.S. crime laboratory that specializes in the scientific authentication and research of travel and personal identification documents and related issues. The HSI Forensic Laboratory provides a broad range of forensic, intelligence, and investigative support services for ICE, DHS, and many other U.S. and foreign law enforcement agencies. The HSI Forensic Laboratory currently operates one legacy system, Laboratory system, Laboratory Information Management System (LIMS) to conduct normal operations.

LIMS, a commercial-off-the shelf (COTS) case management tool, assists the HSI Forensic Laboratory in conducting a variety of activities, including recording and tracking evidence and preparing reports. This enterprise software automates the natural steps in a latent print workflow process which standardizes documentation practices among examiners and provides overall consistency throughout the department.

2.0 Technical Landscape

Section 2.0 provides a high-level overview of the current, on premises, systems architecture, technology, and environments.

2.1 System Overview

Hardware Boundary - The production environment is hosted in the ICE Cloud AZURE Government service offering. We currently have two cloud servers for production and two for test and development.

Disaster recovery is covered by zone redundancy.

Software Boundary - The system is implemented using the following:

• Virtual Servers - Microsoft Windows 2022

• Databases - SQL Enterprise edition hosted on one of the two virtual servers

Both the SQL and Windows operating systems are maintained at the highest level of patching with the latest service packs installed and implemented. Monthly and quarterly scans are used to verify that all platforms are current for software and patching.

Database Boundary - The system is configured with one (1) copy of the production database with snapshots and backups making it easily replicated in case of a catastrophic zone or region failure in the ICE Cloud infrastructure. Both the testing and development configurations have unique database systems separate from the product units and do not share information with each other or with the main production databases systems. All databases are scanned monthly during the monthly security scans to verify that each is current on software and patches.

All databases are within the system certification boundary, and each is verified by the ISSO to ensure they meet all security requirements.

Outside the Boundary - The system boundary does not include client (workstations and printers) and infrastructure devices (routers, switches, firewalls, etc.), which are used by the HSI Forensic Laboratory staff. Client and infrastructure devices are covered under other associated security packages and are managed by other ICE organizations and are outside the management and financial control of system resources.

Diagram 1: System Boundary

Diagram 2: Network Diagram / System Overview (note: production will duplicate the below diagram)

2.2 Hosting Environment

• Support includes but is not limited to O&M service for hardware/software, network and security infrastructure, application support, and project management as needed. Support for infrastructure as requested such as build/deploy servers in the Azure ICE Cloud host environment, cloud storage area network, LDAP/AD servers, Load balancer(s), etc. Support is provided 24x7 and includes both below services as described below.

• Service includes initial system installation, up/down monitoring, and touch labor support. If there are failures that interrupt basic services a ticket will be opened, and the owner notified there is a problem.

• Service includes operating system (OS) installation and configuration, version control, OS management, storage management and backup and restore, and database and middleware support.

Table 1: Operational Service Levels

Level 1 Level 2 Level 3 Basic Managed Application Acquisition, configuration, connectivity, virtual machine configuration, PM, accounting and chargeback, inventory control on GFE

Virtual machine provisioning and configuration Maintain Patch and Version Control Operating System (OS) management Storage management and backup and restore operations Database and middleware support

Database and Middleware support Application porting, install, and migration when needed Quality Control and System Assurance; Security C&A Section 508 support

Characteristics Characteristics Characteristics Monitoring of virtual machines (up/down, start/stop timing Basic Infrastructure Secured network port to WAN on all 3 LANs

Tools and processes to manage VM’s via metrics Set service levels for system uptime within control of this contract Support DR and COOP levels of service

Integrated tools to manage Application uptime Application support integrated with infrastructure support Assured migration of applications as needed

3.0 Scope of Work

ICE requires all FL applications to be developed and maintained in an operational and efficient manner to ensure the highest level of support for the user community. O&M support includes adaptive maintenance, perfective maintenance, corrective maintenance, and preventative maintenance to include enhancements to any of the systems and any requisite updates to the system documentation as requested by the Government. O&M support includes modifications to the application due to changes in the larger HSI mission, application configuration changes, security changes, infrastructure changes, and minor regulatory or agency changes. The Contractor shall provide personnel to support the O&M requirements of the laboratory’s LIMS systems as well as any environment configuration elements in the ICE GovCloud infrastructure. The Contractor shall provide the support in accordance with existing ICE OCIO development and operations and maintenance policies in place for ICE applications. This includes:

• The use of HashiCorp’s Terraform product for infrastructure provisioning.

• The use of Red Hat’s Ansible for post provisioning and configuration of applications.

• The use of HashiCorp’s Vault product for management of application and infrastructure secrets.

The use of Cloud Bees Jenkins for orchestration of application pipelines.

• The use of Nexus for storing repository artifacts and objects required for delivery or functionality of the LIMS application.

• The use of GitHub for storing all code and scripts associated with FL applications.

• The integration of all tagging and cloud usage best practices with the development and deployment of the LIMS application.

• The coordination of any cloud products or service configurations with the Government ITPM or

Cloud Team to ensure adherence to best practices.

• The use of FedRAMP High certified products and services (any moderate or low services must be approved by IAD prior to use or implementation).

• The Contractor will shall utilize the JIRA application to track and document both development and operational tasks and activities and to request support from the Application Services Branch (ASB) Cloud Team.

The Contractor shall utilize the Confluence and/or ELMS application to document and store any application logical design, architectural design or technical schematics related to any of the FL applications.

Additional detail on the level of System Administration support required for the HSI Forensic Laboratory hosting environment is documented in the PWS Section 4.0.

• Adaptive – modifying the system to cope with changes in the application platform.

• Perfective – implementing new or changed user requirements which concern functional enhancements to the application platform.

• Corrective – diagnosing and fixing errors, possibly ones found by users.

• Preventive – increasing software maintainability or reliability to prevent problems in the future

The Contractor shall work seamlessly with other project teams, OCIO, and ICE Program Staff across the organization tasked with executing the Agency’s mission. Evolving and sometimes complex environmental factors may influence day-to-day workflow. The Contractor shall be required to collaborate with staff outside OCIO and the HSI Forensic Laboratory program, as well as interact with other vendor teams supporting ICE, OCIO or the HSI Forensic Laboratory to foster a collaborative working environment. The laboratory has strict accreditation standards which will impact priorities with regards to cost, schedule, and resource accountability.

The Contractors shall maintain, manage, operate, and sustain supporting services and architectural solutions as part of agile sustainment. This includes adjusting to and meeting changing requirements and expectations. The O&M Systems Support Service team will follow the Agile software development methods, based on iterative and incremental improvements. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that promotes foreseen interactions throughout the enhancement cycle.

All system enhancements performed will follow agile principals as defined by the Government. This includes formal planning, grooming, reviews, collaboration, and commitment to deliverable size and timeline.

All ICE systems shall comply with the following guidelines and regulations:

o DHS Acquisition Management Directive 102-01 Handbook o ICE Enterprise Systems Assurance Plan o ICE System Lifecycle Management (SLM) Handbook o ICE Technical Architecture Guidebook o ICE Technical Reference Model (TRM) (Standards Profile)

The Contractor shall identify any hardware, software, and/or licenses required for its proposed solution. The Government is prepared to provide any hardware and software items that are included within the ICE TRM that would reasonably be utilized by Contractors Offerors for the system development. Test and evaluation tools listed within the TRM are not provided as Government Furnished Equipment (GFE).

o 4300A DHS Information Security Policy o 4300A Sensitive Systems Handbook

The following documents are applicable to understanding the target ICE/HSI systems:

• International Information Systems Security Certification Consortium (ISC2) Standards

• National Industrial Security Program Operating Manual (NISPOM), February 28, 2006National

Institute of Standards and Technology (NIST) Computer Security Resource Center (CSRC)

• Guidelines

• Special Publications o Standards

• NIST Special Publication 800-37, Guide for the Certification and Accreditation of Federal

Information Systems

• Federal Information Processing Standard (FIPS) 199

• Federal Information Security Management Act (FISMA), November 22, 2002

• Federal Information Technology Security Assessment Framework (FITSAF), November 28,

• Federal OMB Circular A-130, Management of Federal Information Resources

• Federal Privacy Act of 1974 (As Amended)

• Federal Records Act

• DHS Management Directive (MD) 4300.1, Information Technology Systems Security, November

03, 2008DHS MD Volume 11000 – Security

• DHS Office of Chief Information Officer (OCIO) E-Government Act Report 2008

Please note that if newer versions of these documents are officially released, the Contractor shall comply with the updated versions within the timeframe established by the Government.

4.0 Tasks

The Contractor shall provide qualified, experienced personnel to deliver support for the continued System Maintenance, Operations and Services tasks in support of the LIMS system. The contractor shall include fixes to ongoing system and future changes to maintain a robust system during evolving mission needs required by DHS, ICE OCIO and the System Owner. This requirement includes the tasks, described in the PWS Section 4.0.

4.1 Task One –Support and Contract Oversight

• Provide supervision of the contractor IT technical staff.

• System Administration Services for hardware, software and GovCloud support

• Develop documents, summaries, written reports, and presentations for use by the HSI Forensic

Laboratory Program Managers, OCIO and Government personnel.

• The Contractor shall provide monthly reports documenting the HSI Forensic Laboratory system integrity, security vulnerabilities, security violations, and all areas of concern.

• Present briefings to Government personnel as requested.

• The Contractor shall provide working papers demonstrating continued, on-going efforts to maintain a Systems Engineering Lifecycle (SELC) Plan as required by Government policy and procedures.

• The Contractor shall provide updates to all System Lifecycle Management (SLM) documents as the system is altered or enhancements are released.

• Attend ICE and DHS Change Control Board (CCB) for each release, as needed.

• Analyze data integrity and consistency to obtain a quantitative basis for decision making and resource allocation.

• Develop strategies to exploit the various internal and external data sources to refine the intelligence analysis process as well as implementing process improvements and re- engineering methodologies for continued modernization of the system.

• Populating and maintaining a master backlog. Action items shall include any issues and risks identified.

• Perform configuration management oversight, including configuration identification, change control, status accounting and verification.

• Participate in collaboration activities with Government technical and non-technical working groups.

• Ensure that technical documentation is complete and up to date.

• Provide subject matter expertise as needed in areas of data ingestion, data source connections, initiatives, and enhancements.

• Attend and speak for the O&M team at meetings (e.g.… Integrated Project Teams (IPT), briefings, etc.) as requested by the Government.

• The Contractor shall provide the support in accordance with existing ICE OCIO development and operations and maintenance policies in place for ICE applications.

• The Contractor shall use HashiCorp’s Terraform product for supporting the automation of infrastructure provisioning via Infrastructure as Code (IaC).

• The Contractor shall use Red Hat’s Ansible for post provisioning and configuration of applications as applicable.

• The Contractor shall use HashiCorp’s Vault product for management of application and infrastructure secrets as applicable.

• The Contractor shall use Cloud Bees Jenkins for orchestration of application pipelines as applicable.

• The Contractor shall use Nexus for storing repository artifacts and objects required for delivery or functionality of FL applications as applicable.

• The Contractor shall use GitHub for storing all code and scripts associated with FL applications as applicable.

• The Contractor shall ensure integration of all tagging and cloud usage best practices with the development and deployment of any cloud-based FL applications.

• The Contractor shall manage coordination of any cloud products or service configurations with the Government ITPM or Cloud Team to ensure adherence to best practices to include cloud cost control and reporting.

• The Contractor shall use FedRAMP High certified products and services (any moderate or low services must be approved by IAD prior to use or implementation).

• The Contractor shall utilize the JIRA application to track and document both development and operational tasks and activities and to request support from the Application Services Branch (ASB) Cloud Team.

• The Contractor shall utilize the Confluence and/or ELMS application to document and store any application logical design, architectural design or technical schematics related to any of the FL applications. The Contractor shall evaluate and recommend to the ICE FL Program potential cloud products or services that will reduce cost(s), boost performance or provide more robust availability/recovery over existing FL application architectures. These recommendations will be informative in nature and the FL Program will provide final approval for any changes in the application architecture.

4.2 Task Two – Database Management

The Contractor shall support database updates to the data store. This includes database structural changes to support system enhancements and defect corrections and the writing of database scripts to update or query information in the database. This includes maintenance of the testing and training environment of each of the systems. The Contractor shall support ad-hoc queries as requested by the program office and/or perform data analysis as requested.

• The Contractor shall support the HSI Forensic Laboratory databases and conduct maintenance.

• The Contractor shall evaluate and recommend to the ICE FL Program potential cloud database products that will reduce cost(s), boost performance or provide more robust availability/recovery over existing Relational Database Services (RDS) or Infrastructure as a Service (IaaS) implementations supporting FL application databases. These recommendations will be informative in nature and the FL Program will provide final approval for any changes in the application architecture.

• Provide technical consulting in the definition, design, and creation of a database environment.

• Provide RDS or other database product(s) that may be applicable to FL application Databases and associated recovery strategies for FL applications databases in the cloud. The Contractor should include the use of storage replication and storage lifecycle policies within the CSP’s environment and the use and implementation of multi-availability zone and multi-region replication strategies as well. Conduct trend analysis, statistical methods and advanced technical research techniques to develop strategies based on the Government’s requirements.

• Provide technical assistance in uploading data, scripting, report development and the movement of records between the interface partners, and open-source databases.

• Improve applications to help solve database solutions to business problems, data architectures, database management system facilities and capabilities, and the operation and tuning of databases.

• Maintain databases with respect to access methods; access time; batch processes; device allocation; validation checks; organization; protection and security; and documentation. This includes maintenance of database dictionaries and integration of systems through database design.

• Support and develop new scripts as required that support data collection, data conversion & analysis and metrics processing.

• Install and upgrade the database server and application tools as needed.

• Allocate system storage and planning future storage requirements for the database system.

• Modify the database structure, as necessary.

• Enroll users and maintain system security. Ensure compliance with database vendor license agreement.

• Control and monitor user access to the database.

• Monitor and optimize the performance of the database.

• Plan for backup and recovery of database information.

• Maintain and verify archived data from daily backups.

• Support back up and restoration of databases and conduct yearly contingency testing with the ICE

OCIO Disaster Recovery group.

• Generate various reports by querying from database as needed.

http://en.wikipedia.org/wiki/Installation_(computer_programs) http://en.wikipedia.org/wiki/Planning http://en.wikipedia.org/wiki/Computer_security http://en.wikipedia.org/wiki/License_agreement http://en.wikipedia.org/wiki/License_agreement http://en.wikipedia.org/wiki/User_(computing) http://en.wikipedia.org/wiki/Program_optimization http://en.wikipedia.org/wiki/Backup http://en.wikipedia.org/wiki/Archive

• Generate and maintain the HSI Forensic Laboratory LIMS systems data dictionaries.

• Provide troubleshooting and support maintenance of each database application and provide reporting.

• Create and maintain application user accounts and provide technical support in accordance with

DHS security regulations.

4.3 Task Three – Operational Support

As part of daily work activities, the Contractor shall provide IT operational support which includes support of interfaces and data sources, database support, reporting support, system tuning, and system administration activities, which shall include monitoring and compiling data statistics for batch and interface production processing; and identifying corrupt files or processes. In the addition, the Contractor will shall complete and maintain, on periodic basis, database security findings for maintaining accreditation and perform database administration duties of existing system including backup and recovery.

In addition, the Contractor shall provide system administration activities to include regular monitoring of system resource utilization, disk storage utilization, identification of corrupt files or processes, system archiving, data archiving, installing operating system/software updates/versions and performing application backups; correcting flaws in software applications that escaped detection during development and testing of the system, or that have been introduced during previous maintenance activities; and improving software attributes such as performance, memory usage, and documentation.

ICE requires all applications to be maintained in an operational and efficient manner to ensure the highest level of support for the user community. Application solutions and services will need to be able to intercommunicate with other systems, solutions, and services within the same environment. These interoperability requirements will be explicitly documented in release requests.

Integrate federal cyber security requirements to include, but not limited to the DHS 4300A Sensitive System Policy, applicable National Institute of Technology (NIST) 800 series documentation, and the Federal Information Security Management Act (FISMA) of 2014 into technical solutions and provide strategies to meet on-going operational security controls in areas such as security patching, configuration management, authentication, and security monitoring.

4.3.1 Support the Production Environment

Gov Cloud, Infrastructure as a Service (IaaS) is a cloud-based on-demand service, which is where the Contractor shall do most of the work. The Government will own the construction of the solution(s) deployed and/or installed into the application platform, which uses IaaS as the foundation; (deployment to and supporting of) the application platform will be the responsibility of the Contractor. Any changes to “production” shall be implemented (including applications, COTS/GOTS, patches, supporting application, and service pack(s) updates) with the express written approval from ICE CCB and DHS ICCB, as required.

The Contractor shall deploy and maintain standard runtime topologies as per best practices. The Contractor must make available all topologies /topology decisions for each environment for ICE review.

Since the Contractor is explicitly responsible for application platform availability and performance, the ultimate topology and non-function configuration resides with the Contractor. The Contractor shall apply product updates and/or maintenance/patches, to keep the platform current with all necessary requirements.

Such updates must be validated for functional and non- functional reliability and coordinated with the Government.

Since this application is in production, changes to the code where application software change requests are required: i.e. ‘break/fix, System Change Requests, etc.’ situations are required to adhere to ICE change control practices. The Contractor shall perform all product installations and configuration as per best practice guidance supplied by product development team’s or vendor’s documentation, as applicable.

The production application environment is hosted in the ICE Cloud and will be controlled and managed by the Contractor to the degree that Cloud Hosting Policy allows (limited access to networking, identity, and access management, CSP marketplace and other related services may result in the Contractor having to coordinate with the ICE Cloud Team to configure or make available certain cloud services). Code migrations must follow the code progression governance policies set out in testing. Release candidates ready for pre-production are required and the Contractor shall execute production deployments once the necessary approval is granted. There are various pre-production environments necessary to support ICE.

Primary users of the pre-production application platform will be the designated testers, trainers/trainees or operations team, which will require support access granted by the Contactor.

The Contractor shall support two (2) categories of technical services for the application function(s): 1) product installation/deployment and 2) operational support. Traditionally at ICE Software Maintenance as Tiers 1, 2 and 3, however, only Tier 2 and 3 are applicable to this requirement. Additional detail is provided in the PWS Sections 4.7.1 and 4.7.2 below.

Conduct the following environment related changes/upgrades as needed:

• Patch incompatibilities – These represent systems changes that must be implemented in order to ensure that the application(s) software is compatible with impending ICE patch implementations.

• Infrastructure Upgrade – These upgrades may require a hosted support changes to the application to ensure that the system will continue to function as required under the infrastructure upgrade.

• Regulatory Changes – Where regulatory changes impose new requirements, hosted support change modifications may need to be implemented to accommodate these new requirements.

Interface Upgrade – When changes to application result in new interface requirements or changes to existing end points, the interfaces may require modification in order to ensure that the respective system continues to function properly.

• Provide support to Help Desk activities is performed in response to a user request or a notification that an application has an issue or an outage.

• Provide personnel to serve as an Outage Coordination and Incident Reporting point of contact.

• Investigate application issues, outages; provide root cause analysis reports; follow up on system bug reports.

• Review/view logs for success/fail messages.

• Create IaaS Service Desk Tickets to resolve server space issues, network problems and other engineering issues related to the support of the system.

• Ad hoc researching issues in question and writing responses to data calls.

• Review security bulletins and determine impact to the systems and supporting infrastructure.

• Respond to user emails and phone calls – question, problems.

4.3.2 Performance Tuning

The Contractor shall be required to conduct performance tuning of the system as operational needs arise.

The Contractor shall provide the program office with guidance for system performance improvements for a more stable and robust system. The Contractor shall analyze the current performance of the HSI Forensic Laboratory applications (production) to establish a baseline of performance.

4.3.3 Configuration Management

The Contractor shall conduct application-level configuration management for all O&M changes made to the system. The Contractor shall also handle all requests for changes to established baselines and configuration management thereof via the ICE approved SCR process, including the chartering and conducting of a system specific Change Control Board as required. The Contractor shall assign proper identification of all configuration items in accordance with agreed upon conventions.

4.4 Task Four – System Security/ISSO Support

The HSI Forensic Laboratory Systems O&M Support Services team is responsible for the ongoing support of the environment and application. All security patches must be implemented and reported, as completed. In addition, the vendor shall:

• Ensure LIMS complies with DHS/ICE standards for security, disaster recovery, and accreditation.

• Work with the assigned ICE Information System Security Officer (ISSO) to address system security requirements including reporting on patch management, and updating/providing security documentation, if requested.

• Support the Authority to Operate (ATO) renewals as required

• Support the Ongoing Authorization (OA) to operate as required

• Ensure system administration training is up to date for any resource managing the environments and provide verification to the assigned ISSO.

• Remediate security gaps and vulnerabilities per the direction of federal staff

• Participate in activities that will support the Authority to Operate (ATO) and Ongoing Authority

(AO) processes in the systems development lifecycle

• Adhere to the ICE change control board policy and procedures

4.5 Task Five – Queries

• Populate DHS databases or any other designated database as required.

• Access and perform research on designated automated intelligence databases for identifying information of interest to the customer, downloading the identified information to an appropriate medium and editing the information into format(s) to be specified by the customer.

• Conduct Ad hoc queries.

• Conduct Batching and Imports of DHS and other systems’ data.

• Write SQL database queries and modifications.

4.6 Task Six – Reporting

The Contractor shall support ad-hoc data requests, standard reports, field problem report fixes, analytical report requests and fixes in an array of formats, to include Crystal Reports. The reporting functions include, but are not limited to, the following:

• Creation of new standard reports

• Modification of standard reports

• Creation of ad-hoc reports

4.7 Task Seven – System Enhancements/Maintenance

• Monitor ongoing enhancements and maintenance of the database applications to include the addition of new modules and enhanced search capabilities.

• Perform database modifications as required.

• The Contractor shall perform modifications to each of the databases based on the following levels of priority:

The Government will provide the Contractor with a modification request instructing the Contractor on when to complete the change and the level of priority assigned. Below is a break-down of the priority structure for the HSI Forensic Laboratory modifications and the corresponding response time required by the Contractor.

Level 0: Modify system immediately (may require Emergency Change Request) Level 1: Modify system within twenty-four (24) hours (may require Emergency Change Request) Level 2: Modify system within forty-eight (48) hours (may require Emergency Change Request) Level 3: Modify system within seventy-two (72) hours – one hundred (100) hours

• The Contractor shall create and maintain a log of all modifications each database

• The Contractor shall prepare the modification for approval by the designated ICE Program

Manager.

• The Contractor shall make and document changes to the system by providing a monthly activity report to the ICE Program Manager.

• Ensures that documentation is complete and up to date.

4.7.1 Tier 1 Support

Generally, Tier 1 support will be provided by the Government. IT issues that cannot be resolved at the Tier 1 level will be forwarded to the Tier 2 or Tier 3 Support entities.

4.7.2 Tier 2 Support and Tier 3 Support

• All trouble issues that cannot be resolved by the Government are automatically turned over to the Contractor

• The Contractor shall report the status of the trouble ticket using the ICE approved tracking tool.

• This work includes, but is not limited to, activities such as: patching systems, running scripts, and minor fixes.

• The Contractor shall develop an application feedback loop, whereas systemic issues identified during common Tier 1, 2, and 3 escalation procedures are routinely evaluated and reviewed with the Information Technology Project Manager (ITPM) to assess the need for a System Change Request (SCR) in future release.

• Known issues that cannot be addressed through an SCR shall be documented and coordinated with the ICE Service Desk for inclusion as a troubleshooting script

• If the Contractor’s Tier 2 support cannot close the trouble ticket or perform the required tasks, then the trouble ticket shall be referred to the Contractor’s Tier 3 support.

• The Government will prioritize all outstanding bugs and SCRs and incorporate them into the backlog. These issues will be resolved by the Contractor once the Government identifies them as the highest priority.

• All maintenance activities that reach this level shall have an SCR opened and be reported using the ICE approved tracking tool.

• SCRs will be prioritized and agreed to by the authorized Government personnel and entered the ICE approved management-tracking tool.

• Prior to working on a system modification, the Contractor and the Government ITPM or PM shall designate the modification as minor, moderate, or major (see table below for classification).

• Known issues that cannot be addressed through an SCR shall be documented and coordinated with the Office of the Chief Information Officer (OCIO) Operations Tier 1 Help Desk for inclusion as a troubleshooting script within the Service Desk tool.

• The Contractor shall respond to all Software Maintenance Tier 3 trouble tickets in accordance with the service level agreements agreed upon by the Government. The Government will prioritize all outstanding bugs and SCRs and incorporate them into the development backlog.

These issues will be resolved by the Contractor once the Government identifies them as the highest priority. Software changes to applications are based upon the submission of an SCR.

Software changes are classified as minor, moderate, or major changes, where:

Type Change Estimated Effort Required Number of SCRs Minor Change Up to 40 Hours 30 Moderate Change Up to 250 Hours 16 Major Change Up to 1000 Hours 11

The SCRs above may be broken out into smaller SCRs if the need doesn’t arise for large system changes;

however, the overall estimated effort required will not change. The chart above reflects the maximum SCR hours for all contract periods for which the Contractor shall be responsible.

The number of SCRs is intended to cover the base and all option periods of this contract.

4.8 Task Eight – Establish and Support of Interfaces and Data Imports/Exports

The Contractor shall establish and support interfaces that feed into and out of the applications, this includes the import and export of data to and from business partners. The Contractor shall provide on-going support for interfaces that provide data to various components and reports as required.

5.0 Team Resources

The Contractor shall maintain adequate staffing levels while mitigating risks associated with absenteeism, staffing gaps and high turnover. The Contractor proposed staffing plan is incorporated into this Performance Work Statement as Appendix A. The Contractor shall ensure that the required level of support is maintained at all times. The Contractor shall ensure that adequate support personnel and all critical functions are present during all hours of operation. If for any reason the Contractor’s staffing levels are not maintained due to vacation, leave, appointments, gaps, turnover etc., the Contractor shall provide an e-mail notification to the Contracting Officer’s Representative (COR) in advance while otherwise providing a fully qualified replacement. The Contractor shall ensure that the support staffing levels remain at 95% at all times during hours of operation and ensure that the support staffing levels as identified as critical functions at 98% at all times during hours of operation. If the support staffing levels fall below 95% or critical function staffing levels fall below 98% and tasks are not being met at any time within a 30-day period of performance, the Government will withhold 10% of the fixed monthly rate for each 30-day period of performance staffing levels are short. If the staffing shortfall is corrected within 30 days, a penalty will not be applied to the next monthly rate where the requirement is adequately staffed.

Key Personnel (KP) is identified by the Government and documented in PWS Section 5.1. The Contractor shall be responsible for staffing the KP and to provide statements of qualifications for these individuals.

KP shall be current full-time employees, contingent hires will not be accepted as KP submissions. All KP shall have expertise in the Agile development methodology and experience using many of the tools included in the Tool Suite previously identified.

The Government expects that the KP either represent or demonstrate equivalent capabilities and qualifications in accordance with the following labor categories. The KP’s skillset shall meet or exceed the minimum requirements set forth in this performance work statement. Additionally, the KP shall be provided as the equivalent of at least one full-time personnel during the period of performance, and any KP changes shall be in accordance with the Homeland Security Acquisition Manual (HSAM) clause 3052.215-70 Key Personnel or Facilities (Dec 2003). Additionally, any proposed KP replacement’s resume must be submitted and demonstrate possession of a comparable skill and experience level.

Contractor personnel performing work under this PWS will require access to Sensitive but Unclassified, (SBU), For Official Use Only (FOUO) data. The Contractor shall not employ any foreign subcontractor participation.

5.1 Software Developer Systems Junior (Key Personnel):

This role provides support for implementation, troubleshooting and maintenance of IT systems. Manages IT system infrastructure and any processes related to these systems. Provides support to IT systems including day-to-day operations, monitoring and problem resolution for all client problems. Provides second level problem identification, diagnosis and resolution of problems. Provides support for the dispatch system and hardware problems and remain involved in the resolution process. Provides support for the escalation and communication of status to agency management and internal customers. Must possess experience in one or more systems, architectures, and associated hardware. This resource must have experience with DHS Data Center and GovCloud environments, configurations, and processes in order to effectively support the management of HSI Forensic Laboratory applications.

Minimum Education/Experience: B.S. degree in Computer Science or related IT field or equivalent education and work experience. Three (3) plus years of relevant experience to include at least 1+ year experience for each item below:

• Lead the planning, design, and implementation for network, computing and storage technologies.

• Experience in the development and customization of Crystal reports

• Experience with design documents, architecture plans, construction of infrastructure systems and applications.

• Technical lead in the design and installation of cloud based non-production and production systems.

• Manages existing hardware and software configurations. Documents solutions, including build and maintenance plans.

• Installs O/S releases and system upgrades; evaluates and installs patches; and resolves software-related problems.

• Maintains files and monitors system configuration to ensure data integrity.

• Proactively manages and supports all non-production and production environments.

• Provides technical support required to maintain configuration control files.

• Identifies and resolves errors or problems with existing programs.

• Proficient in cloud support services

• Knowledge in COOP planning and preparedness

• Subject matter expert and strong working level knowledge of: Network Architecture (WAN/LAN/Capacity Planning / Data Center Topologies), Database Technologies and Storage Architecture.

• Experience with SQL/MSQL

• Strong knowledge of operating systems including Windows/Linux.

• Virtualization technologies (AppLogic, VMWare and Hyper-V)

• Windows Operating Systems

• Web Server technologies (IIS, Apache)

• Open sources technologies

• Proficient in the use of Cloud Bees Jenkins, Nexus, Mavin, GitHub, JIRA, Confluence, YAML, JSON, and SAML technologies and products.

6.0 Project Deliverables

Specific deliverables related to each activity are outlined below.

6.1 Transition-In Activities

The primary focus of the Transition-In activities will be to gather and analyze information pertaining to new and release schedules, artifacts, and other documents. The purpose of this is twofold:

1. Identify any issues or gaps and develop a plan to promptly address them if applicable, and

2. Establish an integrated project management methodology and process structure that will guide quality assurance initiatives and assist in getting the early project deliverables completed within defined parameters.

The Contractor shall complete the Transition -In activities within 30 days after contract award. The activities shall include but not limited to the following:

1. Inventory and orderly transfer of all Government Furnished Equipment (GFE),

2. Coordination of work.

3. Onboarding of staff

The Contractor shall provide a Transition-In Plan that addresses the following activities, including adding resources and transition support responsibilities. Activities include supporting all tasks and deliverables required in this PWS by making available the personnel and documentation required to facilitate a successful Transition-In:

Week Description of Activities

• Identify POC for all administrative actions (in processing, training, etc.)

• Verify security clearance compliance with new personnel

• Receive PIV Cards at an ICE approved location

• Submit required documentation for IRMNET account request

• Complete required ICE In-Processing trainings & registration

• Request access to needed restricted areas

• Request elevated network access (if applicable)

• Security Clearance Actions and Status

• Transition-In Timeline

• Follow up and complete any In-processing items

• Orientation on policies and procedures

• Complete on-boarding training

• Become familiar with the environment

• Become knowledgeable of the tiers of network topology (LAN, WAN, DHS OneNet, ICEGov Cloud etc.), how they relate to one another & support levels

• Become familiar with the Change Management process & board requirements

• Review Service Level Agreements (SLAs) & escalation process

• Gain familiarity with the programs & administrative tools

• Review list of servers and how maintenance and patching are conducted

• Set-up development environments in coordination with existing teams

• Establish or gain access to established code repositories

• Sign (sub) hand receipt for all assigned equipment

• Review available Data Models

• Read through available documentation, analyze requirements, ask questions on instructional steps, with the intention of providing feedback and/or recommended changes to the COR

• Close out of any Transition-in activities

6.2 Transition-Out

The Contractor shall be responsible for the Transition-Out of all technical activities. The Contractor shall provide a Transition-Out Plan and a supporting project schedule identifying the support necessary to coordinate the transfer of all activities. Management activities for Transition-Out shall be incorporated into the Project Schedule. The Contractor shall transition systems with no disruption in operational services.

The Contractor shall be responsible for the technical activities, as part of the Phase-Out plan:

• Transfer of all Government Furnished Equipment/Property (GFE/GFP), inventory, software and licenses

• Transfer of documentation currently in progress

• Transfer of all software code in progress

• Coordinate transition with DHS/ICE IT personnel

• Baseline release schedules

• Fully support the transition of application requirements to the Government Project Manager

• Ensure that ICE exit clearance process is completed.

6.3 System Lifecycle Management (SLM) Deliverables

The Contractor shall provide SLM deliverables as required for System Maintenance Services projects. All documentation shall be prepared in accordance with the guidelines specified by the SLM.

6.4 Quarterly Progress Report

The Contractor shall prepare a quarterly progress and QASP report. These reports are due within fifteen

(15) calendar days after the completion of the quarter under review. The quarterly reports can be delivered via email and shall contain the following:

• Description of work accomplished (Accomplishments)

• Work planned for the following quarter (Planned Activities)

• QASP statistics

• Deviations from planned activities

• Open risks and issues

6.5 Certification and Accreditation (C&A) Documentation

The Contractor shall be responsible for maintaining and updating existing C&A artifacts to stay current with DHS/ICE and Federal requirements. These C&A updates will be required every three years unless a major change impacts security. The Contractor shall also be responsible for supporting the Information Systems Security Officer (ISSO) for any annual C&A activities, upon request (i.e. self-assessments, contingency plan tests, vulnerability scans, etc.).

6.6 Quality Assurance Surveillance Plan

The Contractor shall provide a Quality Assurance Surveillance Plan (QASP) 30 days after award. The QASP is the document used by the Government to evaluate Contractor’s actions while implementing the PWS. It is designed to provide an effective surveillance method of monitoring Contractor performance for each listed task in the PWS.

The QASP provides a systematic method to evaluate the services the Contractor is required to furnish.

The Contractor, and not the Government, is responsible for management and quality control actions to meet the terms and conditions of this requirement The role of the Government is quality assurance monitoring to ensure that the requirement and deliverables are achieved.

The Contractor shall be required to develop a comprehensive program of inspections and monitoring actions. Once the quality control program is approved by the Government, careful application of the process and standards presented in the QASP document will ensure a robust quality assurance program.

The Contracting Officer’s Representative (COR) will notify the Contractor of acceptance or required modifications to the plan. The plan shall be updated as changes occur and shall be submitted for review and acceptance by the Government.

The metrics may be dependent upon the details surrounding the Contractor’s own technical approach in meeting the Performance Standards in the PWS Section 7.0.

6.7 Monthly Report

The Contractor shall be responsible for delivering a monthly report via electronic copy to the ITPM, Contracting Officer, COR/ACOR. Report will be reviewed with contractor either in person live at the customers office or via an electronic meeting with the ITPM, Contracting Officer, COR/ACOR and product owner. The report should specifically cover current roadmaps, hurdles, risks, impediments and any metrics on work performed. This may include security statistics as far as progress or performance on reporting and maintaining steps for the ATO and/or Ongoing Authorization, scan results and remediations, number of JIRA tickets opened and closed, and any indices as required in the PWS Section 4.0.

6.8 Deliverables Table

The Contractor shall provide the following deliverables via email to the ITPM and COR, unless noted otherwise:

Deliverable Frequency Recipients

Any work first produced such as configuration scripts, security scripts, software written, user manuals, data models, interface control documents, user manuals, technical descriptions of the software and scripts, user operations manuals, system maintenance manuals, and anything else first produced under

As Required Electronic copy – ITPM &

COR/ACOR

Support and Contract…