AFRL Sample Tasks RFI_Posted 2025.04.16.pdf

AFRL DSRC TASK ORDER FOR HITS-U-III

GENERAL DESCRIPTION:

AFRL administers and operates two of the five DoD HPC DSRCs established under the auspices of the DoD HPC HPCMP. Located at Wright-Patterson Air Force Base, Ohio, the AFRL DSRC utilizes world-class, high-performance computers (HPCs), cutting-edge applications, and expert staff scientists to help the United States maintain its technological and military supremacy. The AFRL DSRC offers a full spectrum of computational capabilities for the Department of Defense (DoD) Science and Technology and Test and Evaluation communities to include powerful parallel processors, high-speed networks, applications software, and comprehensive storage solutions. The second DSRC under AFRL organizational management is the Maui High Performance Computing Center (MHPCC) located at Kihei, Maui, Hawaii. The AFRL DSRC operates the HPCMP enterprise Help Desk which provides assistance to the HPCMP user community. The AFRL DSRC is also the executive agent for centralized user authentication development and production services for the DoD HPCMP. AFRL DSRC also provides operational and administrative support for select HPC class machines for other customers. The contractor shall refer to current Competitive Technical Library documents to ensure they understand the overall breadth of coverage required to support the DSRC. The Competitive Technical Library is the baseline with which changes in resource requirements and performance shall be measured.

The AFRL DSRC also operates several specialized classified processing facilities.

Those facilities include Sensitive Compartmented Information (SCI) Facilities (SCIF) and Special Access Program Facilities (SAPF) where Top Secret, SCI, and Special Access Program (SAP) materials are processed. Contractors at the AFRL DSRC may be required to work with the appropriate DoD organizations to obtain and maintain a Top-Secret security clearance. All contractor employees must be able to obtain and maintain the security clearance level required to perform assigned tasks. All contractor personnel who may, at some point, be assigned to perform work in a SCIF or SAPF, must be eligible for a Top-Secret clearance based on a current T5 investigation.

Contractor personnel nominated for SCI or SAP access must meet eligibility criteria.

PWS 3.0 Requirements:

3.1 AFRL DSRC OPERATIONS SUPPORT

This PWS element includes all minimal support required for basic operations of the AFRL DSRC to be ready to deliver HPC capabilities and assist users. The requirements are somewhat independent of the number of HPC systems operating and HPC users being supported. The requirements are inclusive for a DSRC facility with appropriate operational elements of total electrical power and mechanical cooling capacity (not tied specifically to any HPC systems but representative of overall total capacity), basic DREN and SDREN networking enclaves with various operational workstations and servers, safety and security programs, and business processes such as procurement, asset/inventory control, change management, and configuration management.

Baseline AFRL DSRC operations support will ensure the following:

• Operation and delivery of HPC capabilities 24 hours/day, 7 days/week, 365 days/year (includes government holidays, although reduced staff expected during government holidays)

• Facilities, support infrastructure, and non-HPC systems remain operational and reliable

• Cybersecurity support is continuous and consistent with effective risk assessment and management so that network enclaves maintain appropriate authorities to operate

• Hardware and software maintenance agreements are up to date

• Configurations are managed effectively

3.1.1 Operations Monitoring

The Contractor shall provide support to monitor the AFRL DSRC facilities, infrastructure elements, and HPC systems for 24x7 operations. This includes all unclassified and classified areas.

3.1.2 Facilities Engineering Support

The Contractor shall provide support to AFRL DSRC efforts related but not limited to minor building alterations, environmental monitoring, electrical power and backup subsystems, and mechanical support systems. Services include planning through engineering designs and assessments, as well as other functions needed to execute facility projects.

3.1.2.1 Perform all activities to support planning, design and oversight of HPC facility changes/upgrades to ensure the unique HPC IT requirements are satisfied. This includes but is not limited to the addition/modification/replacement of mechanical and electrical plant equipment, raised floor systems, fire detection/suppression systems (such as VESDA, chemical fire suppression), building management systems (such as EMMS), security systems for HPC rooms.

3.1.2.2 Perform timely analysis of facility-related activities which impact HPC operations.

3.1.2.3 Plan, design, recommend, initiate, execute and/or support upgrades and optimization in power, cooling, space, physical security affecting facility operations, and infrastructure including backup power systems.

3.1.2.4 Support continuity of operation planning (COOP) activities to improve facility infrastructure resiliency and faster recovery from unscheduled outages, emergencies and/or disasters.

3.1.2.5 Coordinate facility planning and work, as needed, with local base Civil Engineering, Army Corp of Engineers, vendors, and/or other government stakeholders to ensure effective communication of facility activities. This coordination should include DSRC Government staff so that in the event of non-communication, the Government can assist.

3.1.2.6 Operate facility infrastructure in order to optimize HPC floor space utilization and systems availability and capacity.

3.1.2.7 Perform site preparation and coordinate facility requirements for environmental services, mechanical and electrical distribution and systems monitoring design/installation.

3.1.2.8 Technology Insertion (TI) site preparation must coordinate and be accomplished within time parameters of TI acquisitions. Typical site preparations are 90 days, from the time TI system specifications are provided until the site can accept delivery of the TI system(s) but are subject to change depending on TI orders.

3.1.2.9 Document and maintain configuration of facilities, including requirements, design documentation and construction drawings of past, current, and future facility projects.

3.1.2.10 Provide root cause analysis of facility outages and “near-miss” events which could have resulted in a facility outage or facility degradation.

3.1.2.11 The DSRCs operate and maintain separate buildings with environments at multiple classification levels. Success of the program is dependent upon the contractor’s ability to plan, design, recommend, initiate, execute and/or support upgrades in power, cooling, space, security, and infrastructure. At some DSRCs, certified contractor personnel will be required to operate sit down forklift trucks to support infrastructure upgrades; forklift operators will need to have a certificate on file with COTR approval and will be required to have and properly use personal protective equipment (e.g., hard hat, steel toed shoes, and eye protection). Contractors found to be operating unsafely will have privileges revoked until completing a new certification.

3.1.2.12 Provide trend analysis of electrical and mechanical usage and recommend options to reduce overall energy and water consumption and improve plant operation.

3.1.3 Cybersecurity Support for DSRC Operations

The baseline operational DREN, SDREN, and/or Above-Secret network enclaves in the AFRL DSRC requires authorization to operate and connect to the wide-area networks.

The Contractor shall assist the government cybersecurity staff in attaining and sustaining the appropriate authorizations to operate. Cybersecurity requirements related to DSRC operations are identified in section 3.7. Cybersecurity support in classified enclaves other than SDREN will be addressed in individual DSRC tasks.

3.1.3.1 Ensure compliance with the IC, SAP, and DSRC Computer Security (COMPUSEC) programs in accordance with Government Directives listed in paragraph 4.1.4.

3.1.3.2 Prepare for and participate in security audits or inspections using eMASS or current system to meet requirements for DoD 8500.10 current version, IC 503 current version, and/or JSIG current version, or any superseding guidance depending on authorizing official.

3.1.3.3 Implement and track status of all DoD IAVM and vendor security bulletins.

3.1.3.3.1 Identify and gather all relevant DoD IAVM (Information Assurance Vulnerability Management) and vendor security bulletins.

3.1.3.3.2 Use the required tracking system to monitor the implementation status of each bulletin.

3.1.3.3.3 Conduct a risk assessment for each identified vulnerability.

3.1.3.3.4 Implement mitigation measures for identified vulnerabilities and other relevant security guidance.

3.1.3.3.5 Regularly update the tracking system with the status of each implemented mitigation measure.

3.1.3.3.6 Conduct periodic reviews and report on the overall status of implemented IAVM and vendor security bulletins.

3.1.3.4 Inspect DSRC assets, including HPC systems maintained by 3rd party vendors, for compliance with cybersecurity/IA policy and accomplish corrective action. Provide results to ISSM weekly.

3.1.3.4.1 Develop an inspection checklist based on relevant cybersecurity and IA policies, such as those outlined in ICD 503 and JSIG, or any superseding guidance.

3.1.3.4.2 Inspect DSRC assets, including HPC systems maintained by 3rd party vendors, for compliance with the developed inspection checklist.

3.1.3.4.3 Document any non-compliance issues and corrective actions required.

3.1.3.4.4 Provide a weekly report to the ISSM (Information System Security Manager) detailing the results of the inspection and the progress of corrective actions.

3.1.3.5 Assemble and document the artifacts for RMF in the preparation of Approving Official Assessment and Authorization packages

3.1.3.5.1 Gather all necessary artifacts for the RMF (Risk Management Framework) process, such as the system security plan, security assessment report, and plan of action and milestones.

3.1.3.5.2 Utilize Xacta, eMASS (Enterprise Mission Assurance Support Service) or other tools to document and manage the artifacts in accordance with the RMF process.

3.1.3.5.3 Develop the Approving Official Assessment and Authorization packages, using the artifacts gathered and documented in step 1 and 2.

3.1.3.5.4 Review and finalize the packages, ensuring that they meet the requirements of the relevant RMF authorities, such as those outlined in ICD 503 and JSIG, or any superseding guidance.

3.1.3.6 Maintain a Plan of Action and Milestones (POA&M) for any non-compliant issues

3.1.3.6.1 Review the uploaded documents, such as the system security plan, security assessment report, and plan of action and milestones, to identify any non-compliant issues.

3.1.3.6.2 Prioritize the identified vulnerabilities based on their severity, such as CAT 1 (high), CAT 2 (medium), and CAT 3 (low) vulnerabilities.

3.1.3.6.3 Develop a timeline for implementing fixes for each identified vulnerability, taking into account the severity of the vulnerability and the resources available.

3.1.3.6.4 Regularly track the progress of the implemented fixes and update the POA&M at least quarterly.

3.1.3.6.5 Regularly report the status of the POA&M to the relevant authorities, such as the ISSM, and provide updates on the progress of the implemented fixes.

3.1.3.6.6 Utilize Xacta and eMASS (Enterprise Mission Assurance Support Service) tools to manage and document the POA&M, in accordance with the RMF (Risk Management Framework) process.

3.1.3.6.7 Review and finalize the POA&M, ensuring that it meets the requirements of the relevant RMF authorities, such as those outlined in ICD 503 and JSIG, or any superseding guidance.

3.1.3.7 Promote cybersecurity awareness among all Distributed Supercomputing Resource Center (DSRC) personnel by emphasizing the importance of maintaining a strong cybersecurity posture and adhering to relevant cybersecurity policies and procedures outlined in the Joint Security Implementation Guide (JSIG), Intelligence Community Directive (ICD) 503, and Committee on National Security Systems Instruction (CNSSI). Compliance with these standards, or any superseding guidance, guarantees the readiness and standardization of the civilian, military, and contractor workforce.

3.1.3.8 Track compliance with TASKORD and all other government directed cybersecurity bulletins.

3.1.3.9 Assist in debugging networking issues for customers at any classification level. Security at DSRCs with Above-Secret Systems: Vendor must consider DSRCs that support Above Secret, which may be more challenging and an increasing requirement as the program is moving forward into that arena.

3.1.3.10 To protect information in accordance with OPSEC and information security policies, the following tasks should be performed:

3.1.3.10.1 Conduct an OPSEC assessment to identify potential

vulnerabilities and threats to the information.

3.1.3.10.2 Implement information security policies and procedures to safeguard the information from unauthorized access, use, disclosure, disruption, modification, or destruction.

3.1.3.10.3 Train personnel on OPSEC and information security policies and procedures to ensure compliance and awareness.

3.1.3.10.4 Regularly review and update OPSEC and information security policies and procedures to ensure they remain current and effective in protecting the information.

3.1.3.11 Ensure proper handling of classified information following DODM

5200.01 Volumes 1, 2 and 3, along with all local room Standard Operating Procedures and Emergency Action Plans (EAPs).

3.1.3.11.1 Regularly reviewing and staying updated with the Department of Defense Manual (DODM) 5200.01, which provides guidelines for handling classified information.

3.1.3.11.2 Implementing and enforcing the Standard Operating

Procedures (SOPs) for the local room where classified information is stored or handled, to ensure that all personnel follow the same procedures for handling, storing, and disposing of classified information.

3.1.3.11.3 Regularly reviewing and updating the Emergency Action Plans (EAPs) to ensure that all personnel know what to do in case of an emergency involving classified information, such as a natural disaster or a security breach.

3.1.3.11.4 Conducting regular training and awareness programs for all personnel to ensure that they understand the guidelines, procedures, and plans for handling classified information.

3.1.3.12 Perform physical security functions in accordance with secure facilities operating policies and procedures; to include open/close duties for the rooms and to alarm and disarm as well. The contractor will not be responsible for the recall roster.

3.1.3.13 Adopt the DSRCs’ computer incident response plans and support incident responses in accordance with the Incident Response Plans (IRPs), along with IRPs for the required networks or authorizing officials.

3.1.3.13.1 Review and understand the DSRCs' computer incident response plans and the IRPs for the required networks or authorizing officials.

3.1.3.13.2 Implement the incident response procedures outlined in the IRPs, including the use of automated mechanisms to support incident response.

3.1.3.13.3 Provide incident response training to DSRC personnel consistent with assigned roles and responsibilities.

3.1.4 System Administration

For baseline AFRL DSRC operations, there are a number of non-HPC devices such as workstations, servers, repositories, and virtual communication equipment. The Contractor shall provide system administration support in keeping these systems operating reliably and securely.

3.1.4.1 Configure operating systems and associated supporting software and utilities to optimize system performance and system services. Perform an analysis of the benefits and risks of implementing available hardware and software updates prior to implementing changes. Coordinate with local ISSM lead to ensure compliance with DISA Security Technology Implementation Guides (STIGs), Security Requirements Guides (SRGs) requirements to include all manual STIG/SRG checks. Provide completed STIG/SRG checklist to local

ISSM.

3.1.4.2 Administer, manage and optimize system performance data. Utilize backups and other approaches to ensure continuity of operations. Conduct yearly backup tests.

3.1.4.3 Lead and conduct meetings with original equipment manufacturer (OEM) personnel as necessary to administer systems.

3.1.4.4 Maintain installed software and software licenses, except for those directly supported by the OEMs.

3.1.4.5 Ensure all software updates are maintained to include the vendor patch releases

3.1.4.6 Maintain classified and unclassified data storage systems.

3.1.4.7 Provide 2nd tier support to users for all computer systems.

3.1.4.8 Maintain utilization, availability, and operation and maintenance logs.

Conduct log reviews at least weekly.

3.1.4.9 Perform causal analysis when directed by the Government or when data indicate a trend requiring further analysis.

3.1.4.10 Provide cybersecurity RMF support on a continuous, risk-management based approach and during inspections to achieve and maintain Assess and Authorization or Assess-Only Authority to Operate (ATO) status.

3.1.4.11 Create and maintain the documentation of all systems, configurations, concept of operations, and details on the authentication to systems which are required to support RMF effort.

3.1.4.12 Perform activities necessary to ensure HPC systems and infrastructure systems, subsystems, and components are operating efficiently to support achievement of established levels of systems availability.

3.1.4.13 Operate and maintain local and program wide enterprise systems, subsystems, and components in accordance with HPCMP and DSRC documented procedures and processes. This includes coordination with the system vendors/original equipment manufacturers (OEMs) who directly maintain the base Operating System (OS) and configuration-specific software stack(s) for managing their delivered HPC systems.

3.1.4.14 Define, document, and execute recovery and continued operations plans as necessary.

3.1.4.15 Ensure system outages are scheduled and coordinated at least 7 days in advance, with host organizations to minimize customer and HPC user impact and that users of the systems are informed of events which could/will impact systems availability.

3.1.4.16 Promptly notify staff and customers of all scheduled and unscheduled system outages to ensure minimal disruption to customers and HPC users.

3.1.4.17 Provide reports and assessments of utilization, system availability, workload and throughput. Note that some of the information may be considered classified.

3.1.5 Network Engineering and Administration

For baseline DSRC operations, the DREN, SDREN and/or Above-Secret enclaves require engineering and system administration support for the various equipment items such as routers and switches. The Contractor shall provide network engineering and system administration for the network devices. In addition, in accordance with DoD HPCMP direction and JFHQ-DODIN Comply-to-Connect (C2C) Concept of Operations instructions, the Contractor shall administer/maintain the necessary hardware and software (currently Forescout) at the AFRL DSRC to continue enforcing C2C compliance on the DREN and SDREN.

3.1.5.1 Configure networks and associated supporting software and utilities to optimize systems performance and systems services. Perform an analysis of the benefits and risks of implementing available hardware and software updates prior to implementing changes.

3.1.5.2 The contractor shall coordinate naming conventions and naming services to meet DoD standards.

3.1.5.2.1 The contractor shall name routers, switches, and all network gear according to DoD Naming conventions.

3.1.5.2.2 The contractor shall name all network interconnects (end devices connecting to network gear) according to DoD Naming conventions.

3.1.5.2.3 The contractor shall name inter building connections IAW local Communication Group's guidance.

3.1.5.2.4 The contractor shall create and maintain documentation establishing the relation of critical connections to specific rack(s)/server(s).

3.1.5.3 Maintain utilization, availability and performance data, and operation and maintenance logs.

3.1.5.3.1 Network Monitoring:

• The Contractor shall regularly monitor the network to ensure it is running smoothly and to identify any potential issues.

• The Contractor shall utilize network monitoring tools to collect and analyze data on network utilization, availability, and performance.

• Data Collection:

• The Contractor shall collect and store data on network utilization, availability, and performance metrics.

• The Contractor shall identify trends, spot potential issues, and optimize network performance.

3.1.5.3.2 Log Management:

• The Contractor shall maintain detailed operation and maintenance logs of all network and network device maintenance activities, to include any changes made to the network, any incidents or issues that occurred, and any steps taken to resolve them.

• The Contractor shall keep these logs in a centralized location

• The Contractor shall regularly review and annotate the review to ensure they are accurate and up-to-date.

3.1.5.3.3 Reporting:

• The Contractor shall regularly generate reports on network utilization, availability, and performance data, and provide them to relevant stakeholders.

• The Contractor shall create and deliver reports that help identify trends, highlight areas for improvement, and provide insights into the overall health and performance of the network.

3.1.5.3.4 Network Auditing:

• The Contractor shall perform regular network audits to ensure that the network is secure, optimized, and compliant with industry standards and regulations.

• The Contractor shall audit for unauthorized devices, reviewing firewall rules, security compliance, and verifying that all network devices are properly configured.

3.1.5.3.5 Capacity Planning:

• The Contractor shall plan for future network capacity by analyzing current network utilization and performance data, as well as forecasting future network growth.

• The Contractor shall prepare and deliver strategies to ensure the network has sufficient capacity to meet future demands and can prevent performance issues caused by network congestion.

• The Contractor shall ensure all test equipment is operational in the TADE for testing purposes.

3.1.5.4 Perform causal analysis when directed by the Government or when data indicates a trend requiring further analysis.

3.1.5.4.1 The Contractor shall gather relevant network data from various sources, such as performance logs, error reports, and user feedback.

3.1.5.4.2 The Contractor shall correlate this data with external factors like recent changes in network configuration, software updates, or hardware installations.

3.1.5.4.3 The Contractor shall use analytical tools and techniques to identify trends and patterns in network performance metrics. This might include statistical analysis, machine learning algorithms, or visualizations to detect anomalies or recurring issues.

3.1.5.4.4 The Contractor shall perform a root cause analysis to determine the fundamental reasons for the identified issues. This involves using techniques such as the "5 Whys," fishbone diagrams, or fault tree analysis to trace problems back to their source.

3.1.5.4.5 The Contractor shall prepare detailed reports on the findings from the causal analysis, including the identified root causes, impact assessments, and recommended corrective actions.

3.1.5.4.5 The Contractor shall assist in debugging network issues or slowdowns for customers, maintaining a proactive stance to anticipate avoidable problems.

3.1.5.5 Provide up to 3rd Tier Support to users for all network systems.

3.1.5.5.1 The Contractor's Network administrators shall be in compliance with current DoD 8570 or DoD 8140.

3.1.5.5.2 The Contractor shall provide a qualified and knowledgeable back up for Tier 3 principal.

3.1.5.5.3 The Contractor shall ensure appropriate workload distribution among network administrators and provide quarterly the average number of tickets handled per support technician.

3.1.5.5.4 The Contractor shall respond to new tickets within 24 hours Monday - Friday.

3.1.5.6 The Contractor shall provide complete cybersecurity support for network devices ensuring they remain in compliance at all times coordinating (with all monitoring activities).

3.1.5.6.1 The Contractor shall regularly update IDPS to detect and respond to suspicious activity ensuring configuration is set to monitor all traffic entering and leaving the network.

3.1.5.6.2 The Contractor shall ensure that redundant paths exist for critical systems to maintain network availability even during attacks or failures.

3.1.5.6.3 The Contractor shall implement continuous network monitoring to detect and respond to unauthorized access or unusual activity in real-time.

3.1.5.6.4 The Contractor shall perform network off-peak vulnerability scans of no less than once monthly.

3.1.5.7 Maintain the network topology, intersystem connection, and data flow diagrams which support RMF activities and update at least semi-annually, or within 10 days of government direction.

3.1.5.7.1 The Contractor shall create and maintain a clear communication plan for regularly sharing updated diagrams with relevant stakeholders, including IT teams, security personnel, and government representatives.

3.1.5.7.2 The Contractor shall ensure that all enclave drawings and supporting documentation contains a version history, date of last update and summary of changes.

3.1.5.7.3 The Contractor shall create and update network Disaster Recovery diagrams semi-annually.

3.1.5.7.4 The Contractor shall include the diagram in the disaster recovery plan.

3.1.5.7.5 The Contractor shall have zero errors on Network diagram.

3.1.5.7.6 The Contractor shall verify through peer review and audit that all Network diagrams are correct and current

3.1.5.7.7 The Contractor shall update and submit, quarterly, the DREN diagrams for HPCMP CSSP MOA which must include Site name Physical address, Current review date, IDS, network device model numbers, Back-end connections and IP Addresses of network devices.

3.1.5.7.8 The Contractor shall update and submit, quarterly, the SDREN diagrams for HPCMP CSSP MOA which must include Site name Physical address, Current review date, IDS, network device model numbers, Back-end connections and IP Addresses of network devices.

3.1.5.8 Maintain all network devices in accordance with DoD policies (e.g.

IAVMs/STIGs/SRGs). Verification is completed via results of HBSS, RADIX and ACAS scans or current technologies adopted by HPCMP.

3.1.5.8.1 The Contractor shall ensure all Network gear is STIG complaint within 60 days of initial release.

3.1.5.8.2 The Contractor shall implement and test solutions in the TADE related to IAVMs, Juniper JUNOS, Arista EOS, Cisco IOS, Cisco NX-OS, Brocade OS within 3 days or first available maintenance window.

3.1.5.8.3 The Contractor shall implement a successful tested solution into production environment regarding IAVMs, JUNOS, Arista EOS, Cisco IOS, NX-OS, Brocade OS, within 5 days or first available maintenance window.

3.1.5.9 Develop and maintain a comprehensive network diagram for the AFRL

DSRC.

3.1.5.10 Develop and maintain a comprehensive dataflow diagram for the AFRL

DSRC.

3.1.5.11 Maintain the AFRL DSRC’s Firewall Action Request process and procedure to include detailed VLAN, router, and switch Access Control List.

3.1.5.12 Provide immediate notifications to government staff of any network degradation affecting local operations.

3.1.5.13 Assist in debugging network issues or slowdowns for customers, maintaining a proactive stance to anticipate avoidable problems.

3.1.5.14 Monitor and interpret trends to measure efficacy

3.1.6 Database Administration

The AFRL DSRC maintains databases for baseline operations support. This includes but is not limited to environmental monitoring data, service tickets, maintenance logs, etc. The Contractor shall provide database administration support for these systems.

3.1.6.1 The contractor shall perform database administration on databases local to the AFRL DSRC that require skills beyond those required for System Administration.

3.1.6.2 The contractor shall maintain installed database software and database licenses.

3.1.6.3 The Contractor shall utilize backups and other approaches to ensure continuity of operations.

3.1.6.4 The Contractor shall administer, manage, and optimize database data and database software.

3.1.6.5 The Contractor shall support Help Desk activities and user allocation and access tools.

3.1.6.6 The Contractor shall ensure that all database software patches are applied per cybersecurity requirements.

3.1.6.7 The contractor shall patch the systems at a minimum of every 30 days or within the patch cycle from the vendor, or whichever can be patched the earliest.

3.1.6.8 In case of IAVMs, TASKORDS, Situational Awareness Reports, or other directives, the contractor shall patch within the required timeframe.

3.1.6.9 The Contractor shall ensure all databases are compliant to DISA STIGs/SRGs and submit STIG/SRG checklist results to ISSM as directed by the government.

3.1.6.10 The contractor shall complete STIGS/SRGS in accordance with the HPCMP STIG Management SOP.

3.1.6.11 The contractor shall complete STIGs/SRGs yearly or within 60 days of the release of an updated or new STIG/SRG, whichever is sooner.

3.1.6.12 The contractor shall close out open STIG findings within the timeline in the HPCMP STIG Management SOP.

3.1.6.13 The Contractor shall provide cybersecurity support, to include creating and maintaining documentation of databases in support of sustaining and obtaining authorizations to operate.

3.1.6.14 The Contractor shall develop and sustain local database applications and tools utilized by the AFRL DSRC.

3.1.6.15 The Contractor shall develop and support database solutions that interface with multiple applications in support of authentication and authorization services.

3.1.6.16 The contractor shall thoroughly document the authentication and authorization processes that leverage databases at the AFRL DSRC.

3.1.7 Software Development

The Contractor shall develop, and support execution of specialized software used in AFRL DSRC operations. This includes but is not limited to applications to monitor facility environmental controls and automate operational processes. Software in this context includes, but is not limited to, job scheduling/workload management, container orchestration, and virtualization software which may or may not be provided on HPC systems by the vendor or original equipment manufacturer (OEM), as well as user authentication software. Such support includes developing, configuring, and maintaining the software for operational use. Introduced in FY2022, the HPCMP Workload Characterization (WC) system is used to aggregate and display performance and utilization metrics from HPCMP HPC systems and other systems deemed appropriate. Support includes developing, integrating, configuring, and maintaining the WC software, utilities/tools utilized to gather metrics, reporting tools such as dashboards and other data products, stored data, and user profiles/roles on the WC system.

Support also includes maintaining system administration guides, data management policies and other related documentation.

3.1.7.1 Develop, maintain and support software in accordance with:

• DISA Application Security and Development STIGs/SRGs.

• Industry best processes and practices known to reduce cost, schedule and performance risk.

• Software systems design and development based on systems engineering principles and processes to include:

• Software systems architected to support open system concepts, exploit Commercial off-the-shelf (COTS) computer system products and Government off-the-shelf (GOTS), and allow incremental improvements based on modular, reusable, and extensible software.

• Government and commercial software reuse opportunities before developing new software.

• Selection of software development tools and environments in context of the systems and software engineering factors that influence overall life-cycle costs, risks, and the potential for interoperability.

• Documented software measurement processes in planning and tracking the software developed/acquired.

• Provide source code scan results after each sprint cycle or as directed by the government.

• Assess and improve the software development process and the associated software product in accordance with contractor established software engineering practices.

• Government regulations and vendor agreements.

3.1.8 Maintenance Support

The Contractor shall provide support for AFRL DSRC hardware and software maintenance agreements related to facility equipment and supporting infrastructure (networks, non-HPC servers, etc.) used in routine operations. Note: Portions of the hardware and software maintenance are currently performed via OEM or third-party maintenance personnel under contracts acquired directly by the Government.

3.1.8.1 If applicable at the DSRC, maintain hardware, software and software licenses according to manufacturer’s specifications and agreements.

3.1.8.2 Schedule maintenance to ensure minimal disruption to customers.

3.1.8.3 Coordinate with OEM or third-party maintenance personnel as necessary to initiate and/or schedule maintenance activities. Ensure any third-party maintenance adheres to contractual requirements.

3.1.8.4 Ensure data on HPCMP systems is properly protected and secured.

3.1.8.5 Maintain report of maintenance status for all hardware and software items in a format that can be directly imported into the RMF control database.

3.1.8.6 Maintenance contracts handled by vendor are reviewed, updated, and renewed without loss of maintenance coverage

3.1.8.7 Upcoming maintenance is discussed with government with enough time to coordinate with operational users and general user notice will be sent 10 calendar days prior to maintenance.

3.1.9 Inventory Management Support

The Contractor shall assist the government in identifying and documenting equipment inventory for asset management. This contract does not handle Government Furnished Property (GFP); however, the Contractor shall provide support to the AFRL DSRC in identifying and documenting equipment inventory for asset management.

3.1.9.1 Support the Government by maintaining an inventory of all HPCMP and DSRC equipment in government-required formats/systems and provide inventory reports.

3.1.9.2 Support the Government in decommissioning and disposing of communications-computer systems no longer needed or serviceable in accordance with Government requirements. Update hardware and software inventory, as well as applicable infrastructure diagrams and system security plans.

3.1.9.3 Schedule and perform audits in support of Government property control and provide results to the Government.

3.1.9.4 Where applicable, actively participate in all configuration and change management programs.

3.1.9.5 Process all requests for loaned equipment in accordance with approved procedures.

3.1.9.6 Perform acceptance and documentation of new equipment deliveries.

3.1.9.7 Sanitize equipment to remove all data and support the Government in disposal according to local policies

3.1.10 Configuration and Change Management

Various system and subsystem configurations often change within the AFRL DSRC and effective management processes are executed. The Contractor shall support the configuration and change management processes by assisting in identifying and documenting configurations and associated changes.

3.1.11 Procurement Support

The AFRL DSRC has the ability to purchase various items needed to support routine operations. This includes, but is not limited to, office supplies and equipment, facility infrastructure items, and HPC infrastructure equipment, as well as hardware/software maintenance. The Contractor shall provide procurement/purchasing support for any AFRL DSRC requirements.

3.1.12 Continuity of Operations

Technical support is required to reconstitute critical operations, defined by the government, at alternative location(s) in the event of a disaster that causes the shutdown of AFRL DSRC operations at the primary site. The Contractor shall assist in establishing continuity of operations and disaster recovery for such events and shall support any exercises/simulations.

3.1.12.1 Ensure and document the capability to perform limited operations and recover quickly from events that damage critical communications-computer system equipment or destroy essential data.

3.1.12.2 Document the timeline of events, perform causal analysis and provide recommendation for prevention of unscheduled outages.

3.1.12.3 Perform planning and activities necessary to anticipate, respond to and resolve computer system outages to minimize impact upon the system users and determine actions which could prevent future occurrences.

3.1.12.4 Participate in contingency exercises and training by the host installation.

3.1.12.5 Test and evaluate recovery and continued operations plans.

3.1.12.6 Connect and coordinate with all the DSRCs to ensure that best practices are implemented across the enterprise.

3.1.12.7 The contractor shall assist with failover to provide continuity amongst the centers.

3.1.13 Technical Project Support

The AFRL DSRC may execute studies and projects related (but not limited) to facility upgrades, infrastructure changes, HPC capacity increases, and HPC system installation and integration. The Contractor shall provide technical support to AFRL DSRC studies and projects, whether at the local or enterprise level. This support shall include technology planning and assessments and engineering support during project execution.

3.1.13.1 Support the Government in strategic planning and capital planning.

Ensure alignment with the HPCMP and DSRC plans, goals and strategic direction.

3.1.13.2 Assist the Government in developing HPC and DSRC architectures.

Identify and document architectures and standards.

3.1.13.3 Coordinate all requirements with the Government project manager(s), customer(s), DSRC staff, and host installation support organizations, as appropriate.

3.1.13.4 Perform/coordinate studies and analysis, perform engineering design and prepare project proposals as directed.

3.1.13.5 Develop detailed cost estimates, activity schedules, and detailed integration plans.

3.1.13.6 Identify recommended modifications to facilities, communications infrastructure, networking, support systems, storage, software, and operating procedures required to successfully execute project requirements.

3.1.13.7 Manage and execute HPCMP and DSRC technology projects. Systems requiring integration services may be acquired by the contractor or provided by other sources. All systems are accountable to the Government.

3.1.13.8 Acquire HPC resources in accordance with approved HPC investment program management plans.

3.1.13.9 Manage resources throughout their life cycles.

3.1.13.10 Implement and formally test system upgrades and replacements.

3.1.13.11 Perform configuration management of all hardware and software.

3.1.13.12 Perform all activities necessary to plan, procure and implement changes to the facility including power, cooling and network connectivity to accommodate the installation of a new HPC system or related subsystems.

3.2 DELIVER UNCLASSIFIED HPC CAPABILITIES AT THE AFRL DSRC

Delivering HPC capabilities includes requirements in direct support of installing, integrating, and operating HPC systems for the HPCMP. The baseline scope in this PWS section includes allocated, shared unclassified HPC systems as well as shared special-purpose unclassified systems. The unclassified HPC systems include supporting Technology Insertion (TI) systems and systems funded via Congressional adds for the HPCMP. This PWS element also includes the HPC-related infrastructure that ties directly to delivering unclassified HPC capabilities, such as data movement, data storage (mid-tier and archive), and user interfaces/portals to unclassified HPC systems. The AFRL DSRC currently hosts three (3) allocated, shared unclassified HPC systems with the possibility of hosting more in the future.

3.2.1 Infrastructure Support for Unclassified HPC Systems

The Contractor shall provide technical support to include, but not be limited to, system engineering, system integration, system administration, and database administration for HPC-related infrastructure directly tied to delivering unclassified HPC capabilities. This includes (but is not limited to) HPC data movement and storage systems, portals, servers, routers, and switches that have direct relationships/connections to HPC systems. This does not include those systems or devices that are part of the AFRL DSRC baseline operations identified in paragraph 3.1 and associated subparagraphs.

3.2.2 Unclassified HPC System Installation and Integration

This requirement includes preparation of the AFRL DSRC facilities for installing, integrating, accepting, and operating unclassified HPC systems. The Contractor shall provide engineering support for connecting the unclassified HPC systems to the AFRL DSRC infrastructure, both facilities-related and HPC-delivery-related.

3.2.3 Application Management Support for Unclassified HPC

The HPC original equipment manufacturers (OEMs) typically provide levels of support, maintenance, and warranties for hardware and software delivered with an operational HPC system. For those software components that are to be installed and/or modified to support HPCMP workloads/use cases, the Contractor shall install, modify, and manage those applications for unclassified HPC system utility. This includes (but is not limited

to) job schedulers, reservation/dedicated partition services, data management/movement services, data visualizers, HPC applications, and user interfaces/portals.

3.2.4 Cybersecurity Support for Unclassified HPC Systems

This requirement is related to cybersecurity support for unclassified HPC needed in addition to the AFRL DSRC operations cybersecurity support identified in paragraph

3.1.3. The Contractor shall provide this additional cybersecurity support to include (but not be limited to) HPC infrastructure and HPC systems. Cybersecurity requirements related to unclassified HPC systems are identified in section 3.7.

3.2.5 Unclassified HPC System Support

Unclassified HPC systems may be managed and administered by OEMs, third-party entities, government personnel, or the Contractor. The Contractor shall provide additional unclassified HPC system administration support including (but not limited to) job scheduler configuration, regular backups of critical data, assistance with troubleshooting issues on all allocated unclassified HPC systems at the AFRL DSRC.

3.2.6 Additional unclassified HPC requirements

To be determined.

3.3 DELIVER CLASSIFIED HPC CAPABILITIES AT THE AFRL DSRC

Delivering HPC capabilities includes requirements in direct support of installing, integrating, and operating HPC systems for the HPCMP. The baseline scope in this PWS section includes allocated, shared classified HPC systems as well as shared special-purpose classified HPC systems. The classified HPC systems include supporting Technology Insertion (TI) systems and systems funded via Congressional adds for the HPCMP. This PWS element also includes the HPC-related infrastructure that ties directly to delivering classified HPC capabilities, such as data movement, data storage (mid-tier and archive), and user interfaces/portals to classified HPC systems.

The AFRL DSRC currently hosts three (3) allocated, shared classified HPC systems with the possibility of hosting more in the future.

3.3.1 Infrastructure Support for Classified HPC

HPC-related infrastructure directly tied to delivering classified HPC capabilities. This includes (but is not limited to) HPC data movement and storage systems, portals, servers, routers, and switches that have direct relationships/connections to HPC

3.3.2 Classified HPC System Installation and Integration

integrating, accepting, and operating classified HPC systems. The Contractor shall provide engineering support for connecting the classified HPC systems to the DSRC infrastructure, both facilities-related and HPC-delivery-related.

3.3.3 Application Management Support for Classified HPC

maintenance, and warranties for hardware and software delivered with an operational HPC system. For those software components that are to be installed and/or modified to support HPCMP workloads/use cases, the Contractor shall install, modify, and manage those applications for classified HPC system utility. This includes (but is not limited to) job schedulers, reservation/dedicated partition services, data management/movement services, data visualizers, HPC applications, and user interfaces/portals.

3.3.4 Cybersecurity Support for Classified HPC Systems

This requirement is related to cybersecurity support for classified HPC needed in addition to the AFRL DSRC operations cybersecurity support identified in paragraph

3.1.3. The Contractor shall provide this additional cybersecurity support to include (but related to classified HPC systems are identified in section 3.7.

3.3.5 Classified HPC System Support

Classified HPC systems may be managed and administered by OEMs, third-party additional classified HPC system administration support including (but not limited to) job scheduler configuration, regular backups of critical data, assistance with troubleshooting issues on all allocated classified HPC systems at the AFRL DSRC.

3.3.6 Additional classified HPC requirements

3.4 DELIVER SPECIALIZED HPC CAPABILITIES AT THE AFRL DSRC

Delivering HPC capabilities includes requirements in direct support of installing, integrating, and operating HPC systems for the HPCMP. The baseline scope in this PWS section includes specialized HPC systems used to explore new technologies, alternative service delivery models, or emerging non-traditional workloads/use cases.

These systems are typically not shared or allocated the same as the systems in PWS paragraphs 3.2 and 3.3, and can be hosted in various environments (unclassified, classified, isolated, standalone). This PWS element also includes the HPC-related infrastructure that ties directly to delivering specialized HPC capabilities, such as data movement, data storage (mid-tier and archive), and user interfaces/portals to specialized HPC systems.

3.4.1 Infrastructure Support for Specialized HPC

HPC-related infrastructure directly tied to delivering specialized HPC capabilities. This includes (but is not limited to) HPC data movement and storage systems, portals, servers, routers, and switches that have direct relationships/connections to HPC

3.4.2 Specialized HPC System Installation and Integration

integrating, accepting, and operating specialized HPC systems. The Contractor shall provide engineering support for connecting the specialized HPC systems to the AFRL DSRC infrastructure, both facilities-related and HPC-delivery-related.

3.4.3 Application Management Support for Specialized HPC

maintenance, and warranties for hardware and software delivered with an operational HPC system. For those software components that are to be installed and/or modified to support HPCMP workloads/use cases, the Contractor shall install, modify, and manage those applications for specialized HPC system utility. This includes (but is not limited

to) job schedulers, reservation/dedicated partition services, data management/movement services, data visualizers, HPC applications, and user interfaces/portals.

3.4.4 Cybersecurity Support for Specialized HPC Systems

This requirement is related to cybersecurity support for specialized HPC needed in addition to the AFRL DSRC operations cybersecurity support identified in paragraph

3.1.3. The Contractor shall provide this additional cybersecurity support to include (but related to Specialized HPC systems are identified in section 3.7.

3.4.5 Specialized HPC System Support

Specialized HPC systems may be managed and administered by OEMs, third-party additional specialized HPC system administration support including (but not limited to) job scheduler configuration, regular backups of critical data, assistance with troubleshooting issues on specialized HPC systems at the AFRL DSRC.

3.4.6 Additional specialized HPC requirements

3.5 ASSIST HPC USERS AT THE AFRL DSRC

This PWS element includes all baseline support required for assisting AFRL HPC users.

Enterprise Helpdesk requirements can be found in the Enterprise HPC Helpdesk Task Order.

3.5.1 Basic Support Requirements

The contractor shall provide HPC customer support services (Monday-Friday, 8:00 am to 4:30 pm EST, 8.5 hours each day) to local and remote users of AFRL DSRC unclassified and classified resources.

3.5.1.1 Support for deployed HPC systems, cloud services, or customer-funded systems shall be determined as the requirements are developed through Special Projects.

3.5.2 AFRL DSRC Help Desk Liaisons

The Contractor shall provide support staff at the AFRL DSRC to follow up on service requests issued by the Basic HPC Help Desk in PWS paragraph 3.5.1. These local AFRL DSRC support staff are required to be familiar with the AFRL DSRC’s HPC systems, infrastructure, and other support elements in order to address each Help Desk service request sent for follow-up. Support is required for both unclassified and classified HPC capabilities at the AFRL DSRC.

Account Support:

3.5.2.1 The AFRL Liaison shall troubleshoot users' HPCMP and Portal account access issues.

3.5.2.2 The AFRL Liaison shall support the AFRL SAM in troubleshooting AFRL staff accounts.

3.5.2.3 The AFRL Liaison shall facilitate the Privileged Access (PA) account process for the AFRL DSRC.

3.5.2.3.1 The AFRL Liaison shall identify…