FY25 RFI_PWS for BOA-On-Ramping - 20240719 Final RFI.pdf

TECHNOLOGY INSERTION REQUEST FOR INFORMATION

PERFORMANCE WORK STATEMENT (PWS)

16 July 2024

1.0 OBJECTIVE

The primary objective of this Basic Ordering Agreement (BOA) is to provide the High- Performance Computing Modernization Program (HPCMP) with world-class high performance computing capabilities for the United States Department of Defense (DOD). The BOA holders will be responsible for providing the Government with balanced, commercially available, production-grade High Performance Computing (HPC) systems, which contain an appropriate combination of processor, memory, disk Input/output (I/O), interconnect, and Operating System (OS) capabilities to conduct complex, tightly coupled, large-scale, scientific calculations.

2.0 DESCRIPTION

2.1 Locations

Requirements for each order will be located primarily at the following four DSRCs and will encompass the unique requirements of each center:

• Air Force Research Laboratory (AFRL) DSRC, Wright-Patterson Air Force Base, Ohio

• Army Research Laboratory (ARL) DSRC, Aberdeen Proving Ground, Maryland

• U.S. Navy DSRC, John C. Stennis Space Center, Mississippi

• U.S. Army Corps of Engineers, Engineer Research and Development Center

(ERDC) DSRC, Vicksburg, Mississippi

2.2 System Manufacturing Capabilities

To satisfactorily complete the work, the contractor must be a United States of America Original Equipment Manufacturer (OEM) with expert knowledge of Federal Government and industry-recognized national standards for advanced computing and communication technologies. Targeted system attributes, at the order level, may include the following, with specific orders having varying requirements. These are meant to be illustrative of an exemplar HPC system.

• A high ratio of peak CPU memory bandwidth to peak CPU floating point capability.

• 256 GB to 4 TB of random-access memory per compute node with single bit correction and multibit error detection.

• No less than 512 total compute nodes spanning a range of node types to include:

• A range of supported processor socket counts, and

• Nodes with and without support for PCIe devices (e.g. nonvolatile memory express (NVMe) storage, general purpose graphic processing units (GPGPUs))

• Multiple petabyte scalable high-performance file systems with all metadata stored on solid state devices, no single point of failure, and protection from double storage device failures (e.g. RAID6)

• Aggregate data transfer rates between the compute nodes and each file systems ranging from 20 GB/s to 1 TB/s.

• An ability to simultaneously mount multiple file systems on all login nodes and all compute nodes or a subset of file systems on a subset of nodes.

• A high number (500,000+) of I/O operations per second (IOPs) using 1, 32, and 128 active nodes for the following file operations for each parallel file system:

• Open,

• Stat,

• Open and stat simultaneously, and

• Unlink.

• A tightly integrated interconnect with:

• Each link having at least 100 Gbps of bandwidth per processor socket, and

• An end-to-end latency that is no greater than 3µs (three microseconds).

• A well-tuned Federal Information Protection Standard (FIPS) compliant operating system (OS) with low jitter, configured in accordance with DISA security technical implementation guides (STIG).

• A 10/40/100 gigabit Ethernet interface (or higher data rate) for both internal and external connectivity with IPv4 and IPv6 dual-stack functionality

• Workload management software that supports SELinux and can launch jobs using container-based solutions (e.g. Docker, Shifter, Apptainer)

• An appropriate number of job scheduling nodes with:

• Suitable job scheduling software and

• Adequate memory, processing capability, and redundancy.

• Use of HPCMP Kerberos and HPCMP Public Key Infrastructure (PKI) authentication software.

• Use of HPCMP Secure Shell (SSH) communication software furnished by the Government.

• Adherence to the following power quality standards:

• Computer Business Equipment Manufacturers Association (CBEMA),

• Information Technology Industry Council (ITIC),

• Semiconductor Equipment and Materials International (SEMI) F47, and

• International Electrotechnical Commission (IEC) 61000-4-11/34.

• Three-phase power cabinet supply with a preference for a voltage of 480V

• A liquid-cooling solution that removes no less than 98% of the heat generated by the system.

• Login nodes and data transfer nodes which contain the same processor type as the compute nodes to facilitate compiling, data transfer, and other interactive functions, but contain more memory per core, additional 10/40/100 Gbps network interfaces, and with support for machine learning accelerators (e.g. GPGPUs).

• A Test and Development System (TDS) that facilitates effective system software testing, user code porting, and base system configuration management that is a smaller version of the base system, configured with similar hardware, software, file system, and I/O attributes.

• An ability to execute as many of the software packages listed on the HPCMP’s software web page as possible: https://centers.hpc.mil/software.

• No less than three suites of high-level language compilers targeting each processor type included in the system:

• GNU Compiler Collection (version 12.3 or higher) and

• At least one commercially supported suite.

• No less than two implementations of MPI with at least one being compliant with the MPI- 3.1+ standard.

• Support for JavaScript, Go, and Perl scripting languages.

• Latest version of Python 3 and Jupyter notebooks with as many of the following extensions as possible:

• NumPy,

• MPI4Py,

• SciPy,

• PyTorch, and

• IPython.

• A development environment that includes the following features:

• Multiprocessing APIs (including OpenMP)

• Standard computational libraries (including BLAS, BLACS, FFTW, LAPACK, and ScaLAPACK)

• Accelerator computational libraries/compilers (including vendor specific, OpenCL, and OpenACC)

• Performance libraries including the Performance API (PAPI)

• Integrated development environment

• Application performance instrumentation tools including:

• No less than one code profiler that interfaces with PAPI, and

• No less than one tool that provides MPI messaging statistics.

• A high level of system reliability with a high overall effectiveness level and a low number of user/job interrupts.

• A high level of performance over processor ranges of no less than 8,192 and not greater than 65,536 processor cores on high performance applications and libraries including the following codes:

• GAMESS

• CTH

• HYCOM

• FFTW

2.3 Administration and Maintenance

System Administration of the systems may be required as part of the overall specific system order requirements. When required, System Administration shall be performed by the Offeror 24 hours per day, 7 days per week (over seven (7) one-year options at the individual order level) with BOA holders ensuring that (1) the monthly system effectiveness level does not fall below 97%, and (2) the monthly number of user/job interrupts does not exceed the maximum allowable (which is determined by dividing the number of nodes in the base system by one hundred). Requirements for System Administration will be dependent on the specific order and shall be identified for the order/system in any Request for Proposal submitted to the BOA holders to support individual Technical Insertion requirements.

In all requirements, Hardware and Software maintenance for the systems shall be maintained by the Offeror with 24 hours per day, 7 days per week system maintenance (over five one-year options at the individual order level) and therefore must ensure that (1) the monthly system effectiveness level does not fall below 97%,

(2) the response time for remedial maintenance is less than four hours (some orders may require a short response time), (3) new software is tested and ready for operation prior to implementation on the base system, and (4) software/hardware security vulnerabilities are addressed in a timely manner.

3.0 SECURITY REQUIREMENTS

Since the HPCMP will predominantly place awarded system at DOD facilities, BOA holders must have sufficient personnel with no less than SECRET level clearances for the purposes of system installation, capability testing, effectiveness level testing, administration, and maintenance.

All personnel requiring privileged access to any system and/or supporting network/security components must have:

• An adjudicated Tier 5 background investigation,

• A clearance commensurate with the classification of the system,

• An Information Assurance Technician (IAT) Level II or III certification (see https://disa.mil/NewsandEvents/Training), and

• Computing environment certifications specific to all relevant operating systems.

4.0 PUBLIC AFFAIRS

The BOA holder must not publicly disclose any data generated or reviewed under this agreement. The BOA holder must refer all requests for information concerning projects to the Contracting Officer for comment. Prior to release, the Contracting Officer must approve any articles for any publication.

5.0 SUBMITTALS

The Performance Work Statement (PWS) identifies all submittal requirements at the order level. These may include but are not limited to:

• Safety plans,

• Quality control plans,

• Project schedules,

• Testing plans,

• Testing results, and

• Closeout documentation.

6.0 REPORTING REQUIREMENTS

The Performance Work Statement (PWS) establishes all reporting requirements at the order level. Typical reporting requirements are monthly however, each order will identify the specific requirements for that procurement.

7.0 POINTS OF CONTACT

The following lists the BOA level POCs. The Performance Work Statement (PWS) may identify further POCs at the individual order level. All correspondence related to this BOA and all orders must include the Contracting Officer.

7.1 Contracting Officer:

Melissa K. Lynn melissa.k.lynn@usace.army.mil

7.2 Acquisition Project Manager:

Lloyd W. Slonaker, Jr.

lloyd.slonaker@us.af.mil