Technical Specifications.pdf

2024 NRL HPC Expansion Requirements Document

NRL code 8200 is acquiring a high performance computing system for scientific computing;

computational fluid dynamics, structural dynamics and thermal dynamics. The HPC will have the following features:

Compute Nodes:

o At least 10 compute node chassis with 4 processors per 2U node chassis

Each processor should have at least 48 cores

Intel-based processors - Intel Xeon comparable or better

Include option to add compute nodes, and price for adding additional nodes o At least 1024 GB of RAM per compute node (DDR5 4800 MHz or better) o At least 7TB SSD storage per node o Power:

Second level power redundancy (i.e. if the primary supply fails, there is at least a second power supply to provide the required power) with power rating specified.

o Cooling:

Air Cooled o Networking:

1x 1GbE Dedicated IPMI or better

2x 10 GbE or better

1x 100Gb/s HDR or better o Form Factor:

2U Rackmount per 4 nodes

Login/Head Nodes:

o 2x head nodes

Intel-based processors - Intel Xeon comparable or better o 256 GB of RAM per node or better (DDR5 4800 MHz or better) o At least 96 TB of storage

Must be configured in RAID 5 o NOTE: prefer Login/head node to match compute node, but not required.

o Power:

Second level power redundancy (i.e. if the primary supply fails, there is at least a second power supply to provide the required power) with power rating specified.

o Cooling:

Air Cooled o Networking:

1x 1GbE Dedicated IPMI or better

2x 10 GbE or better

1x 100Gb/s HDR or better

Juniper SRX 1500 to connect login nodes and Head/Admin node to NRL internal network o Form Factor:

2U Rackmount

Storage Node:

o 1x Storage array node

Intel-based processors - Intel Xeon comparable or better o 128 GB of RAM per node or better (DDR5 4800 MHz or better) o At least 96 TB of storage

Must be configured in RAID 5 o Power:

Second level power redundancy (i.e. if the primary supply fails, there is at least a second power supply to provide the required power) with power rating specified.

o Cooling:

Air Cooled o Networking:

1x 1GbE Dedicated IPMI or better

2x 10 GbE or better

1x 100Gb/s HDR or better o Form Factor:

2U Rackmount

System Software:

o CentOS with desktop environment installed o Bright Cluster Manager or similar software o Slurm job scheduler or similar software o Intel, GCC, OpenMPI, and MPICH installed

System Networking:

o Mellanox Quantum HDR InfiniBand Switch or better o 48 port 1G Ethernet ToR Management switch or better o 48 port 10G BASE-T switch or better

System Power:

o 208 V 3-phase power o Provide estimate of power usage o Existing room is NOT a raised floor, PDU cables must exit the top of the system o PDUs must make use of existing Liebert APM 90k-F UPS model NRB91CCSA0A0095.

This portion of installation may be subcontracted and included in the overall quote. See installation guidance below.

o Installation Guidance:

HPC room is on an independent breaker, allowing for power isolation/cutoff to the room during installation.

Breaker panel already exists. Breakers for this new HPC system should be provided if necessary.

VAC output from the rack mountable modules has six (6) 15kVA modules, three

(3) are currently in use, three (3) are currently available.

Current power draw is 45kVA. Maximum allowable power draw is currently

90kVA/kW (max allowable of UPS).

System Cooling:

o Networking and power cable bundles must be arranged to ensure adequate airflow and cooling for system o Liebert DCD active cooled (with fans) chilled water door or similar to cool HPC rack o Provide estimate of required cooling o Must be cooled with existing Liebert DCP coolant pumping unit with Model #

DCP200RA2BB0.

o Modify facility chilled water lines to accommodate the chilled water door. This portion of installation may be subcontracted and included in the overall quote. See installation guidance below.

o Installation Guidance:

Overhead chilled water supply header is already in place for cooling lines.

Header has an approximately 3” (inch) outer diameter (OD).

Piping will need to be installed to connect header to chilled door. Piping will need to be routed horizontally approximately 5’ (feet) plus additional distance, depending on chilled water door inlet connection location.

Piping will need to be dropped from the overhead lines approximately 29”

(inches) to chilled water door.

Miscellaneous:

o 42U Rack, 800x1200mm fully assembled with leveling feet o Build and test the system prior to delivery to NRL to ensure power up, functionality of all nodes, and demonstrate sufficient cooling for the system under stressing computational loads. Any preinstalled software should also be tested.

o Deliver and fully install the HPC into the existing NRL facility, being careful to fulfill the interface requirements specified under the ‘System Power’ and ‘System Cooling’ sections o Keyboard, Video, and Mouse (KVM) for the rack system along with necessary KVM switch/connection.

o Install team must be U.S. citizens. If members of the install team are dual citizens, NRL must be notified in advance.

Support:

o 3 years warranty on all hardware o 3 years support on all hardware o Provide up to one-day on-site training of cluster management software with the assurance that additional support could be requested through the standard company support structure.

o Contractor must run LINPACK on site after system has been installed for at least 24 hours without any errors o US-based hardware/technical support required for the duration of the support contract (at least 36 months). Please detail how hardware/technical support is to be requested (phone, email, online portal, etc.) and whether it is provided by the contractor who is directly supplying the HPC system, or the hardware manufacturer.