Frequently Asked Questions (FAQ).pdf

Questions & Answers Regarding Request for Quote (RFQ)

RFQ Number: RFQID# 1372034

High Computing System

NUMBER SPECIFICATION

QUESTION/CLARIFICATION

REQUESTED

Question 1 Compute Section/ Compute Nodes On the compute nodes, it calls out at least 1TB of SSD storage per node.

Would the buyer accept 960GB

SSDs? If not, our next increment is

1.92TB SSDs.

Answer 1 960GB is acceptable.

Question 2 Compute Section/ Storage On the storage requirement, the RFQ identifies: At least 96TB of backup storage

Is this raw or useable storage? If useable, is RAID 6 configuration acceptable?

Answer 2 Raw is acceptable.

Question 3 Power Section Since no rack has been requested where would these PDU's be installed? Please provide types of connectors required for the Liebert

UPS.

Do you have racks already? If so, what brand and part number

Answer 3 While it does not explicitly state it in the requirements, a rack is to be provided to house the system including the PDUs. Coming off of the Liebert UPS is 3-4 foot long cables with CS8365 standard plugs that is capable of delivering 10kA at

240V.

Question 4 Power Section Again, since we have not been provided any power details, I need to know exactly what the end-user is looking for here. Typically, it would be Liebert that would do the power transformation.

Answer 4 The current Liebert UPS (Liebert

APM 90k-F, Model #

NRA91CCSA0A0095) has rectifiers

NUMBER SPECIFICATION

QUESTION/CLARIFICATION

REQUESTED

that can be rack mounted into the system. Each rectifier is capable

15kW/kVA. I expect performers to provide the rectifiers required to power the system; performers can work with Liebert if necessary.

Question 5 Power Section Again, need more information as to

PDU requirements - connectors -verticle - horizontal - number of whips per rack, how many racks will be supplied by the end-user.

Answer 5 Again, I expect a rack to be provided and whatever PDUs are required to power the system

Question 6 Power Section If they want a 3rd party to perform these services then we would have to get our services team involved to meet with the end-user and partner to build a statement of work. Please confirm this

Answer 6 Performer is to provide rack mounted chilled water door to cool system that is compatible with existing Liebert

DCP pumping unit.

Question 7 Power Section I am not sure how this is relevant to our quote since end-user is responsible for the Liebert request

Liebert part numbers

Answer 7 Liebert DCP Model #

DCP200RABB0

Question 8 Power Section Liebert DCD fanless chilled water door or similar to cool HPC rack o -

End-user to supply the rack and chilled water door.

Answer 8 Liebert DCD Model #

DCD35A6330A001

Question 9 Cooling Section Modify chilled water lines to accommodate DCD chilled water door/ is this a on site install project / requirement end-user to handle this.

Provide estimate of required cooling request detail

REQUESTED

Answer 9 Need to provide estimate for how much heat the system needs to dissipate to size the cooling required.

Question 10 Cooling Section Fully install high performance computer into exiting NRL facility.

Please provide a statement of work with regards to these requirements.

Answer 10 Performer can build system off site and then deliver the system and integrate with power and cooling on site, or performer can build system and integrate with power and cooling on site.

Question 11 Miscellaneous Section 3 years warranty on all hardware/

3 years support on all hardware o

Added/ Provide on-site training on use of system, including submitting a job via the scheduling software (i.e.

slurm) and running common software packages on the system o Please provide more details as to what level training is required

Answer 11 Training will be on how the system operates. How the operating system manages the nodes, how the job submission software works (can just provide a simple example like ‘Hello

World’), how the storage works, etc.

Question 12 Liebert DCD fanless chilled water door or similar to cool HPC rack o -

End-user to supply the rack and chilled water door.

Answer 12 PDUs must make use of existing

Liebert APM UPS o

Question 13 Is this a funded project? And will the government approve manufacturers other than HP ?

Answer 13 Yes this is Funded. You can use any manufacturer you want as long as you meet specs.

REQUESTED

Question 14 By identifying in the proposal what is

GSA-based, GSA-pending and open market, will that meet the requirements of the RFQ?

Answer 14 Yes.

Question 15 On the RFQ it states “Ability to connect…to internal NRL networks”.

What is needed to connect the two login and the head/admin node to internal NRL networks? What connectivity are they?

Answer 15 An Ethernet switch is needed to connect to the NRL network. A 10G

Ethernet switch.

Question 16 Is it okay to use DDR4-2933MHz memory?

Answer 16 Yes, DDR4-2933MHz memory instead of DDR4-2666MHz memory is okay to use.

Question 17 What is the kW Per Cabinet?

Answer 17 You are sizing the cabinet, so you need to provide this information.

Given what I asked for I would guess around 30kW.

Question 18 What is the Existing Chilled Water

Temperature for DCP/Cooling Door?

Answer 18 It has its own chiller outside and varies a little bit given outside conditions but it is generally 60C.

Question 19 What is the Desired Ambient

Temperature inside Room (i.e. Inlet air temperature entering racks)?

Answer 19 The room also has a separate unit that controls the room temperature to

70C.

Question 20 What is the Desired Cabinet Size?

Width (600MM or 750MM) 42U or

47U Cabinet Height?

Answer 20 42U in height, 600mm in width.

Question 21 How Many DCP Pumping Units does

NRL have today?

Answer 21 Just DCP pumping unit.

REQUESTED

Question 22 Is NRL running Redundant DCP

Pumps or Single?

Answer 22 There are two pumps in the unit, but only 1 runs at a time.

Question 23 Could we get the Serial Number from the existing DCP to see how old the unit is? May have some upgrades available to improve performance and efficiency of their current pump depending on age.

Answer 23 It was installed in 2011, Model

#DCP200RA2BB0. I am not currently in the office and will be out all week. I am trying to get someone to go check and see if the serial number is on the machine. I will send it if I can get it.

Question 24 Does NRL have existing Humidity

Controls in place?

Answer 24 There is another Emerson cooling unit (Challenger Air) in the room that controls the temperature and humidity in the room.

Question 26 Could we get the Serial Number from the existing DCP to see how old the unit is? May have some upgrades available to improve performance and efficiency of their current pump depending on age.

Answer 26 It was installed in 2011, Model

#DCP200RA2BB0. I am not currently in the office and will be out all week. I am trying to get someone to go check and see if the serial number is on the machine. I will send it if I can get it.

Question 27 Does NRL have existing Humidity

Controls in place?

Answer 27 There is another Emerson cooling unit (Challenger Air) in the room that controls the temperature and humidity in the room.

REQUESTED

Question 28 NRL has 90kVA Liebert APM UPS system. The 90kW system supports up to 6x 15kW Power Modules.

How many 15kW Power Modules are currently installed in the UPS?

Answer 28 There are currently two modules in the system.

Question 29 Is the UPS feeding a separate Power

Distribution Panel where the 3-4ft power whips with CS8365 are connected?

Or does the UPS have an integrated

Power Distribution Panel included within the Maintenance Bypass

Cabinet as shown below where the power whips are connected?

Answer 29 A bypass cabinet is included and the power whips come off the bypass panel.

Question 30 For Login Node: Please confirm what

RAID level the drives will use.

Additionally, provide CPU requirements such as the core count.

Answer 30 Match the CPUs with the compute nodes. So however many cores you have for each compute node is what I want for the login node. RAID 1 would be good for the login node so that there is redundancy.

Question 31 For Head/Admin Node - Compute nodes: 1500 Cores will not be a valid option for a single node system. The

REQUESTED

highest core count CPU available is only 56 cores, any each node may typically use Dual CPU Sockets.

Please clarify the required core count per system.

Answer 31 I have a nearly 10 year old system that is 360 cores.

Question 32 NRL has 90kVA Liebert APM UPS system. The 90kW system supports up to 6x 15kW Power Modules.

How many 15kW Power Modules are currently installed in the UPS?

Answer 32 There are currently two modules in the system.

Question 33 GPU Nodes: Please clarify the total core count required, as the RFQ mentions "4 CPU cores per GPU" which we cannot determine what to provide based on this information.

- RAID Level info is required for this system as well.

Answer 33 If a GPU node has 4 GPUs then the

GPU node requires at least 16 cores.

Again same thing as with the login and head nodes for RAID level.

Question 34 For the Interconnect: Please specify further requirements, such as port count, etc.

Answer 34 The interconnect to be non-blocking.

Question 35 For Storage: Please specify what

RAID level this will require?

Answer 35 There is no need for there to be any redundancy in the storage.

Question 36 Do you mean 2 Quad Core

Processors or do you actually mean four processors?

Answer 36 Are you referring to the GPU nodes requirements. I mean at least 4 cores per GPU. So if you size the system to have 4 GPUs per GPU node, then that node has to have at least 16 CPU cores. You can achieve 16 PCU cores with 2 8 core processors.

REQUESTED

Question 37 Are bright Cluster Manager Slurm

Job Scheduler and Intel MPI

Software?

Answer 37 Yes, these are software packages.

Question 38 Are Operating System is wanted and is to be installed?

Answer 38 RedHat or CentOS.

Question 39 Is a Power Module required for redundancy, or can it be at full capacity?

Answer 39 A redundancy would be sufficient but is not required. It can be at full capacity.