N0017423R0028.pdf

SEE ADDENDUM

(No Collect Calls)

N0017423R0028 25-Apr-2023

b. TELEPHONE NUMBER

301- 744-6843

8. OFFER DUE DATE/LOCAL TIME

02:00 PM 12 May 2023

5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

STANDARD FORM 1449 (REV. 2/2012)

Prescribed by GSA – FAR (48 CFR) 53.212

(TYPE OR PRINT)

(SIGNATURE OF CONTRACTING OFFICER)

ADDENDA ARE

26. TOTAL AWARD AMOUNT (For Gov t. Use Only )

23.

CODE 10. THIS ACQUISITION IS

SUCH ADDRESS IN OFFER

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT

BELOW IS CHECKED

TELEPHONE NO.

N001749. ISSUED BY

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a. UNLESS BLOCK

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME

ERIN B BOWLING

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NUMBER

(TYPE OR PRINT)

30b. NAME AND TITLE OF SIGNER 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a.UNITED STATES OF AMERICA

0 27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1. 52.212-4. FAR 52.212-3. 52.212-5 ARE ATTACHED.

25. ACCOUNTING AND APPROPRIATION DATA

1. REQUISITION NUMBER

20.

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED.

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, AND 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

ARE NOT ATTACHED

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

SET FORTH HEREIN, IS ACCEPTED AS TO ITEMS:

. YOUR OFFER ON SOLICITATION

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

% FOR:SET ASIDE:UNRESTRICTED ORX

SMALL BUSINESS

17a.CONTRACTOR/ CODE FACILITY

OFFEROR CODE

NSWC IHD C02

4081 N. JACKSON RD BLDG 841

INDIAN HEAD MD 20640

18a. PAYMENT WILL BE MADE BY CODE

RATED ORDER UNDER

DPAS (15 CFR 700)

13a. THIS CONTRACT IS AX

13b. RATING

DO-C9

CODE15. DELIVER TO CODE N00174 16. ADMINISTERED BY

12. DISCOUNT TERMS11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

X SEE SCHEDULE

14. METHOD OF SOLICITATION

RFQ IFB RFPX

NSWC IHD

RECEIVING

4522 MCMAHAN ROAD B116

INDIAN HEAD MD 20640-5125

TEL: 301-744-4221 FAX:

FAX:

TEL: 3017446891 SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

8(A)

HUBZONE SMALL

BUSINESS

SIZE STANDARD:

34000000

NAICS:

541512

X

OFFER DATED

29. AWARD OF CONTRACT: REF.

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND

EMAIL:

TEL:

31c. DATE SIGNED

SEE SCHEDULE

SCHEDULE OF SUPPLIES/ SERVICESITEM NO. QUANTITY UNIT UNIT PRICE AMOUNT

24.22.21.19.

WOMEN-OWNED SMALL BUSINESS (WOSB)

ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

(CONTINUED)

PAGE 2 OF102

ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED: ______________________________________________________

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED

32b. SIGNATURE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32c. DATE 32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE 32f . TELEPHONE NUMBER OF AUTHORIZED GOVERNMENT REPRESENTATIVE

37. CHECK NUMBER

FINALPARTIALCOMPLETE

36. PAYMENT35. AMOUNT VERIFIED

CORRECT FOR

34. VOUCHER NUMBER

FINAL

33. SHIP NUMBER

PARTIAL

38. S/R ACCOUNT NUMBER 39. S/R VOUCHER NUMBER 40. PAID BY

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER 41c. DATE

42a. RECEIVED BY (Print)

42b. RECEIVED AT (Location)

42c. DATE REC'D (YY/MM/DD) 42d. TOTAL CONTAINERS

STANDARD FORM 1449 (REV. 2/2012) BACK

Prescribed by GSA – FAR (48 CFR) 53.212

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

SEE SCHEDULE

20.

SCHEDULE OF SUPPLIES/ SERVICES

21.

QUANTITY UNIT

22. 23.

UNIT PRICE

24.

AMOUNT

19.

ITEM NO.

N0017423R0028

Section SF 30 - BLOCK 14 CONTINUATION PAGE

STATEMENT OF WORK (SOW)

STATEMENT OF WORK

NSWC INDIAN HEAD DIVISION SUPERCOMPUTING

FACILITY

SUMMARY: Described in this Statement of Work (SOW) are the minimum requirements for the hardware and software, shipping, installation and software configurations, training, and five‐year warranty and support for the cluster high performance supercomputer (HPC) to be procured to establish the Naval Surface Warfare Center, Indian Head Division (NSWC IHD), MD supercomputing facility. At the end of installation, software configuration, and training, the HPC will be on the NSWC IHD network for end users to remotely connect and use for their day‐to‐day modeling and simulations (M&S) needs. Vendor may be versed with the hardware configuration listed herein. However, there are differences in how the HPC needs to be configured and made operational. Throughout this SOW, there are options for the selected vendor to suggest modifications in consultations with the NSWC IHD Technical POC and IT department, in that order. Due to the unique requirements for hardware and software configurations, compatibility with the cyber security and STIGS requirements, the five‐year warranty, and maintenance requirement with Next Business Day (NBD) service at a secured facility, NSWC IHD is compelled to receive proposals / bids from only vendors who satisfy the following requirements:

Vendor implies the original manufacturer or a manufacturer’s designated reseller of cluster HPC. In the latter case, the reseller and manufacturer agree that the requirements, in their totality, apply equally to the reseller and original manufacturer.

Vendor must have a proven track record of manufacturing cluster HPC, optimizing its performance for scientific and engineering computing in a multi‐user environment, and selecting components that enhances the performance. The quote must include supplemental brochure / documentation and years of experience of supplying HPC.

Vendor must have a proven track record of having supplied and maintained / maintaining similar multi‐node HPC to Department of Defense (DoD) laboratory/ies. This is to ensure that vendor has experience and is versed with the cyber security and STIGS to be applied by the NSWC IHD IT team after installation. This requirement will mitigate any incompatibility of the hardware and software with cyber security and STIGS requirements before being delivered, which will otherwise render the HPC unusable on the NSWC IHD network. Please note that the cyber security and STIGS requirement being practiced by Department of Energy (DOE) laboratories may be different from the DoD laboratories.

Hence, the focus is on the DoD laboratories.

Vendor must be familiar with and practicing TAA and GSA compliance of the hardware and software. The compliance applies to individual components as well as to the assembled racks. NSWC IHD, at its own discretion, reserves the right to waive this requirement for a small percentage of components. For this waiver, vendor must summarize the list of non‐compliant components at the end of their quote and agree to replace such components free‐of‐cost in the case of failure during the maintenance period of 5 years.

The five‐year warranty and maintenance by vendor of all the components including UPS batteries and the assembled HPC includes 24x7 telephone support, dedicated port‐to‐port secured remote connection to the HPC for trouble shooting, and next business day replacement of the failed/failing components. It is inevitable that the HPC will contain components manufactured by a third party. NSWC IHD will not be responsible to contact any third party for its warranty and maintenance support needs. The vendor will be solely responsible to fulfill the warranty and maintenance support of all components manufactured by the vendor (manufacturer or reseller as the case may be) or any third party. The vendor will set up a telephone number to directly reach the support instead of routing the caller through a number of options.

The next business day support service will include both overnight shipping of failed/failing components as indicated by the cluster management software and/or technical support and visit by vendor’s technician to NSWC IHD for the replacement.

Vendor must have security‐cleared U.S. National technicians for the secured port‐to‐port remote connections and the next business day part replacement services. HPC, after installation and commissioning, will not be accessible by a non‐cleared technician even if he/she is a U.S. National.

Quote Structure: HPC will consist of several numbers of nodes and other components that make the HPC as given in Table 1 and described further. NSWC IHD must tag these as assets and enter them into the inventory. However, it will not be possible to individually tag and inventory individual components of the HPC. For this reason, vendors must provide the quote for individual racks in the format given below so that only racks are tagged as individual assets containing the components listed in the quote. An automated quote system may not produce a quote format based on the aforementioned rack grouping. Quotes not formatted by rack, as seen in the example below, will not be accepted. The vendor will attach a supplemental quote grouped by rack (Rack Number 1: contains these and these items, Rack Number 2: contains these and these items, e.g.) referring to components in each rack to the line number or group of line numbers in the automated system quote. Additionally, vendors will attach the elevation of racks, electrical, and cooling requirements. The format for the main automated quote or the supplemental

1. quote will be:

Hardware

Item No 1: Rack 1 containing which nodes and/or how many UPS with Price

…. And so on.

Item No L: Rack L containing …….

Software

Item No L+1: Description and price

… And so on.

Item No. M:

Uninterrupted Power Supply (UPS), if UPS is to be provided in a separate rack

Item No M+1: Description and price

…And so on Item No N:……

Shipping

Item No N+1 – Shipping (FOB) with price. All shipments (if more than one) should be combined as one item for the purpose of the quote.

Installation and Software Configuration

Item No N+2: Installation and Final Configuration of Software

Training

Item No N+3: Training of NSWC IHD IT Staffs

Warranty and Support

Item No N+4: 5 Years warranty/Support with 24x7 telephone support and next business day (NBD) replacement.

Vendor may absorb cost of some of the items above in the hardware cost, for example, shipping, installation and final configuration of software, etc. In that case, the cost of such item must be listed as $0.00 in the summary line items. At the least, the cost for warranty and 5‐year support must be absorbed in the hardware cost to enable simplified procurement as envisaged in this announcement.

2. HARDWARE: HPC hardware and software configuration and capacity (C&C) are given in Table 1 specifying the type and number of nodes, processors’ base clock frequency, RAM memory percore, total RAM memory, type, and capacity of hard drive(s) on the nodes, and NVIDIA GPU if applicable. Also given are some of the key components required to make the cluster an HPC. Vendor will provide other needed components, but not given in Table 1, of the specifications to maintain or exceed the overall performance and maintainability of the HPC.

Hence is the emphasis on the experience of the vendor (reseller / manufacturer) described above. The salient features are explained below to highlight differences mentioned above and remove ambiguity to help vendors to account for all the listed hardware and services.

2.1. Processor Cores: All processors are AMD EPYC series or better. All nodes are dual sockets (processors) nodes. The numbers of cores in Table 1 are the base number of cores (sum of cores for two processors) before hyperthreading. Many engineering software in use at NSWC

IHD either do not utilize or do not perform optimally after enabling hyperthreading. Hence, the increase in number of threads (generally twice) enabled by hyperthreading will not be considered.

2.2. Definition of Minimum Requirements: Table 1 gives the minimum requirements of critical components. Vendor may provide the minimum or better than minimum configuration at its own discretion which may translate to the total cost.

AMD processors or any other processor with better base clock frequency and/or larger number of cores per processor are allowed with restrictions. The better clock frequency is permitted if the required clock frequency of Table 1 has met the end of line (EOL). A larger number of cores per processor will not reduce the number of nodes in individual category. For that matter, the minimum number of master, login, disc, and compute nodes must be maintained independent of the higher clock frequency of the processors.

The vendor may choose to provide single socket (processor) nodes. The single socket nodes must be uniform throughout the system. In that case, the number of node in each category must double.

On the contrary, the number of nodes has to increase uniformly in all categories if the processor meets the base clock frequency requirement but has a smaller number of cores. This will increase the total hard drive and number of GPUs if applicable, and may become cost prohibitive.

RAM memory per core will remain as given in Table 1 or larger. If the total number of cores on each node is more than the specified value, then the total RAM memory on the node will also increase to meet the requirement.

All processor to memory channels must be populated with RAM DIMMs on all nodes for optimal performance. Total RAM memory on nodes is allowed to exceed the minimum requirement if the DIMM capacity being offered is such that populating all channels increases the total RAM memory.

The number, type, and capacity of hard drive(s) per node will be as specified. The total hard drive on one or more categories of nodes will increase if the number of nodes increases.

2.3. Master Nodes: The required configuration has two master nodes for traditional functions of administering, managing, controlling nodes, and providing the system and user applications software. All software will be installed on the master nodes and made available to users on the login nodes. Similarly, all the audit files will reside on master nodes. For this reason, master nodes have larger disk space that cannot be reduced. The two master nodes can be configured in three different ways as given later or in a better, alternate way in consultation with the NSWC IHD Technical POC and IT department, in that order.

Vendor may prefer to use software installed on some other class of nodes (login or disk) which will provide all the functionalities of master nodes, and decide to remove the master nodes from the configuration. Removal of master nodes from the configuration is not permitted to maintain uniformity across all vendors bidding for the contract. Such removal will be detrimental to the HPC performance by not having other functions as envisaged for the HPC.

2.4. Login Nodes: Login nodes are for users’ interactive work that will include editing and developing source code, compilation, single / multiple cores execution for not more than 15 minutes to test the software, and submitting final runs on compute nodes. Vendor in consultation with NSWC IHD IT team will decide the final configuration as given later. In addition to their work and data on login nodes, users may also need additional space to install software in their home folder. For this reason, login nodes are configured with large disk space which can be increased, but not reduced.

2.5. Compute Nodes: These are traditional compute nodes for parallel computing. In contrast to diskless compute nodes, the disk on compute nodes serves two purposes by having two partitions – (a) the operating system (OS) partition with enough space for future upgrades will contain SWAP space equal in size to the RAM memory on the node to act as the virtual memory, and (b) the second partition will be the “local scratch”. If software running on a node needs to write scratch data that does not need to be preserved, then such data will be written locally on the local scratch for optimal performance. Vendor has the option to eliminate two partitions and provide space for both function in one sole partition.

Compute Nodes with GPU: These are same as compute nodes described above with the exception that they have one NVIDIA A100 GPU per node. The number of GPUs cannot be reduced below 8 in total. increasing the number of these nodes due to the lower number of cores per processor will increase the number of GPUs which will be cost prohibitive and not advised.

2.6. Large Memory Compute & Visualization Nodes with GPU: Their function and configuration will be same as the compute nodes with GPU. Their RAM size per core is twice (8GB) compared to compute nodes (4GB). Accordingly, the SWAP space size on their disk will also be twice the size of those on the compute nodes.

2.7. Disk Node: With users’ home folders mounted on login nodes, they will have folders for the long‐term data storage on the disk node. The output files during software execution needing to be stored will be written on the disk node. For this reason, the disk space on the disk node is the largest and can be increased, but not reduced.

2.8. Infiniband and Switches: These are the backbone of the cluster HPC. Vendor must provide the minimum required given in Table 1 or better. Vendor may choose a better option to boost the inter‐node data transfer rate and data transfer bandwidth. Vendor may also choose a better option if the specified capacity has reached end of line (EOL). Vendor will connect all the racks with and configure the Infiniband for the entire cluster to work as one HPC. Furthermore, they need to be configured for easy additions of future rack(s). The capacity and numbers of switches for users’ connectivity must allow, at a minimum, 50 concurrent users without network latency.

2.9. TPM 2.0 or Higher Enabled: As per DON’s critical requirements, all nodes must be TPM

2.0 or higher enabled for encryption. This must be provided and stated in the quote for each node, each class of node, or as a note at the end stating: “All nodes have TPM 2.0 or Higher Enabled”. Vendor will provide necessary hardware on the motherboard of every node to enable TPM 2.0 or higher.

2.10. No Bluetooth / No Wireless: As the second critical requirement, there must not be any Bluetooth or Wireless hardware on any node. It must be mentioned on each node or as a note at the end stating: “No Bluetooth / No Wireless”.

2.11. TAA and GSA Compliance: As the third critical requirement, all hardware must be TAA and GSA compliant. NSWC IHD may allow a very small percentage of non‐TAA and GSA compliant hardware at its discretion. Vendors must list the non‐compliant components at the end of the quote. Vendors must minimize the non‐compliant components. The non‐ compliant components provided in the list may disqualify the vendor without further discussion.

2.13 No Hardware or Software RAID: As the fourth critical requirement, there must not be hardware enabled or software enabled RAID on any node and HPC as a whole.

2.14. Humidity and Temperature Sensors: Vendor must provide enough humidity and temperature sensors in every rack. These sensors must be set up to trigger the HPC shutdown if the humidity and/or temperature reaches the upperty safe limit.

2.15. Built‐in console: Vendor must include one, preferably two, built‐in console(s) for system administration, exploration, and maintenance of the entire HPC as a whole.

3. SOFTWARE

The needed software is to be provided in part by the vendor and rest by NSWC IHD as follows:

Vendor will quote and provide the cluster management software with 5‐year support. All the provisions for the support listed above apply equally to the software.

NSWC IHD will provide the Red Hat Enterprise Linux (RHEL) Operating System (OS), Intel oneAPI developer suites even for the AMD processors based system, AMD Zen Software Studio developer suites for the AMD processor based system, and SLURM resource management software. Vendor will provide the list in the quote for number of RHEL OS licenses needed for the master nodes, login nodes, compute nodes, compute nodes with GPU, large memory compute and visualization nodes with GPU, and disc node. Vendors will confirm the number of licenses required with the DON RHEL POC before listing them in the quote.

Vendor will install and configure all software (provided by the vendor or NSWC IHD) as given in the installation section.

4. UNINTERRUPTED POWER SUPPLY (UPS)

In the event of a power outage, the purpose of the UPS is to delete all waiting jobs in the SLURM queues, terminate all the users’ jobs running at the time, and shut down the entire

HPC. Vendor will determine the time required “H Minutes”. The UPS number and total capacity needs to enable safe completion of termination and shutdown sequences. HPC will be started manually after the power is restored. Vendor has the choice of (a) the preferred method of installing UPS in the lower half of racks while the upper racks have nodes, or (b) the entire UPS installed in a separate rack and connected with the HPC.

5. SHIPPING

Vendor has the following options for the shipping.

Vendor can ship the individual components of HPC and assemble them in the rack, adhering to the rack configuration as was provided in the quote or as supplemental quote.

Vendor can ship the fully assembled racks per the quote.

Vendor can also choose to mix the above two modes for shipping.

Vendors must provide all shipping tracking numbers to NSWC IHD Technical POC for tracking and receiving.

6. INSTALLATION AND SOFTWARE CONFIGURATION

The chosen vendor will install the hardware, install and configure software, and make the HPC ready for use as follows:

6.1. The vendor will assemble all the components in the racks, so that each rack has the components as provided in the quote / supplemental quote. The racks will be installed in the server room and connected to the wall electrical outlet as pre‐determined by the vendor (during the visit mentioned later.)

6.2. Vendor will install RHEL on all nodes in the presence of NSWC IHD IT staff. The purpose of IT staffs’ presence during this installation is not to help or suggest installation. Vendor can avail the technical support from RHEL as required on the licenses provided by NSWC IHD.

Vendor will install:

Full version of RHEL, not the minimal or customized version, “devel” and “group packages”, which are not installed by default.

NSWC IHD will provide the list of the required “devel” and “group packages”. The list will also contain a few select configurations independent from those of the vendor. The vendor will configure the OS for the best performance of the HPC.

6.3. In consultation with the NSWC IHD Technical POC / IT team, the vendor will choose one of the three options given below or an alternate, better option to configure the two master nodes:

(a) One master node performing all the functions and the other acting as standby and the two switching their function at regular intervals, or

(b) Both master nodes operating together and splitting the workload nearly equally.

6.4. In consultation with the NSWC IHD IT team, vendor will choose one of the following options to configure the login nodes. Both options require the users’ home folders to be mounted on the login nodes.

(a) Users’ login to the master node and work on the login nodes with their home folder mounted on the login nodes, or

Users directly login to login nodes and work.

6.5 Vendor, in consultation with NSWC IHD IT staffs, will install InfiniBand and setup all network switches including those to be used by users for the remote connection. As mentioned earlier, the number of switches and their setup for users must be to provide concurrent remote connections to 50 users without affecting network bandwidth available on the HPC.

6.6. Vendor will install cluster management software, interface it with the built‐in console for the display of the system and system status, and to query, administer, explore, and manage individual nodes, as well as HPC as a whole.

6.7. Vendor will help the NSWC IHD IT team while the latter implements STIGS security measures.

6.8. Vendor will install NSWC IHD‐provided Intel one API development software. Vendor can avail the technical support from Intel on the license provided by NSWC IHD. The software will have default compilation flags and documentation for setup of the environmental variables. These must be provided to users through CRON at the time of their account creation.

6.9. Vendor will install NSWC IHD‐provided AMD Zen Software Studio development package for the AMD processor based system only. This is a free software available from AMD.

However, NSWC IHD will facilitate and vendor can avail the technical support from AMD, if required. Like one API, the default compilation flag and environment variable setup must be provided to individual users by default.

6.10. Vendor will also install NSWC IHD‐provided resource management software SLURM by SchedMD. SLURM is a free software, but NSWC IHD will procure one‐year technical support from SchedMD which the vendor can choose to utilize during the installation.

6.11. It is often found that the default path after OS installation does not include the paths for key packages and utilities or the default path is altered during STIGS. Vendor will make all the needed efforts in consultation with the NSWC IHD IT team to provide them in the default path. It is preferred that /etc/bashrc is copied to users’ /.bashrc file for further customization by the users.

6.12. Vendor must provide support to NSWC IHD IT staffs during STIGS that will occur before training after all the software has been installed and stress test has been successfully completed.

7. TRAINING

Vendor will include in the quote and provide training to NSWC IHD IT staffs. The training will start after the STIGS customization has been completed by the IT staffs in consultation with the vendor technical staffs. Vendor will determine the number of days needed for this first training and quote it as a separate line item. Vendor will also provide 2 days follow‐up refresher training after one year and account for this refresher training as the training cost in the quote. NSWC IHD IT may require yearly training in subsequent years for which vendor will submit a new quote when the need arises. The initial and refresher trainings will include, but are not limited to:

7.1. Shutting down and booting the entire HPC.

7.2. Shutting down and isolating individual node(s), if required.

7.3. Use of the cluster management software through the built‐in console.

7.4. Checking the general health of the HPC on the console.

7.5. Troubleshooting any error(s) or message(s) displayed by the cluster management software.

7.6. Setting and changing the soft and hard limit of disk quota for users.

7.7. Upgrading OS and the sequence, if any.

7.8. Upgrading other software as required. Etc.

8. WARRANTY AND SUPPORT

Vendor will provide a 5‐year warranty and support for all the hardware including UPS and software. The software in this section refers to the cluster management software provided by the vendor. The 5‐year warranty and support will include the following:

8.1. It may be the case that the vendor procures hardware and/or software from a third‐ party and provides the HPC to NSWC IHD as a package. The 5‐year warranty and support apply solely to the vendor and not to such third‐party vendors. It implies that NSWC IHD will contact the vendor for any warranty and/or support issues and it will be the sole responsibility of the vendor to resolve the issue on its own or in consultation with the third‐party vendor.

NSWC IHD will not be responsible to directly contact the third‐party vendor.

8.2. 24x7 telephone support for any hardware / software issues. Vendor will assign a U.S.

Citizen (preferably with a security clearance) support personnel at their end for this telephone support. Vendor will also provide the single dial phone number to reach the support instead of being routed through several options after dialing.

8.3. NBD replacement of any hardware that fails during the 5 years. A U.S. Citizen technician with U.S Citizenship and a security clearance is required for visiting NSWC IHD for the NBD service at a short notice.

8.4. In select circumstances, NSWC IHD may provide a dedicated port‐to‐port connection to the vendor’s support personnel to remotely access the HPC for troubleshooting or help.

Vendor must designate personnel with a security clearance at their end for this purpose.

8.5. HPC performance depends not only on the state‐of‐the‐art hardware and software, but also on how well the OS is installed, configured, upgraded, and maintained on the nodes. For this reason, even though NSWC IHD will procure and provide RHEL OS licenses, the maintenance of RHEL OS will rest on the vendor in association with the NSWC IHD IT staffs and RHEL team for troubleshooting, any dependencies during upgrades, etc. The vendor will have free access to RHEL via the licenses procured by NSWC IHD.

8.6. NSWC IHD does not expect any support needed for Intel oneAPI and AMD Zen Software Studio. However, a few rare occasions may be foreseen when the vendor’s support may be required, for example, conflict of the future upgrades of these two software programs with the OS or other software, future upgrades being installed in a new folder that changes the environmental variables, etc. NSWC IHD will make earnest attempt to solve such issues by contacting Intel and AMD respectively. But the vendor agrees to troubleshoot or provide support for any conflict with the RHEL OS.

8.7. Similar to above, the vendor’s support may be needed, and vendor will provide the support for any future issues with SLURM, with or without the help of SchedMD. The issue foreseen, but not limited to, is if future RHEL OS upgrades conflict with the installed SLURM, or vice versa, that disable the queues.

8.8. UPS battery life is in general, three years. The 5‐year support applies to all the UPS and UPS batteries for uninterrupted operation of the HPC. This implies that the vendor will replace the batteries free of cost at the end of their life as a part of the maintenance. Vendor, should include the batteries replacement cost in the quote if necessary for one time replacement in 5 years. However, batteries are susceptible to fail before the end of their life.

Hence, the vendor will provide the best quality batteries and replace them free‐of‐cost as a part of the 5‐year warranty/ support service.

9. OTHERS

9.1. Vendor’s Visit to NSWC IHD: The technical personnel of the selected vendor will visit NSWC IHD at the earliest after securing the contract to:

Determine the optimal position of the HPC racks in the server room.

Advise on the dedicated electrical connection to be drawn to the server room and type / number of wall / under floor outlets for the HPC.

Advise on the present capacity of the air‐conditioning and on the standby unit to be installed by NSWC IHD. The room has under floor air cooling and the present capacity is considered sufficient for the HPC.

NSWC IHD IT team is designing a secure procedure to connect the HPC to the internal network for end users to connect to the HPC machine from their desk without network interruptions and loss of bandwidth while meeting the security and STIGS measures. The vendor’s visiting team will consult and advise the NSWC IHD IT team on this subject matter.

9.2. CRON Development‐1: It is being increasingly found that the scratch files written on local scratch during program executions are not deleted automatically after execution completes. They need to be manually deleted more frequently than desired. Vendor in collaboration with NSWC IHD IT team, will develop and install CRON script which will automatically detect the program termination and delete all files created temporarily by the program on local scratch.

9.3. CRON DEVELOPMENT‐2: HPC has NVIDIA GPUs which will require NVIDIA drivers. It is the case that RHEL OS upgrades do not upgrade the NVIDIA drivers and these drivers need to be re‐ built after the RHEL kernel upgrades. This may not be the case for the multi‐node HPC or for the RHEL version being installed. Vendor will confirm this with RHEL during OS installation. If that is the case, vendor in collaboration with NSWC IHD IT team will develop and install a CRON script which will detect kernel upgrade(s) and automatically build NVIDIA drivers (if required) on individual nodes.

9.4. Stress Test: After completion of the software installation and configuration, vendor will carry out stress testing by running the HPC at 100% load for a specified time. Any defective component or the components that fails during stress testing will be replaced and stress testing will be repeated till the test shows all components working without failure.

9.5. File System: Vendor will determine the optimal file system for HPC in consultation with the NSWC IHD Technical POC and IT team, in that order. The file system will play a major role in the optimal performance of the HPC. There may be a few file systems that are considered superior, but they may not be allowed due to the security and STIGS requirements. Hence, determining the optimal file system within the security measures is critical. However, the selected file system must be capable to scale up in the future for accommodating additional racks.

Table 1: Final Configuration and Capacity (C&C) of HPC and Other Requirements for NSWC IHD Supercomputing Facility (refer to section 2 of

SOW).

All Processors are AMD EPYC Series or Better.

ITEM

DESCRIPTION

NUMBER

HARDWARE

NODES COMPONENTS NUMBERS

Login

Number of Nodes 4

Base Clock Frequency of Processor 2.2 GHz

Total Number of Cores 256

RAM Memory Per Core 8 GB DDR4 3200 ECC

Total RAM Memory 2.05 TB DDR4 3200 ECC

Hard Drive Per Node 6.4 TB NVMe

Total Hard Drives 25.6 TB NVMe

Master

Number of Nodes 2

Base Clock Frequency of Processor 3.2 GHz

Total Number of Cores 64

RAM Memory Per Core 8 GB DDR4 3200 ECC

Total RAM Memory 512 GB DDR4 3200 ECC

Hard Drive Per Node 6.4 TB NVMe

Total Hard Drives 12.8 TB NVMe

Compute without

Number of Nodes 24

Base Clock Frequency of Processor 2.2 GHz

GPU Total Number of Cores 1536

RAM Memory Per Core 4 GB DDR4 3200 ECC

Total RAM Memory 6.14 TB DDR4 3200 ECC

Hard Drive Per Node 1.6 TB NVMe

Total Hard Drives 38.4 TB NVMe

Compute with GPU

Number of Nodes 8

Base Clock Frequency of Processor 2.2 GHz

Total Number of Cores 512

RAM Memory Per Core 4 GB DDR4 3200 ECC

Total RAM Memory 2.05 TB DDR4 3200 ECC

Hard Drive Per Node 1.6 TB NVMe

Total Hard Drives 12.8 TB NVMe

NVIDIA A100 40 GB PCIe GPU per Node 1

Total NVIDIA A100 40 GB PCIe GPUs 8

Large Memory Compute and Visualization with GPU

Number of Nodes 8

Base Clock Frequency of Processor 2.2 GHz

Total Number of Cores 512

RAM Memory Per Core 8 GB DDR4 3200 ECC

Total RAM Memory 4.1 TB DDR4 3200 ECC

Hard Drive Per Node 1.6 TB NVMe

Total Hard Drives 12.8 TB NVMe

NVIDIA A100 40 GB PCIe GPU per Node 1

Total NVIDIA A100 40 GB PCIe GPUs 8

Disk

Number of Nodes 1

Base Clock Frequency of Processor 3.0 GHz

Total Number of Cores 32

RAM Memory Per Core 8 GB DDR4 3200 ECC

Total RAM Memory 256 GB DDR4 3200 ECC

480 GB SATA SSD Hard Drive Per Node 2

18TB SAS 12G HDD Hard Drives 12

OTHER HARDWARE

Intra/Inter Connectivity

InfiniBand HDR 100 with 100/200 GB Switches

As needed

10 GB Switches for Management/Remote Access

As needed for 50 Concurrent Users

Built‐in Console

Management Console with KVM Switches

9 UPS To Support Entire System (see notes) As Needed

Racks Enough racks to house the above hardware and UPS and scalable for future additions

As Needed

Temperature and Humidity sensors that are interfaced with the cluster management software to notify their values reaching upper safe limits

SOFTWARE

To be provided, installed / Cluster Management Software configured by the vendor

5 Years Support

To be provided by

NSWC IHD

and

Red Hat Enterprise Linux (RHEL) operating system (OS) for various nodes depending on their functions to be procured on DON

ELA contract with DLT

5 years Premium support recurring cost installed/con figured by the vendor

Intel oneAPI for Cluster with MKL & VTUNE

5 Years Support, 20

AMD Zen Software Studio containing (a) AOCC 3.2, (b) AOCL LibM 3.9, © uProf 3.6,

(d) ZenDNN 3.3, and (e) Spack

5 years Support, 20

Resource Management Software SLURM

1 Year Support with Option to Renew the

Support, 50 Concurrent Users

MobaXterm software for users to connect to the HPC using Windows machine with Linux environment ported to Windows

1 Year License recurring cost

OTHERS

Vendor will visit NSWC IHD at the earliest possible after the NSWC IHD contract/PO award and before shipping to survey the installation site

Visit and suggest any improvements needed for the racks’ placement, electrical and cooling.

Shipping Shipping to Site (FOB)

Installation

Racks ready hardware and software installations and their configurations for ready‐to‐be‐deployed on RDTE network

As per the SOW

STIGS

Help to the NSWC IHD IT Staffs during STIGS

Stress Tests

Entire System Loaded 100% For Specified

Time After installation at NSWC IHD

Training

Training on site NSWC IHD IT Staffs as specified in this SOW after installation and at the end of one year as described above

WARRANTY / SUPPORT

Hardware / Software

5 Yrs. Full Hardware / Software Maintenance Contract with 24x7 Telephone Support and Next Business Day (NBD) Hardware

Replacement Support by a U.S. Citizen and Security Cleared Support Personnel. Vendor / manufacturer (solely or combined) will be solely responsible for this 5 years warranty / support although some other vendor may be the manufacturer of the part/s to be replaced. NSWC IHD will hold vendor / manufacturer responsible and the sole point of contact for any warranty/support needs. UPS batteries are known to fail before their stipulated lifetime, or their lifetime may be less than 5 years. Vendor may include the replacement cost in the total cost of UPS in the quote.

NOTES

A

In the event of power failure, vendor will determine the time needed to terminate jobs waiting in the SLURM queues, terminate all the ongoing users' runs, and safely shutdown the machine. UPS capacity must provide the power needed to safely execute this shutdown sequence without any stress to the machine. The machine will be powered up manually after power restoration. Users will restart the terminated runs from initial state or backup as needed.

B

All hardware must be TAA and GSA compliant. This must be noted in the quote either in the beginning or at the end. Vendor must provide the list of any non‐compliant components at the end of the quote. NSWC IHD reserves the right at its discretion to approve or disapprove such non‐compliant components. The disapproval will lead to rejection without any discussion with the vendor.

C

TPM 2.0 or Higher must be enabled on all motherboards. This must be noted in the beginning or at the end of the quote.

D

NO WIRELESS / BLUETOOTH – It must be noted at the beginning or at the end of the quote and the HPC must not have any Wireless or Bluetooth hardware.

E

NO HARDWARE / SOFTWARE RAID – It must be noted at the beginning or at the end of the quote and the HPC must not have Hardware or Software enabled RAID.

Section SF 1449 - CONTINUATION SHEET

ITEM NO SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

Hardware

FFP

NET AMT

0001AA 1 Each Login Node

FFP

Login Nodes Key Component are:

(a) Number of Nodes - 4

(b) Base Clock Frequency of Processors - 2.2 GHz

(c) Total Number of Cores - 256

(d) RAM Memory Per Core - 8 GB DDR4 3200 ECC

(e) Total RAM Memory per Node - 2.05 TB DDR4 3200 ECC

(f) Hard Drive Per Node - 6.4 TB NVMe

(g) Total Hard Drives - 25.6 TB NVMe NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

FOB: Destination

PSC CD: 7B20

0001AB 1 Each Master Nodes

FFP

Master Nodes Key Components are:

(a) Number of Nodes - 2

(b) Base Clock Frequency of Processors - 3.2 GHz

(c) Total Number of Cores per Node - 64

(d) RAM Memory Per Core - 8 GB DDR4 3200 ECC

(e) Total RAM Memory per Node - 512 GB DDR4 3200 ECC

(f) Hard Drive Per Node - 6.4 TB NVMe (g) Total Hard Drives 12.8 TB NVMe NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AC 1 Each Compute Nodes without GPU

FFP

Compute Nodes without GPU Key Component are:

(a) Number of Nodes - 24

(b) Base Clock Frequency of Processors - 2.2 GHz

(c) Total Number of Cores - 1536

(d) RAM Memory Per Core - 4 GB DDR4 3200 ECC

(e) Total RAM Memory per Node - 6.14 TB DDR4 3200 ECC

(f) Hard Drive Per Node - 1.6 TB NVMe (g) Total Hard Drives 38.4 TB NVMe NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AD 1 Each Compute Nodes with GPU

FFP

Compute Nodes with GPU Key Component are:

(a) Number of Nodes - 8

(b) Base Clock Frequency of Processors - 2.2 GHz

(c) Total Number of Cores- 1536

(d) RAM Memory Per Core - 4 GB DDR4 3200 ECC

(e) Total RAM Memory - 6.14 TB DDR4 3200 ECC

(f) Hard Drive Per Node - 1.6 TB NVMe

(g) Total Hard Drives - 38.4 TB NVMe NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AE 1 Each Large Memory Compute Nodes & Visualizati

FFP

Large Memory Compute Nodes with GPU Key Component are:

(a) Number of Nodes - 8

(b) Base Clock Frequency of Processors - 2.2 GHz

(c) Total Number of Cores - 512

(d) RAM Memory Per Core - 8 GB DDR4 3200 ECC

(e) Total RAM Memory - 4.1 TB DDR4 3200 ECC

(f) Hard Drive Per Node - 1.6 TB NVMe (g) Total Hard Drives - 12.8 TB NVMe

(h) NVIDIA A100 40 GB PCIe GPU - 1

(i) Total NVDIA a100 40 GB P Cle GPUs - 8 NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AF 1 Each Disk Node

FFP

Disk Node Key Component are:

(a) Number of Nodes - 1

(b) Base Clock Frequency of Processors - 3.0 GHz

(c) Total Number of Cores - 32

(d) RAM Memory Per Core - 8 GB DDR4 3200 ECC

(e) Total RAM Memory - 256 GB DDR4 3200 ECC

(f) Hard Drive 480 GB SATA SSD Per Node - 2

(g) Hard Drive 18TB SAS 12G HDD - 12 NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AG 1 Each Intra/Inter Connectivity

FFP

nfiniBand HDR and Network Switches Key Component are:

(a) InfiniBand HDR 100 with 100/200 GB Network Switches as needed for the entire HPC

(b) 10GB Network Switches for Management / Remote Access as needed for 50 concurrent users NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AH 1 Each Built-in Console

FFP

Management Console Key Component are:

(a) Built-in KVM LED Console with KVM switches - 1

NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AJ 1 Each UPS for the HPC

FFP

UPS Key Component are:

(a) Set of UPS to support HPC as needed NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0001AK 1 Each Racks

FFP

Racks Key Component are:

Sufficient Number of Racks to house the NODES and UPS of the HPC and scalable for future additions.

0001AL 1 Each Temperature and Humidity Sensors

FFP

ITemperature and Humidity sensors that are interfaced with the cluster management software to notify their values reaching upper safe limits

0002 1 Each Software

FFP

Cluster Management Software NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

Support Services

FFP

0003AA 1 Each NSWC IHD Site Visit

FFP

The Visit Key Component are:

Vendor will visit NSWC IHD at the earliest possible after the contract/PO award and before shipping to survey the installation site and suggest any improvements needed for the racks’ placement, electrical and cooling.

0003AB 1 Each Installation

FFP

Vendor will install HPC in accordance with the attached SOW. The key installation components are:

HARDWARE INSTALLATION:

Racks ready hardware and software installations and their configurations for ready-to-bedeployed on RDTE network.

SOFTWARE INSTALLATION as described in SOW

STIGS

Stress Test NOTE: The requirements in DFARS 252.211-7003, Item Identification and Valuation, are applicable for this line item. The contractor shall provide DoD unique identification or a DoD recognized unique identification equivalent.

0003AC 1 Each Training

FFP

Training Key Component are:

Training on site NSWC IHD IT Staffs as specified in this

SOW

0003AD 1 Each Hardware/Software Warranty/Support

FFP

Hardware / Software Support for 5 Years after Installation Key Components are:

(a)5 Yrs. Full Hardware / Software Maintenance Contract with 24x7 Telephone Support and Next Business Day (NBD) Hardware Replacement Support by a U.S.

Citizen and Security Cleared Support Personnel.

Vendor / its parent company will be solely responsible for this 5 years warranty / support although some other vendor may be the manufacturer of the parts to be replaced. NSWC IHD will hold vendor / its parent company responsible and the sole point of contact for any warranty/support needs. NSWC IHD will explore and provide, if feasible, port-to-port dedicated network connection to trouble shoot any urgent need that may arise. UPS batteries are known to fail before their stipulated lifetime, or their lifetime may be less than 5 years. Vendor may include the cost in the quote for batteries that may need to be replaced before 5 years

0004 1 Each Shipping

FFP

Shipping - Vendor will ship all hardware and software to NSWC IHD with any of the following two options or their combination:

(1) Vendor can choose to ship all the nodes and UPS assembled inside multiple racks. Preferred method is to place UPS in the lower half and nodes in the upper half of racks. Alternately, nodes and UPS can be placed in separate racks.

(2) As an alternative, vendor can choose to ship all the individual hardware and software in packets and assemble them on site during installation.

Vendor is required to inform NSWC IHD of the mode of shipment and tracking numbers.

INSPECTION AND ACCEPTANCE TERMS

Supplies/services will be inspected/accepted at:

CLIN INSPECT AT INSPECT BY ACCEPT AT ACCEPT BY

0001 N/A N/A N/A N/A 0001AA Destination Government Destination Government 0001AB Destination Government Destination Government 0001AC Destination Government Destination Government 0001AD Destination Government Destination Government 0001AE Destination Government Destination Government 0001AF Destination Government Destination Government 0001AG Destination Government Destination Government 0001AH Destination Government Destination Government 0001AJ Destination Government Destination Government 0001AK Destination Government Destination Government 0001AL Destination Government Destination Government 0002 Destination Government Destination Government 0003 N/A N/A N/A N/A 0003AA Destination Government Destination Government 0003AB Destination Government Destination Government 0003AC Destination Government Destination Government 0003AD Destination Government Destination Government 0004 Destination Government Destination Government

DELIVERY INFORMATION

CLIN DELIVERY DATE QUANTITY SHIP TO ADDRESS DODAAC /

CAGE

0001 N/A N/A N/A N/A

0001AA 4 mths. ADC 1 NSWC IHD

RECEIVING

4522 MCMAHAN ROAD B116

INDIAN HEAD MD 20640-5125

301-744-4221

N00174

0001AB 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AC 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AD 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AE 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AF 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AG 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AH 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AJ 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AK 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0001AL 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0002 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0003 N/A N/A N/A N/A

0003AA 4 mths. ADC 1 NSWC IHD

RECEIVING

4522 MCMAHAN ROAD B116

INDIAN HEAD MD 20640-5125

301-744-4221

0003AB 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0003AC 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0003AD 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

0004 4 mths. ADC 1 (SAME AS PREVIOUS LOCATION)

CLAUSES INCORPORATED BY REFERENCE

52.202-1 Definitions JUN 2020 52.203-3 Gratuities APR 1984 52.203-6 Alt I Restrictions On Subcontractor Sales To The Government

(JUN 2020) -- Alternate I

NOV 2021

52.203-11 Certification And Disclosure Regarding Payments To Influence Certain Federal Transactions

SEP 2007

52.203-12 Limitation On Payments To Influence Certain Federal Transactions

JUN 2020

52.204-21 Basic Safeguarding of Covered Contractor Information Systems

NOV 2021

52.204-23 Prohibition on Contracting for Hardware, Software, and Services Developed or Provided by Kaspersky Lab and Other Covered Entities

NOV 2021

52.209-10 Prohibition on Contracting With Inverted Domestic Corporations

NOV 2015

52.212-4 Contract Terms and Conditions--Commercial Products and Commercial Services

DEC 2022

52.225-1 Buy American--Supplies OCT 2022 52.225-13 Restrictions on Certain…