HPC_Consolidated Questions List Plus Answers.docx

1. VENDORS QUESTIONS: What are the scope and expectations of the research software engineering (RSE) labor? Is the expected labor and skills solely outlined in Sections 3 and 4? If not, what additional labor and skills requirements are expected?

Answer: There is no RSE labor is characterized in section 3 of the SOW. The scope of the RSE labor and NIST’s expectations for RSE labor are fully characterized in section 4.1 of the SOW.

2. VENDORS QUESTIONS: What is the expectation of “availability” of resources (computing and labor) as outlined in Section 6? Please provide a template or draft SLA (service level agreement) for review based on NIST expectations.

Answer: NIST’s expectation for system availability under this contract, using the FIPS 199 definition, is “Low.” Section 6 of the SOW specifies that NIST requires the Awardee to adhere to a Service Level Agreement that the Awardee normally applies to the systems provided through the contract.

3. VENDORS QUESTIONS: Is interactive processing a consideration, or is this entirely batch job processing?

Answer: Quoting from section 3.5 of the SOW, “…NIST researchers will be permitted to submit batch and interactive jobs…”.

4. VENDORS QUESTIONS: What performance and cost considerations are given for ingress and egress network and data transfers?

Answer: With respect to performance NIST, has connectivity to National Research and Education networks such as Internet2 allowing for 100Gbps bandwidth. With respect to cost, section 3.4 of the SOW states “Data transfers to and from Awardee will not incur network bandwidth charges to NIST and will transit National Research & Engineering network infrastructure to the extent NIST & Awardee’s network engineers can control.” NIST will amend the language in the SOW to make NIST’s requirement clearer.

5. VENDORS QUESTIONS: Is ARM or IBM POWER chipset a consideration? Or must this be x86-64 (AMD or Intel) based? Is there a chipset for vendor architecture preference?

Answer: The computational capabilities described in section 1.4.1 Factor 1 of the Instructions and Evaluation document do not stipulate a chipset architecture; only minimum performance. Note that the software build environment also described in that factor must be compatible with the architecture(s) offered.

6. VENDORS QUESTIONS: What chipset accelerator engines are required for NIST applications?

Answer: AMD, Intel and Nvidia products are all acceptable.

7. VENDORS QUESTIONS: What are the minimum and maximum storage requirements?

Answer: For the period of performance NIST anticipates a range of 5-50TB in aggregate for the user data storage described in in section 1.4.1 (Factor 1) of the Instructions and Evaluation document.

8. VENDORS QUESTIONS: What considerations are there for home and archive storage requirements?

Answer: As described in in section 1.4.1 (Factor 1) of the Instructions and Evaluation document, “user data storage of at least 1 Terabyte per user with FIPS-validated encryption at rest”, i.e., ‘home’ storage. NIST researchers will archive their data using NIST’s data storage services.

9. VENDORS QUESTIONS: What is the preference for a high-performance data storage system? Parallel (share nothing), Object (e.g., S3), or NFS v3 or v4 (share everything) filesystem?

Answer: NIST’s preference is a low-latency, high-performance shared file system capable of supporting HPC applications.

10. VENDORS QUESTIONS: What is the preferred job scheduler? SLURM, PBS Pro, Kubernetes? Other?

Answer: NIST researchers have current experience with SLURM, hence that would be preferred.

11. VENDORS QUESTIONS: What are the expectations, needs, and suggested agenda for the onboarding session(s)?

Answer: NIST’s requirements are characterized in section 4.8 of the SOW. For onboarding, NIST researchers will need to understand how to effectively access and utilize the resources and services the Awardee makes available to NIST under this contract as well as how to get help when issues arise.

12. VENDORS QUESTIONS: What constitutes research software engineering (RSE) applications?

Answer: In the context of this contract, software that is produced by the Awardee’s RSE in conjunction with NIST researchers to assist the researchers in the performance of their work while utilizing the Awardee’s system resources.

13. VENDORS QUESTIONS: What constitutes AI frameworks, models, and applications?

Answer: In the context of this contract AI frameworks are suites of software tools which provide high-level access to common abstractions used in AI and machine learning applications, often optimized for high-concurrency GPU hardware. Common frameworks include Tensorflow and PyTorch. Models include deep and recurrent neural networks and LLMs. The range of applications is very broad, and includes those which use these models and frameworks to make data-driven conclusions from large data sets.

14. VENDORS QUESTIONS: What constitutes HPC applications? What is the maximum job size (nodes, MPI processes, etc.)

Answer: In the context of this contract, an HPC application is any software that executes on the Awardee’s system resources in the furtherance of NIST’s research. The maximum job size is a function of the system resources the Awardee makes available to NIST.

15. VENDORS QUESTIONS: It is impossible for awardee to “guarantee computational time” if “the specific quantities of the compute, storage, and research software engineering effort to be consumed in the course of NIST’s research can not be precisely specified in advance.”

Can NIST provide a specific minimum compute quantity, that awardee shall guarantee to be available?

Answer: Section 2.1 of the SOW states “Awardee shall guarantee computational time for NIST projects on requested systems within five (5) business days.” This statement and the preceding sentence convey that the Awardee will make one or more systems available to NIST researchers and that when a NIST researcher submits a job to a system made available to NIST the job will initiate within 5 business days. In section 1.4.1 Factor 1 of the Instructions and Evaluation document the relevant requirement states “Provide compute job scheduling capabilities such that NIST-submitted jobs anticipated to complete within 48 hours will be started no later than 5 days after submission.” NIST will amend the SOW language to align with the language used in the Instructions and Evaluation document.

Looking at 3 months of queueing data from one of NIST’s conventional CPU HPC systems, the vast majority of submitted jobs required 40 or fewer nodes, 2,560 or fewer CPUs, 128GB/node, and completed in less than 5 hours. Looking at the same time period for NIST’s small GPU-accelerated HPC system, the vast majority of submitted jobs required 4 or fewer nodes, 160 or fewer CPUs, 575GB/node, and completed in less than 4 hours.

16. VENDORS QUESTIONS: Is NIST open to alternative compute infrastructure that may not meet stated technical specs in the RFP, but otherwise allows NIST to fulfill its computational mandate?

E.g. If a job is typically expected to complete in 48 hours, but may not be queued to execute for up to 5 days; versus choosing an alternative compute type that completes in 70 hrs and executes immediately.

Answer: No, the nature of some of NIST’s research computation is specifically designed for execution on multi-node HPC architectures.

17. VENDORS QUESTIONS: What is the desired concurrency of multinode jobs? How many nodes (or a range) for each compute type (CPU and GPU) are typically required for a multi-node job?

Answer: Looking at 3 months of queueing data from one of NIST’s conventional CPU HPC systems, the vast majority of submitted jobs required 40 or fewer nodes, 2,560 or fewer CPUs, 128GB/node, and completed in less than 5 hours. Looking at the same time period for NIST’s small GPU-accelerated HPC system, the vast majority of submitted jobs required 4 or fewer nodes, 160 or fewer CPUs, 575GB/node, and completed in less than 4 hours.

18. VENDORS QUESTIONS: Can an equivalent high-bandwidth, low latency interconnect be used in lieu of Infiniband?

Answer: Yes.

19. VENDORS QUESTIONS: Does NIST require/favor a software build environment that supports user selection of a library package and particular version?

Answer: Yes, the diversity of NIST’s research encompasses a wide range of software build environments.

20. VENDORS QUESTIONS: Does NIST require reproducibility/traceability of HPC jobs by preserving input/output and job metadata?

Answer: Yes.

21. VENDORS QUESTIONS: Does NIST favor an awardee that can provide a differentiated mix of HPC capacity, to best suit the workloads of the NIST researcher?

Answer: The evaluation factors are stipulated in the Instructions and Evaluation document.

22. VENDORS QUESTIONS: Are illustrations and diagrams included in the total page counts in the Phase I and Phase II submissions?

Answer: Yes.

23. VENDORS QUESTIONS: We were reviewing the recently posted solicitation for HPC services and were a little confused by some of the language. Normally I wouldn’t think that just adding professional services can transform an environment into a Leadership Class Computing Facility.

Does NIST intend to augment the computing and storage environment, or only the services supporting the current HPC cluster?

Answer: Quoting from the Background section of the SOW “Owing to imminent refresh of NIST’s on-premise HPC equipment NIST has an urgent need for external HPC services to support continuation of NIST’s research.” (Emphasis added). During the refresh of NIST’s on-premise HPC equipment, the availability of on-premise computational capacity will be reduced. NIST is soliciting for external services described in the SOW meeting the characteristics described in the Instructions and Evaluation document.

image1.png