RFI 4-B162-I--05428-00 - ALCF Storage.pdf

4-B162-I--05428-00

Argonne National Laboratory Managed by UChicago Argonne, LLC, for the U.S. Department of Energy's Office of Science

June 11, 2024

To: Respondents

Subject: Request for Information – ALCF Storage System

The Argonne Leadership Computing Facility at Argonne National Laboratory is requesting your response to the subject Request for Information (RFI). The RFI is for informational purposes only and may not result in a purchase or a contract award. Please note that all responses to this RFI will be nonbinding. This RFI does not commit Company and/or Government to pay any cost for the preparation and submission of any information in response to this RFI.

PREFACE

The Argonne Leadership Computing Facility (ALCF) currently deploys a high-performance computer architecture named Polaris (HPE Apollo Gen10+) with 560 compute nodes, Slingshot 11 system interconnect with adaptive routing. In this approximate timeframe, the ALCF expects to go live with a 1.25 ExaFLOPS system named Aurora (HPE Cray EX) with more than 10,000 compute nodes, Intel DAOS Burst Buffer and Slingshot 11 system interconnect with Dragonfy topology and adaptive routing.

The ALCF has deployed multiple high speed parallel file systems to support the data generated by researchers and computational scientists. These include two 650 GB/s 100PB Lustre file systems, both using Infiniband HDR. The ALCF is exploring the possibility of a storage solution that will provide the bandwidth, reliability and scalability required for the next-generation of HPC and AI systems.

In addition, we expect other systems, including the subject storage system, to contribute no less than 10,000 additional nodes to the ecosystem. It is expected that the proposed storage system will be connected to one or more of the existing ALCF resources.

Please consult the Aurora fact sheet for more details on the system.

https://www.alcf.anl.gov/aurora

OBJECTIVE

The ALCF is pursuing a storage system that will provide both the capacity and performance necessary to support our deployed systems as well as future systems currently being planned. The solution may use data lifecycle management across multiple storage tiers http://www.uchicagoargonnellc.org/ http://science.energy.gov/ https://www.alcf.anl.gov/aurora to meet our user’s capacity and bandwidth requirements. It should support connectivity to our existing infrastructure, compute clusters and Linux based OS platforms.

Proposals that provide detailed explanations and architectural diagrams describing how this will be accomplished will receive preferential consideration. The response must propose what can be delivered by a Q2CY25 delivery target, and additionally what could be delivered by a Q4CY25 delivery target if the additional half a year would provide enough time for new technologies. The response should also include a description of the lead time based on contract signing required to meet the delivery target specified.

General storage requirements

• Capacity: ~400PB desired

• Peak throughput: ~6TB/s

• Peak IOPS: ~240M/s

• ALCF budget $15-20 million. Fully burdened.

• Redundancy for both data and metadata (e.g. Declustered RAID/Erasure coding)

• Scalable units which enable the expansion of capacity, IOPS, and bandwidth, ideally independent of each other.

• POSIX compliant

• Capable of 30,000 node concurrent mounts

• No single point of failure

• Must be highly available o Minimum of 4 months between outages.

o Ideally all maintenance (hardware, firmware, software) can be accomplished while the system is live with minimal performance impact.

• Security updates o Describe policies related to zero day exploits o Standard OS updates

• Standard client package provided by the vendor

• Support for minimum of 5 years.

• TDS system o We require a second system for testing.

o It should be the smallest system that will allow us to exercise full functionality

Responses should address the following topics

• Detailed description of the architecture, both hardware and software.

o Basic Specifications o Capacity o Bandwidth o IOPS o Storage interfaces supported (POSIX, NFS, S3, etc) o Network interfaces supported (Infiniband, Ethernet, Slingshot, etc) o Physical Characteristics by rack, scalable unit, and the entire system o Physical Footprint o Electrical Requirements o Weight o Cooling Requirements o If a tiered storage solution is proposed address the following as a minimum:

o Describe support for multiple pools. HDD/SSD tiers.

o Automated data movement between tiers

Transparent to users o Do the tiers appear as a single unified namespace or do they have separate namespaces.

o Integration with potential platform storage (optional)

DAOS

o Maintenance o FRU replacements o Management stack upgrade o Firmware upgrade o Describe support for AI/ML workloads o High IOPS o Small IO o Small file reads o Random IO o Client side caching o High concurrency

Use case: In training AI model (such as CosmoFlow) in data parallel mode, during the data loading, each MPI process may spawn multiple threads (num_workers = 2-8) to load data from the storage. In total, there might be (num_gpus) 8 * (num_workers ) 10,000 accessing the storage system concurrently.

o Describe support for RDMA from storage into GPU memory o Scalable metadata performance

Single job could access hundreds of millions of files.

• Use cases: CosmoFlow dataset has 1 million files (in numpy array format or tfrecord format); ResNet50 dataset has 14 million image files.

o We anticipate the datasets for Large Language Models will have many more files (https://github.com/lmmlzn/Awesome-LLMs- Datasets).

• Use case: 10K nodes loading python modules simultaneously https://github.com/lmmlzn/Awesome-LLMs-Datasets https://github.com/lmmlzn/Awesome-LLMs-Datasets o Performance o Comment on number of concurrent streams supported and number need to reach maximum performance o Scalability o Building Block architecture o Redundancy o Describe support for RDMA o Kernel bypass, host CPU bypass options o Reliability o End-to-end Data Integrity o File system(s) supported o Lustre and/or IBM Spectrum Scale should be represented in addition to any other options o For Lustre, must be compatible with existing HPE Lustre client o Must co-exist with existing HPE lustre client o Management tools o Source availability and relevant communities around the tools o Use of an industry standard configuration management tool across the system which will be the primary means to make configuration changes on system o An implementation that allows ALCF to make supported changes via configuration management tool is required o Reporting o Scalability of monitoring tools o Method of data retention for metrics o Ability to integrate with existing monitoring tools, including real time metrics and event data o Compatibility with existing infrastructure o Upgrade paths for capacity and performance o Discussion of tradeoffs of alternate solutions or configurations if appropriate o A five year plan for ongoing support of the proposed storage system following deployment.

o On site parts cache o Cost estimate o Cost of full solution plus annual maintenance and support o Cost per building block o Describe your on-site maintenance options.

o Describe serviceability o How easily is the storage system maintained.

o Describe life-cycle management tools

Any questions regarding the RFI must be sent to the undersigned on or before June 28, 2024. Responses to this RFI are due no later than July 14, 2024, on or before midnight Central Time. Responses are expected to be proprietary and/or business sensitive. Those responses shall have the documents and the associated pages marked accordingly. An official Request for Proposal is anticipated to be submitted by the Laboratory in November 2024.

Responses, as well as questions, for this RFI should be submitted electronically to Jeremy Sladetz at jsladetz@anl.gov. All transmittals to this RFI should reference the following in the subject line: “RFI 4-B162-I--05428-00 - ALCF Storage System.”

Sincerely, Jeremy Sladetz Procurement Specialist II Argonne National Laboratory