Enclosure No. 2B - Appendix B Statement of Work.docx

ALCF Archive Storage SoW

Argonne National Laboratory Appendix B - Statement of Work for ALCF Archive Storage Disk Cache April 29, 2026

1 Introduction The Argonne Leadership Computing Facility (ALCF) currently deploys compute clusters including Aurora and Polaris that can produce large amounts of data very quickly on the multiple high speed parallel filesystems in service. To maintain sufficient usable capacity, we need the ability to efficiently move data to the archive. The ALCF currently maintains a High-Performance Storage System (HPSS) with a 9PB disk cache tier. Data is initially written to the disk cache to provide performance then migrated to tape to maintain capacity. The disk cache and associated data mover servers, highlighted in the diagram below, are at end of life for service and must be replaced.

Tape Data Movers Archive Clients

FC

HPSS Core (HA)

Ethernet

FCData Movers

FDR IBDisk Cache Metadata Repository

Existing Archive Infrastructure

1.1 Executive Summary

The ALCF is pursuing a block storage system that will support high speed parallel data transfers to efficiently ingest large amounts of data into the archive while also providing the bandwidth to simultaneously migrate data to tape. It should include Linux based data mover hosts that support connectivity to our existing network infrastructure (100GbE, 200GbE, 400GbE), and a standard transport to the block storage (ie.eth,ib,fc, sas). The data movers may be external servers or embedded virtual machines.

2 Definitions [MR] Mandatory Requirement - A mandatory requirement represents a level of performance that the response must meet to be considered for evaluation.

[TR] Target Requirement - A target requirement represents the desired level of performance for a given feature.

[MO] Mandatory Option - An optional feature that the response must provide to be considered for evaluation [TO] Target Option - An optional feature that is desirable for the response to meet.

3 Use Cases The following section provides information about how the ALCF operates and does not reflect any requirements or require any response by the Offeror. The information provided should allow the Offeror to better understand how the ALCF will integrate the storage into the wider facility and tailor the response to our specific needs.

3.1 Parallel Connectivity

The existing disk cache storage is accessed by 16 external data mover hosts connected to the storage system through 16 Infiniband host ports, providing the parallelism to maximize aggregate bandwidth. Each data mover host runs a data mover daemon and has the requisite software packages installed.

3.2 Storage Presentation

The block storage is presented to the Linux data mover hosts as raw unformatted LUNs that are imported to the archive software. Virtual volumes are then created from one or more imported LUNs.

3.3 Monitoring

The ALCF monitors storage systems through integration into custom and third-party monitoring solutions that leverage the vendor provided APIs and tools. Third party tools that are currently leveraged are LDMS, Check mk/Nagios, TimescaleDB, and ElasticSearch.

Metrics and event data are collected and analyzed to help with troubleshooting system issues, as well as understand the usage by our users. Users can also make use of the I/O metrics to help optimize their applications.

Consuming structured data are vastly preferred over unstructured data, such as syslog. A system that provides a defined and documented set of RAS messages is preferred over having to discover the meaning of event data that is provided by the system.

4 Requirements

4.1 [MR] Storage System The storage system will be a standalone appliance primarily accessible by Linux data mover hosts that have direct access to the block storage over standard transport. Each data mover host will have access to all block devices in the storage system.

The storage system will present raw unformatted disk to the Linux hosts as /dev/sd[a-h][0-8]. The Linux raw device driver is used to bind a Linux raw character device to a block device. Only block devices may be used. The number of block devices is limited only to that supported by the HPSS server memory capacity. The Current configuration is 512GB of memory for 88 block devices.

4.1.1 [TR] Accessibility Preference will be given to solutions that include internal virtual machines to provide access to the storage. The VMs must deliver the necessary processing power, memory, device channel bandwidth and networking bandwidth to support the data mover processes and workload.

If the proposed solution requires external Linux hosts, the offeror will provide as many as 16 Intel Xeon or AMD based servers. External servers will access the storage over standard transport (ie. eth, ib, fc, sas) connections to host ports on the storage system.

The data mover hosts whether VMs or physical servers must be able to be provisioned with a current RHEL distribution.

Node Requirements:

· 1RU rackmount chassis with redundant PSUs with 208v support.

· Single Socket CPU with 16 3.0Ghz cores. (Intel Xeon or AMD preferred)

· 256GB DDR5 RDIMM RAM

· Networking and I/O options including a minimum of PCIe slots with adapters to support redundantly connected 400GbE.

· Minimum of two 300GB hardware RAID controlled SSD, configured as RAID1 At least 1 GB of memory should be configured on the Mover nodes. More memory is required for nodes that support many devices, especially disks, and have large numbers of concurrent end-user requests. 256GB of memory supports 88 devices in the current configuration. Additionally, the size of the Mover’s internal buffers impacts the Mover’s memory requirements. Two buffers are used by each Mover process to handle I/O requests.

4.2 [MR] Capacity The storage system will provide a minimum of 10PiB usable capacity. The capacity excludes any overhead of data redundancy and reliability and will not include any storage used for hot sparing.

4.3 [TR] Performance If SSD, the storage system should support minimum aggregate throughput of 140GB/s and 5 million IOPS. Block device will sustain ~500 MB/s IO.

If HDD, the storage system should support a minimum aggregate throughput of 90GB/s and 3.2 million IOPS. Block device will sustain ~200MB/s IO.

It should also support a minimum of 16 parallel data streams to achieve 90% of line rate on a 400GbE link.

4.4 [MR] Monitoring The offeror will describe monitoring tools provided with the storage system and identify any data collection processes that can or will run on the data mover hosts.

4.5 Reliability

4.5.1 [MR] Durability The Offeror will describe the durability model of the storage system.

4.5.2 [MR] Serviceability The Offeror will provide a serviceable storage system, where all components will be field-replaceable. In addition, Offeror will describe the individual field replaceable units FRUs), in terms of their impact to operation and integrity of the storage system. Description of FRUs should include, at a minimum, whether that FRU is hot-swappable, warm-swappable, or cold-swappable. The Offeror will describe situations where maintenance of specific serviceable items presents a change to the resiliency or data integrity of the I/O system during that maintenance period.

4.5.3 [MR] Fault Tolerance The storage system will be highly reliable and available. It will be tolerant of electrical and mechanical distribution (power and cooling) events. Tolerance to electrical power events may be mitigated through various techniques such as, but not limited to, the use of diverse power supplies or mechanisms that ensure that data can be drained from caches. Tolerance to mechanical power events will include extended operating ranges for air-cooled equipment where no supplemental air cooling is available. The storage system will be designed with hardware and/or software redundancy schemes that ensure the ability to recover data written to the storage system under a double failure of the same component type. In the case of storage media failures, the rebuild processes will be fully automated and initiated by the storage system. The Offeror will describe the failure modes of the storage system including the caches, specifically single points of failure inherent in the design, as well as hardware and software features that protect against failures. The description will include both electrical and cooling mechanisms, and how faults or degraded electrical and mechanical aspects might affect the storage system. The Offeror will fully describe all storage system reliability, availability, and integrity aspects including a description of all software and hardware redundancy schemes. The Offeror will describe failure modes that would lead to the inability to recover data written to the storage system. The Offeror will describe any additional redundancy schemes available and any other reliability features provided by the storage subsystem.

4.5.4 [MR] Hot Swapping Support The storage system will support hot swapping of field replaceable components with minimal impact to the operating state of the system for all components including power supplies, fans, controllers, storage media, cabling, host adapters, and drive enclosure bays. If hot swapping is not technically possible, the Offeror will identify and describe all components that do not support the hot swapping capability.

4.5.5 [TR] Scheduled Availability The storage system should demonstrate a scheduled availability level of not less than 99.5%. Unscheduled downtimes are defined as any period in which any data stored within the storage system is inaccessible or any period in which new data cannot be stored within the storage system.

4.5.6 [MR] Data Integrity The storage system will have built-in mechanisms to protect against silent data corruption.

4.5.7 [MR] Visible Fault Lights All storage system hardware components will have externally visible fault lights or displays to indicate the failure and location of the failure for any commonly replaced FRU.

4.6 Management Network

4.6.1 [MR] Integration The storage solutions will be integrated into the existing management and monitoring networks maintained by the ALCF. This is an Ethernet-based network used for system builds, management, monitoring, and configuration. Industry standard Ethernet technologies are used to provide various features as required by the various systems installed on this network.

All individually managed units, devices, or components must meet the following minimum network requirements.

· An 8pin, 8 conductor, 1000BASE-T compatible Management Ethernet Port.

· A software interface for configuring a single IPv4 address, through which all necessary system management functionality can be performed and reached by connecting to the Management Ethernet Port and only the Management Ethernet Port.

· An out-of-band management interface through which a minimum set of functionalities is presented such that the IPv4 configuration of the Management Ethernet Port can be established in absence of any additional connectivity. This may be an RS-232 serial interface, a keyboard video and mouse interface, a point-to-point Ethernet interface, or any similar technology that serves to allow an administrator with physical access to the managed unit a mechanism by which the administrator can instantiate the basic communication parameters to enable full functionality of the Management Ethernet Port.

The Offeror should indicate they comply with these requirements.

4.7 Facility

The storage solution will be sited in the data center in the Theory and Computing Sciences Building (Building 240) on the Argonne Campus in Lemont, IL, hereinafter referred to as TCS-DC. The TCS-DC floor is a 48” raised floor over a concrete slab. The raised floor is an ASM FS-Series raised floor system that can support a concentrated load of 1500 lbs. and a uniform load rating of 700 lbs./ft2 when the under-floor steel panel is unaltered.

The planned combined power budget for the storage is 600 kW. 480Vac power is available in TCS-DC. Air cooling will be available at TCS-DC through the plenum beneath the raised floor.

All unpacking/uncrating of equipment may need to be done at the loading dock or in an adjacent corridor.

The loading dock for the TCS-DC will have at least 600 ft2 area available with overhead roll-up doors (8’ by 10’), that will accommodate (1) tractor semi-trailers loading/unloading at a time.

4.7.1 [TR] Minimal Electrical Connections The Offeror’s solution should minimize the total number of electrical connections that are required, both inside and outside the racks. For 415Vac power input, the site will provide either 3P+E or 3P+N+E with IEC60309 plugs preferred for connecting the racks. If the preferred plugs are not available for the proposed system, the Offeror will describe the connection type(s).

4.7.2 [MR] Fault-Tolerant Power Distribution All racks shall be fed from two separate power sources and be able to run unencumbered when only one power source is available. When both power sources are available, the electrical load shall be evenly distributed between both sources except under low power conditions where it is permissible for one power supply to go into a low-power state, causing a small imbalance between sources.

4.7.3 [MR] Tolerance of Power Quality Variation While the electrical power into the TCS-DC is normally clean and stable, it is not conditioned. The proposed system will be tolerant of power quality events. The Offeror will describe the tolerance of their power system to power quality events in terms of both voltage surge and sag, and in duration. 208-240Vac components of the Offeror’s solution may be supported by uninterruptible power supply (UPS) systems in dual-fed configurations that can provide tolerance (short term ride-through) to power quality events.

4.7.4 [MR] Power Factor and Harmonic Current Requirements The power factor of the Offeror’s proposed solution, which includes the I/O subsystem, network components, and other infrastructure, will be 0.95. At benchmark power levels, the maximum total harmonic current and the maximum individual harmonic current levels will meet recommendations provided in Table 2 of IEEE STD 519-2014 for Isc/Il <20.

4.7.5 [MR] Cooling Requirements The air cooling required for the proposed system will be listed in relation to the heat load created by the operation of the system. The Offeror will specify the environmental conditions required in CFM, temperature, and humidity of each solution. The Offeror will specify the cooling envelopes required by each rack type, the minimum and maximum thresholds for the environmental conditions, what alarm set points (warning, alarm, critical, etc.), the allowable excursion magnitudes and durations, and how the previously mentioned items may change at idle, typical/normal, and maximum operating points.

5 Maintenance and Support The Offeror will include 9x5 Next Business Day (NBD) support but will also propose optional 12x7 support model. Support will include problem escalation procedures that can be invoked when necessary. The Offeror is expected to provide on-site support services to achieve timely resolutions to significant hardware or software problems. The Offeror will provide an on-site parts cache that will be used by laboratory personnel to replace failed components.

5.1 [MR] Support Period The Offeror will provide hardware and software maintenance for a period of 5 years with an optional 1 year extension. The maintenance period will begin once the storage system is accepted

5.2 Hardware Support Models

5.2.1 [MR] 9x5 NBD Support Support for hardware will be 9 hours a day, 5 days a week, (0900-1800 Laboratory local time zone) with one hour response time. In the 9x5 maintenance model, Laboratory personnel will provide on-site, on-call 24x7 hardware failure response. These personnel will attempt first-level hardware fault diagnosis and repair actions. The Offeror will provide second-level hardware fault diagnosis and fault determination during the defined support window. If Laboratory personnel cannot repair failing components from the on-site parts cache, then Offeror personnel will be required to make on-site repairs. Redundant or non-critical components should carry 9x5 NBD support contracts.

5.2.2 [TO] 12x7 Support Support for hardware will be 12 hours a day, seven days a week, (0800-2000 Laboratory local time zone) with one hour response time. In the 12/7 maintenance model, Laboratory personnel will provide on-site, on-call 24x7 hardware failure response. These personnel will attempt first-level hardware fault diagnosis and repair actions. The Offeror will provide second-level hardware fault diagnosis and fault determination during the support maintenance window. If Laboratory personnel cannot repair failing components from the onsite parts cache, then Offeror personnel will be required to make on-site repairs. Redundant or non-critical components should carry 9x5 NBD support contracts.

5.3 [MR] Software Support The Offeror will provide software support and maintenance for the duration of the maintenance period. For critical software failures that degrade performance or accessibility impacting production metrics, support will be 24x7 while non-critical software failures will be 9x5 NBD. Software maintenance will include a problem reporting and tracking component as well as regular software updates.

5.4 [MR] On-site Parts Cache The Offeror will provide an on-site parts cache of FRUs, storage controllers and disk drives for the storage system. The size of the parts cache will be based on the Offeror’s MTBF estimates for each FRU and agreed to in an SLA that will be part of the maintenance and support plan. The parts cache SLA will be sufficient to sustain Argonne on-site repair actions on the storage system FRUs to maintain the storage system in fully operational status from the local parts cache without waiting for parts to arrive from the regional parts depot. The Offeror will resupply/refresh the parts cache as it is depleted for the hardware maintenance period. During the self-maintenance period, Argonne will replace failed FRU in the system from the parts cache and RMA failed FRU back to the Offeror during the self-maintenance period. The onsite spares inventory will be backed up by a regional area supply depot to ensure compliance with this Service Level Agreement (SLA). The details of the onsite spares cache will be consistent with normal wear out assumptions and do not cover rare or unforeseen defects. System components will be fully tested and burned in prior to delivery in order to minimize the number of “dead-on-arrival” components.

If the Offeror fails to deliver to the agreed-upon SLA, Argonne during the maintenance period after acceptance, and the Offeror will re-assess the size of the on-site parts cache. Argonne will then approve any on-site parts cache resizing required to re-establish successful Offeror performance to the agreed-upon SLA. Argonne is responsible for timely problem reports, parts cache usage reports, return of failed FRUs to the Offeror, as specified in the maintenance and support plan. Any “failures to deliver on the SLA” that are attributable to Argonne will not trigger a re-assessment of the size of the onsite parts cache.

During full system maintenance, after a failed FRU has been replaced, the damaged FRU will be returned to the Offeror. Once self-maintenance has been entered, Argonne will return to the Offeror any failed FRU that can be repaired. Argonne will retain the failed FRUs that cannot be repaired (as determined by the Offeror) unless a deep failure analysis of specific hardware FRU is required. FRUs that are installed in the storage system become the property of Argonne; FRU removed from the storage system and returned to the Offeror become the property of The Offeror.

5.5 [MR] Problem Escalation The Offeror will provide a detailed description of problem escalation in the event that hardware or software issues are not being addressed adequately.

6 Management Plan The acquisition of this storage system is expected to be a collaborative process between the Offeror and the ALCF, especially if the hardware or software provided is a new offering. The following technical requirements do not require documents to be submitted at the time of the RFP response but will be required if a contract is awarded. The Offeror should respond to these requirements by confirming they will comply with the requirements and understand what must be provided.

6.1 [MR] Management The Offeror will provide a single point-of-contact to work with the ALCF through the delivery and installation of the system. The point of contact is expected to available to participate in conference calls and coordinate meetings with the Offeror’s technical and operational staff.

6.2 [MR] Installation Plan If the Offeror is selected, an installation plan must be provided 90 days prior to the target date of installation. This plan will be reviewed by ALCF with the Offeror and refined to meet the objectives of the project. The plan will include:

· Installation team

· Delivery schedule

· Staging and storage needs

· Shipping plan

· Installation

· Bring-up plan

· Safety plan

6.3 [MR] Acceptance Plan Argonne and the Offeror shall jointly develop and agree upon an Acceptance Test Plan (ATP) no later than 60 days prior to delivery. Following award, the ATP shall be incorporated, as applicable, into the contract documentation through a contract modification. At a minimum, the ATP shall verify that the technical specifications set forth in this Statement of Work (SOW) have been satisfied through a combination of benchmark tests. Following delivery and installation, the system will undergo a series of performance, functionality, and availability tests prior to acceptance.

Acceptance testing will consist of multiple components designed to ensure that the system, as a whole, is high-performing, scalable, resilient, and reliable. The testing shall evaluate the system infrastructure using a combination of benchmarks, forced-failure scenarios, and stability tests. Any requirement identified in the Technical Specifications may give rise to a corresponding acceptance test.

image1.png