SS DRAFT - Attachment 6 - Draft Exhibit A - NOAA_GFDL Computing and Data Storage Resources.pdf

DRAFT DOCUMENT

NOAA Geophysical Fluid Dynamics Laboratory (GFDL) and RDHPCS Computing and Data Storage Resources Located on the Forrestal Campus in Princeton, New Jersey, the NOAA Geophysical Fluid Dynamics Laboratory (GFDL) serves as a primary hub for advanced Earth system modeling, climate simulation, and atmospheric research. Below is an overview of the Research and Development High-Performance Computing System (RDHPCS) computing infrastructure, data storage architectures, and analysis systems located at or directly associated with the Princeton site.

1. Post-Processing and Analysis Cluster (PPAN) While off-site RDHPCS supercomputing systems handle massive numerical integrations, on-site computing at GFDL in Princeton focuses on post-processing, climate diagnostics, data analysis, and workflow automation via the Post-Processing and Analysis (PPAN / Pan) cluster.

Cluster Architecture

● Server Scale: Composed of 164 Dell compute nodes.

● Processors & Memory: Features multiple generations of Intel processors (ranging from

Sandy Bridge to Ice Lake architecture) with host memory configurations between 48 GB and 2 TB RAM per server.

● Interactive Analysis Hosts: Provides approximately 60 dedicated interactive analysis nodes equipped with high-performance filesystem interfaces to allow scientists to inspect and manipulate large datasets interactively.

Node Fast-scratch Space

● Fast-Scratch File System (/vftmp): Local fast-scratch storage mounted on each analysis host ranging from 8 TB to 44 TB in capacity per node.

2. Storage Infrastructure & Data Hierarchy GFDL's data hierarchy is tailored to manage massive Earth system model outputs, transitioning data seamlessly across online, high-performance, and archival tiers. DRAFT

Storage Tier / Directory

Capacity & Characteristics

Primary Use Case Manufacturer, Software & Type Info

User Home Directories (/home & /nbhome)

10 GB default quota per user; daily backups for /home, weekly for /nbhome (no-backup home).

Scripts, user environments, source code, and lightweight configuration files.

Unified NFS Network Storage (Dell EMC Enterprise NAS Storage / Linux

NFS).

High-Performance Work (/work)

Multi-petabyte parallel disk storage.

High bandwidth output target.

Active simulation runs, staging area for data analysis, and raw output processing.

Parallel File System

(POSIX / GPFS &

Lustre High-Performance SAN/NAS storage arrays).

Tape-Backed Archive (/archive)

Massive nearline tape storage capacity exceeding 500 Petabytes, backed by >13 PB front-end disk cache.

Long-term repository for climate experiments, historical model output, and project archives.

HPE Data Management Framework (DMF) middleware managing enterprise tape libraries and front-end disk caches.

Fast Local Scratch (/xtmp)

1.5 PB NAS

NVMe/SSD (Vast) per node.

High-speed temporary file operations during active batch data transformation.

VAST, 100%

NVMe/SSD file system mounted on all PP/AN nodes using a high-speed network connection

(100GBPS).

Data Migration & Management Tools

● Data Migration Engine (HPE DMF): Users manage datasets between disk and tape using HPE DMF data migration facility to stage archived files to fast scratch before execution.

● Group Allocation Tracking (archrpt): GFDL administrative tools monitor tape usage allocations per research group and scientist to manage long-term retention. DRAFT

3. NOAA Operational and Research & Development High-Performance Computing (HPC) Systems NOAA’s Research and Development High-Performance Computing (RDHPCS) program provides essential large computational and storage environments for weather and climate modeling research.

NOAA's High-Performance Computing (HPC) environment is a hybrid strategy that blends on-premise supercomputing with Cloud Service Providers (CSPs) to maximize efficiency, scientific output, and fiscal responsibility while simultaneously ensuring mission integrity and data security. NOAA executes its Weather and Climate Operational Supercomputing System (WCOSS) program through the National Weather Service (NWS) for operations and the Office of the Chief Information Officer (OCIO) plus Oceanic and Atmospheric Research (OAR) for Research and Development (R&D). The operational HPC is provided through a managed service contract where industry provisions the systems, data centers, and associated support as a fully integrated solution. The R&D is tied to specific partnerships with Oak Ridge National Lab (ORNL) provisioned by the Department of Energy (DOE) and the Mississippi State University (MSU) supercomputing partnership, as well as on-premises investments owned by NOAA with major system investments in Fairmont, West Virginia and Princeton, New Jersey.

● Earth System Models and Scientific Workflows: The RDHPCS compute resources are instrumental in the advancements of NOAA’s weather and seasonal to subseasonal, hurricane, and climate prediction capabilities. GFDL Flexible Modeling System (FMS) and FRE (GFDL's run environment) automate model execution at external supercomputer sites while managing data transfer and post-processing queues directly back to the Princeton PPAN system.

● NOAA N-wave Enterprise Network: The NOAA N-wave Enterprise Network connects the RDHPCS computer and data storage systems. The RDHPCS supports multiple high-performance transfer technologies, like Globus and other specialized software to ensure users are able to reliably transfer more than 150 Terabytes (TB)/day of data throughout the RDHPCS.

● Cloud HPC: Through contracts with major cloud service providers, the R&D HPC augments its on-premises investments with an integrated computing capability that spans multiple public clouds. These cloud resources allow NOAA scientists to use new and emerging technologies to further advance NOAA’s prediction capabilities, and in cooperation with the National Weather Service, migrate these capabilities in the operational prediction model.

Geographically Disperse HPC:

The R&D HPC program runs large-scale high-performance computing systems across the nation in Princeton, NJ; Fairmont, WV; Starkville, MS, and Oak Ridge, TN. These systems, and the other IT communications devices co-located at those facilities, and base system environments are procured and administered under separate contracts.

DRAFT

System Name Location System Size Scratch File System

Primary Use

Hera Fairmont, WV 3.27 PetaFLOPS

(PF)

>100PB (shared with Ursa)

Earth System Modeling

Ursa Fairmont, WV 4.25 PF >100PB (shared with Hera)

Earth System Modeling

Mercury Farimont, WV 4-node, dual-socket

AMD EPYC, 96

Cores per socket

1.5 PB HPSS access, Data Transfer and Manipulation

Gaea C5 Oak Ridge, TN 10.22 PF 75 PB Earth System Modeling

Gaea C6 Oak Ridge, TN 15.1 PF 75 PB Earth System Modeling

Gaea C7 Oak Ridge, TN 54.9 PF 75 PB Earth System Modeling

Hercules Starkville, MS 2.10 PF 9 PB (shared with Orion)

Earth System Modeling, and Collaboration

Orion Starkville, MS 3.69 PF 9 PB (shared with Hercules)

Earth System Modeling, and Collaboration

PPAN Princeton, NJ 164 individual nodes, multiple generations of Intel CPU.

5PB (across multiple file systems)

Post-Processing and Analysis