A SOW NFU2 Construction 2_6_2024.pdf

STATEMENT OF WORK

NERSC FACILITY UPGRADE II (NFU2) CONSTRUCTION

DATE: FEBRUARY 6, 2024

1.0 Introduction

Lawrence Berkeley National Laboratory (LBNL) is a multi-program national laboratory with a mission to perform leading multidisciplinary research in the energy sciences, general sciences, and life sciences. The LBNL is operated by the University of California (UC) for the Department of Energy (DOE). It is an independent unit of the UC system and is located adjacent to the UC Berkeley campus.

The National Energy Research Scientific Computing Center (NERSC) is the primary scientific computing facility for the Office of Science in the U.S. Department of Energy.

As one of the largest facilities in the world devoted to providing computational resources and expertise for basic scientific research, NERSC is a world leader in accelerating scientific discovery through computation. NERSC is one of four divisions within LBNL’s Computing Sciences Area, housed in the Shyh Wang Hall computational research and theory facility, also referred to as Building 59.

The NFU2 project will install new electrical and mechanical equipment at existing Building 59 to meet the power and cooling needed to support the NERSC-10 system. The NERSC-10 system will be the next installation of a high-performance supercomputer dedicated to scientific research at LBNL.

NERSC personnel will provide the final approval of potential changes to the scope, which would be communicated through the University Procurement Representative. LBNL is not required to obtain building permits from local governmental agencies. The Facilities Division and the LBNL Protective Services Department act as the Building Official and the Fire Marshal, respectively.

2.0 Objectives

The NFU2 project objective is to expand and augment the power and cooling capacities and capabilities at Shyh Wang Hall (Building 59) to support anticipated needs for future NERSC supercomputer systems and support storage, networking, and other auxiliary systems. The project, to be referred to as NFU2, will specifically include the installation of all infrastructure needed to support a new water-cooled High-Performance Computing (HPC) system, NERSC-10. (For reference, NERSC-10 will use up to 20 megawatts (MW) of peak electrical power, in addition to maintaining the infrastructure required for the operation of existing and contemporaneous systems.)

New electrical service equipment will be installed adjacent to the building at the southeast, sized to accommodate planned needs in the 2025-2030 timeframe, as well as provide expansion capability to support potential future needs for the center.

Additionally, new mechanical equipment will be placed adjacent to the building to the north to provide a new dry cooling capability that will reduce water use intensity at the center.

The NFU2 project scope is defined in the incorporated documents listed in Section 6 below, and includes:

● As an option, a 12.47 kV switch station,

● adding 10 megawatts (MW) of computing electrical capacity through two new secondary selective unit substations (5.0/6.25 MW),

● adding up to 18.6 MW of dry cooling capability that will also substantially reduce water consumption,

● repurposing and adding safety features to two existing 2.5/3.33 MW HPC unit substations,

● electrical distribution from unit substations to subpanels in the mechanical level and to existing (and one new) subpanels in the data center,

● distribution from subpanels to outlet connections on the computer room floor to connect to computer racks,

● the installation of up to six (6) air-cooled heat exchanger banks (ACHE), located on a new (steel) structure, connected in line with the existing cooling water loop, and

● pumps, piping, sensors, controls systems, and other necessary components.

The new switch station and the air-cooled heat exchanger system will be located outside of Shyh Wang Hall on adjacent, undeveloped hillsides to the southeast and north, respectively. The remainder of the new equipment is planned for installation within the building at the lowest (Mechanical) level and the north electric room.

Electrical equipment being added as part of the project will be owned by the University, not utility companies. There will be no required power connections to public utilities as part of this project, eliminating the associated risks to the project schedule.

The project documentation will include a base scope with an assortment of independent options. The independent options will provide the project team with a ready set of work to use as scope contingency to keep the project within the planned budget range without redesign cost and time.

The following table notes the existing state of the referenced item and the proposed modifications within the base scope package for the NFU2 project:

Base Scope Package

Item Existing Base Package

HPC Power Capacity

Eight (8) 2.5/3.33 Million Volt-Amps

Add two (2) 5.0/6.25 MVA substations (total of 10);

re-conductor associated distribution panels

(MVA) Substations

Air-Cooled Heat Exchanger Bays

None Add three (3) ACHE 6-Fan Array Bays

Exterior Equipment Pad

None Build North Pad for up to three (3) ACHE Fan Array Bays

Architectural N/A Screening of ACHE Fan Arrays at north building pad.

The following table notes the options that may be exercised independent of any scope package selected:

Independent Options

Option Title Additive or Deductive (to Base Scope)?

Description

1 Replace VFD of (E) Chilled Water General Pumps

(CWGP) CWGP-1,

CWGP-2

Additive Replace/upgrade variable-frequency drive (VFD) of existing CWGP-1 and CWGP-2 pumps

2 Upgrade motor for Cooling Tower (CT)

CT-1, CT-2, CT-4

Additive Replace fan motors on Cooling Tower Fans CT-1, CT-2 and CT-4 with enclosed units that have functional heaters

3 Replace VFD of (E) Tower Water General Pumps (TWGP)

TWGP-1, TWGP-2

Additive Replace/upgrade VFD of existing TWGP-1 and TWGP-2 pumps

4 Add first platform extension to accommodate six (6) total ACHE Fan Array Bays; add one

(1) ACHE fan unit for a total of four (4)

ACHE

Additive Add first platform extension to accommodate six (6) total ACHE Fan Array Bays; add one

(1) ACHE fan unit for a total of four (4) ACHE units. Architectural screening is scaled with pad extension.

5 Add second/final platform extension to accommodate nine (9) total ACHE Fan Array Bays; add two

(2) ACHE fan units for a total of six (6)

ACHE

Additive Add second/final platform extension to accommodate nine (9) total ACHE Fan Array Bays; add one (2) ACHE fan units for a total of six (6) ACHE units. Architectural screening is scaled with pad extension.

6 Add electrical switch station A8 (SW-A8)

Additive Add new electrical switch station SW-A8.

Build pad enclosure at southeast for electrical switch station SW-A8, duct bank construction, conduits/conductors, connections to 12.47 kV substations in North Electrical Room.

3.0 Task Descriptions

The NFU2 scope will be split into two design packages: The Facility Enhancement and the Site Preparation. The split of design work is required since the computer system vendor will not be selected until after the construction commences in order to meet schedule demands of the project. Facility Enhancement work includes “generic” scope that is not sensitive to the selected HPC system vendor. This includes all mechanical and electrical equipment and the work identified as independent options in spaces other than within the actual data center floor on the second level within the facility. Site Preparation scope will include the specific electrical wiring and cooling water details needed to serve a specific HPC system. This work entails piping manifold modifications or replacements in the computer room plenum space (above the structure but below the raised computer access floor). It also includes the final electrical feeder work from the computer room electrical wall panels along the west and east walls of Room 2102.

In order to support a phased HPC system deployment, specific work in both packages will be identified as Phase 1. This scope will have an early finish date ahead of substantial completion. This scope will be limited to electrical power distribution from existing, modified substations 612 and 613 to branch circuit terminations under the computer floor and necessary underfloor piping manifold modifications. Phase 2 encompasses the balance of work to support the remainder of the HPC system deployment.

3.1 Facility Enhancement Scope

The main components of the base Facility Enhancement work include the addition of 10 MW of electrical power and installation of dry cooling equipment.

Two (2) new 5.0/6.25 MVA unit substations will be located in the building’s north electric room with the low voltage distribution located in the south mechanical level. This allows for more direct routing of 480V power to the electrical feeder panels on the Computer Room directly above.

The north equipment structure will accommodate up to three ACHE 6-fan units. Optional scope expands both the structure and number of ACHE units to be installed. An architectural screen will provide a visual barrier to the equipment at the north pad. New piping connecting the ACHE units to the existing cooling water loop near the existing heat exchangers will be constructed along the west and north exterior walls of the building. The new ACHE units will work in conjunction with the existing evaporative cooling towers to provide cooling water needs for the HPC systems.

The southeast pad will be designed to site the new (additive scope) electrical switch station.

The switch will be located on a new concrete pad and vault structure to bring increased power into Building 59.

A stormwater holding tank is required at the exterior structure to adhere to state and federal regulations. An additional stormwater holding tank would be required with the optional switch station.

Facility Enhancement scope also includes:

● Demolition of existing Cori computer system branch feeders from below the floor to the existing 612 and 613 wall panels in the computer room (Room 2102).

Demolition of existing conduit, bus duct, junction boxes and feeders from existing substations 612 and 613 to wall panels along the west wall.

Finally, all work required for connectivity and distribution of the power and cooling water equipment discussed above will be included in the Facility Enhancement scope.

3.2 Site Preparation Scope

Electrical Distribution Scope:

The electrical distribution scope includes new branch feeder connections from existing (and one new) wall panels to the computer rack equipment.

Specific Scope:

1. Distribution of branch feeders from both new and existing wall panels along both walls underfloor in cable tray to final locations for HPC racks, cooling distribution units, and auxiliary racks. Connections may be hard-wired directly to the HPC systems, or, more likely, use a receptacle connection.

2. HPC racks and cooling distribution units (CDUs) will be fed directly from wall panels.

HPC racks will likely have two to four power feeds each. CDUs will likely have two power feeds which are redundant and from different panels.

3. Air-cooled auxiliary racks will likely be fed by end-row power distribution units (PDUs). The PDUs are typically fed redundantly from two different wall panels.

Mechanical Distribution Scope:

The mechanical distribution scope includes demolition and rebuild of existing underfloor 8” water pipe manifold assemblies.

4.0 Deliverables

● Subcontractor must provide labor, materials, and equipment per construction documents.

5.0 Schedule

● Expected duration for work activity is approximately 27 months.

● Notice to Proceed (NTP) to begin construction work is approximately 6 weeks from award of subcontract.

● Make-ready for HPC Early Access System (EAS) delivery is approximately 12 months following Notice to Proceed, pending establishment of system delivery dates.

● Substantial Completion for Facility Upgrade work is 24 months following Notice to Proceed.

● Warranty period of 12 months following acceptance of Substantial Completion

6.0 Reference Documents

The Subcontractor must comply with the following documents in accomplishing the work:

• LBNL Construction Details and Design Guidelines (CDDG), latest edition

• Construction Safety Manual, PUB-3000

• Drawings entitled “Building 59 NERSC Facility Upgrade II,” dated 1/29/2024

• Specifications entitled “Building 59 NERSC Facility Upgrade II Project Manual,” dated 1/29/2024

• Geotechnical Investigation Report for the NERSC Facility Upgrade II (NFU2) Project Draft Stormwater Pollution Prevention Plan dated 2/6/2024

• Environmental Specification – Air

• Environmental Specification – Soils

• Environmental Specification – Water

• Soil Management Plan for Lawrence Berkeley National Laboratory Rev. 1 12/2017 (refer to Division 01, Section 013529, ENVIRONMENT, SAFETY, AND HEALTH PROCEDURES, Part 1.9C) https://pim.lbl.gov/resource/documents/construction-details-and-design-guidelines/ https://sites.google.com/lbl.gov/eshmanual/home