NFU2 Facility Upgrade Pre-Bid Conf Slides.pdf

NERSC-10

Facility Upgrade Pre-Bid Conference

Romy Salloum PIMD Deputy Project Manager

February 21, 2024 1:00pm

David Topete PIMD Project Manager

Pre-Bid Conference

• NERSC-10 Facility Upgrade (Facility Upgrade 2)

• Request for Proposal (RFP) No. NFU2-7623

• Wednesday, February 21, 2024 o 1:00pm - 2:00pm (PT) introductions and presentation o 2:00pm - 3:00pm (PT) walkthrough

Explanation or Interpretation of RFP Documents

• This Pre-Bid Conference is for informational purposes only

• Changes resulting from the Pre-Proposal Conference and/or site visit are official only if issued through an amendment to the Request for Proposal (RFP)

• Any inconsistencies between this presentation and the requirements in the RFP materials, the RFP materials shall govern

• Reminder: Per RFP Section 2.1 on page 4, "All correspondence in conjunction with this solicitation must be directed to the University Procurement Representative"

Key Project Staff - LBNL

• PIMD Project Manager: David Topete

• PIMD Deputy Project Manager: Romy Salloum

• University Procurement Rep.: Emily Mastronardi

• EHS Safety Team Lead: Wes Chase

• NERSC Client Representative: Ben Maxwell

• NERSC-10 Proj. Director: Hai Ah Nam

Key Project Staff - Consultants & Support

• A/E Designer of Record: Enercon Services o Mechanical, Electrical, Structural, Fire Protection

• Consultants to Enercon Services o Architectural Gensler o Structural IDA Structural Engineers o Civil JMA Civil

• Geotechnical Engineering: A3GEO

• Construction Manager: TKK

A Brief Introduction to NERSC What is NERSC and NERSC-10?

Dr. Hai Ah Nam NERSC-10 Project Director

NERSC: Mission HPC for DOE Office of Science Research

Biological and Environmental Research Computing

High Energy Physics

Largest funder of physical science research in the U.S.

Nuclear Physics

Basic Energy Sciences

Fusion Energy, Plasma Physics

NERSC Systems Roadmap

NERSC-7: Edison Multicore CPU

NERSC-8: Cori Manycore CPU NESAP Launched:

transition applications to advanced architectures

NERSC-9: Perlmutter CPU and GPU nodes NESAP Expanded Simulation, Learning & Data: Continued transition of applications and support for complex workflows

NERSC-10:

Exa system NESAP Workflows:

Accelerating end-to-end workflows with technology integration

2030+

NERSC-11:

Beyond Moore

Nobel-Prize Winning Users

Saul Perlmutter

Warren Washington

2013 Chemistry 2011 Physics

2007 Peace 2006 Physics for the discovery of the accelerating expansion of the Universe through observations of distant supernovae for the development of multiscale models for complex chemical systems for the discovery of the blackbody form and anisotropy of the cosmic microwave background radiation for their efforts to build up and disseminate greater knowledge about man-made climate change

Martin Karplus

George Smoot

2017 Chemistryfor developing cryo-electron microscopy for the high-resolution structure determination of biomolecules in solution

Joachim Frank SNO Collaboration

2015 Physics for the discovery of neutrino oscillations, which shows that neutrinos have mass

Impacting Science for 50 years

NERSC has been acknowledged in 8,790 refereed scientific publications & high profile journals since 2020

● Nature [47]

● Nature Family of Journals [463]

● Proc. of the National Academy of Sciences [222]

● Science [36]

● Monthly Notices of the Royal Astron. Society [397]

● Physical Review* [4,170]

● Astrophysical Journal [792]

● Physics of Plasmas [685]

NERSC-10 Designed to support complex high performance workflows

The NERSC-10 system accelerates end-to-end DOE SC workflows and enables new modes of scientific discovery through the integration of experiment, data analysis, and simulation.

● Deliver 10x the performance of Perlmutter

● Support for diverse & broad DOE SC workflows

● Provide high speed access to DOE SC community data

● Enable user access in 2027 to maximize productivity

Facility Upgrade Scope of Work

• The Statement of Work for the Facility Upgrade consists of the following:

o Facility Enhancement Scope (currently out for bid) o Site Preparation Scope Design (smaller scope, final runs for HPC system connections)

• Likely to be completed after PO award

• Will issue a change order to selected subcontractor to perform work once design is complete

Facility Enhancement Scope

• Base Bid Scope:

o Add 10 MVA of HPC compute power via (2) 5.0 MVA substations o Refeed select electrical distribution panels from two existing substations o Add 3 air-cooled heat exchanger (ACHE) units into existing closed cooling water loop o New exterior structure to support ACHE units located north of

Shyh Wang Hall

Facility Enhancement Scope (cont.)

• Independent Options:

o Option 1/A: Replace VFDs for CWGP-1, CWGP-2 o Option 2/B: Replace fan motors on CT-1, CT-2, CT-4 o Option 3/C: Replace VFDs for TWGP-1, TWGP-2 o Option 4/D: Increase north slope structure to accommodate 6

ACHE units, install 4 ACHE units o Option 5/E: Increase north slope structure to accommodate 9

ACHE units, install 6 ACHE units o Option 6/F: Install new switch station SW-A8, incl. crosstie between (E) SW-A7 to (N) SW-A8

Scope - Exterior Work

Scope - Interior Work (Mechanical Level)

Scope - Interior Work (Electrical Rooms)

Site Preparation Scope

• Will be a stand-alone design drawing package to be performed by the same construction general subcontractor who performs the Facility Enhancement package

• Construction documents prepared by Enercon team (same designers)

• Start of design of this scope is to be deferred until NERSC selects a computer system vendor in mid-2024. Expecting completion of design by end of calendar year 2024.

Site Preparation Scope (cont.)

• The scope of work will include:

o power and cooling final runs from the wall panels, and o underfloor water piping manifolds to the actual computer racks and cooling distribution units.

• NERSC anticipates portions of the Site Preparation will need to be completed ahead of other portions to accommodate test system deliveries ahead of main system

LBNL Partial Site and Shyh Wang Hall/B59

Lot D (The Pit) Shyh Wang Hall / B59

BioEPIC SSM

Blackberry Gate

N

ALS-U

Potential Site Activities

• BioEPIC o Construction is to be completed late-2024

• SSM/THUP

o Under construction, likely in parallel with NFU2

• ALS-U

o Regular 3-month winter and summer shutdowns leading up to “Dark Time” (June 2026 through June 2027)

Bid Schedule

MILESTONE

Release of RFP

Pre-Bid Conference / Site Walkdown

Offerors Questions Due

Proposals Due

Evaluation Team Selection

Notice of Award (anticipated)*

Substantial Completion (anticipated)*

* Based on DOE review of Procurement documents

DATE

Feb. 6, 2024

Feb. 21, 2024

Feb. 28, 2024

Mar. 20, 2024

Apr. 23, 2024

Sep. 30, 2024

Dec. 22, 2026

Working at LBNL

• Per Div 01, General Requirements:

o Noise is limited per City of Berkeley Noise Ordinance o Typical working hours: 7:30am to 6:00pm, M-F o Winter Shutdown: Christmas Eve through New Year’s Day

• Shyh Wang Hall/B59 will remain operational during Facility Upgrade work, except for any planned outage(s).

Minimizing service interruption is valued.

• Above all, work safely

Site Logistics

• Parking is limited (as experienced today)

• Carpooling is encouraged

• Shuttle service is encouraged o Shuttle buses operate M-F around typical start and end times, except on Lab holidays

• Parking off-site o Rim Way Garage may offer discounted parking rates

GC Setup

• GC to plan for a jobsite trailer

• Potential for some use of B59 office cubicles

• Some use of meeting rooms can be arranged o (LBNL-led meetings will be in B59 meeting rooms)

• Space may be designated (in Mechanical Level) for Site

Superintendent, Site Safety Manager

• Space may be designated for Tier 2 subcontractors

Laydown and Staging

• Limited to undeveloped areas north of Shyh Wang Hall and loading dock

• No further exterior areas will be available

• Interior space in Mechanical Level of B59 will be available

• Deliveries should be scheduled to be “just in time”

• Coordinate use of loading dock/drive aisle with NERSC o Trash and recycling pick-ups weekly on Tuesday morning

Limits of Construction

Laydown and Staging

EH&S / Construction Safety

• The LBNL safety program is demanding, but we believe that safety is good business for everyone.

• The Laboratory and our client, DOE, have identified construction safety as a focus area for excellence and have developed programs to mitigate hazards.

• We are committed to Integrated Safety Management (ISM) each day and we are seeking partners with the same commitment.

https://ism.lbl.gov/ https://ism.lbl.gov/

EH&S / Construction Safety (cont.)

EH&S Requirements

• Title 10, Part 851 of the US Code of Federal Regulations:

https://www.energy.gov/gc/10-cfr-851-worker-safety-and-health-program

• LBNL Master Specifications, Section 013529: Environmental, Safety and Health Procedures o As a result of the variance from the Federal OSHA standards, Federal OSHA standards have been replaced with the Cal/OSHA equivalent EXCEPT for electrical safety standards, which must meet NFPA 70E 2018 at a minimum.

• All projects must adhere to LBNL ESH Manual (PUB3000) o Specifically Chapter 10, Appendix A: LBNL Construction Safety Manual o Each prime subcontractor must submit a Site Specific Safety Plan (SSSP) to EHS before “notice to proceed” is issued.

• Construction Subcontractor Resource Page https://www.energy.gov/gc/10-cfr-851-worker-safety-and-health-program https://sites.google.com/lbl.gov/eshmanual/home https://drive.google.com/file/d/1ALWFOaZwG0rRt3wmy0EzrT-ToXduRtjw/view https://ehs.lbl.gov/resource/documents/occupational-safety/construction-subcontractor-resource-page/

EH&S / Construction Safety (cont.)

Subcontractor Onsite Health and Safety Representative

• Div. 01, Section 013529, “Environment, Health and Safety Procedures” o Option A - Subcontractor Full-time Onsite Presence (Required for NERSC 10) o 10 years of construction health and safety experience o CSP or CIH

Site Access and Security

• All vehicles subject to inspection

• All employees shall be U.S. Citizens and have valid ID subject to Real ID Act o Valid Driver’s License o U.S. Passport o Federally issued Identification o Construction Access Badge is required to enter the site and be at the jobsite o Badge will be issued once GERT and Construction Safety Orientation are complete o Construction Access Badges must be worn at all times

• Badges are arranged by the LBNL Project or Construction Manager

• Deliveries must be coordinated with LBNL

Site Access and Security (cont.)

• No Weapons, Firearms or Ammunition (regardless of Concealed & Carry Permit)

• No Explosives or any other apparatus or material hazardous to the public or property

• No possession or illegal use of controlled substances or intoxicants or being under their influence

• Follow California Rules of the Road On-Site

• Use of Mobile communication devices, including hand-free equipment, while driving is prohibited

• Helmets required if riding motorcycle, bicycle, or using any wheeled sporting equipment

Questions?

Site Preparation Scope (cont.)

• Work associated with repurposing substations 612 and 613 and its wall panels on the east data center wall, and work on substations 689 and 690 and its wall panels along the east wall would be associated with a Phase 1 HPC Delivery.

• Work associated with repurposing wall panels along the west wall of the data center for the new HPC substations should be associated with Phase 2 HPC Delivery work

• Each phase of work will have a different target completion date.