NACS_Pre-Proposal_Conference_Presentation_Final_022717.pdf

National Aeronautics and Space Administration www.nasa.gov

NASA Advanced Computing Services (NACS)

NNA17554082R

February 27, 2017

Pre-Proposal Conference NASA Ames Research Center

Agenda

• 8:30-9:00 Sign-In

• 9:00-9:15 Announcements & Introductions

• 9:15-9:45 Procurement Overview Veronica L. Gutierrez, Contracting Officer

• 9:45-11:00 Statement of Work/Technical Overview

William Thigpen, Branch Chief, Advanced Computing Daniel Q. Duffy, High Performance Computing Lead

• 11:00-11:30 Open Questions & Answers

• 11:30-12:30 Lunch

• 12:30-2:30 Tours

Safety and Concierge

• Emergency exits

• Bathrooms

• Campus Maps are available at the sign-in table

• Lunch Options

• Mega Bites (Building 235)

• Space Bar (Building 3)

Introductions

• Procurement

• Veronica L. Gutierrez, ARC Contracting Officer

• Ayana A. Briscoe, GSFC Contracting Officer

• Technical

• William W. Thigpen, ARC Advanced Computing Branch Chief

• Daniel Q. Duffy, GSFC High Performance Computing Lead

This Pre-Proposal Conference is intended to:

• Provide the current status of the NASA Advanced Computing Services (NACS) acquisition.

• Improve potential offerors’ understanding of the Government’s requirements.

• Enhance the Government’s ability to obtain quality services.

• Increase efficiency in proposal preparation, evaluation, negotiation, and award.

General Guidance

• These slides shall not be interpreted as a comprehensive description of all requirements of the Solicitation. Please refer to the Draft Statement of Work and Draft Request for Proposal (RFP).

• To the extent there are any inconsistencies between this briefing and the solicitation, the solicitation governs.

• Nothing said here today should be construed as a revision unless subsequently confirmed in the Final RFP.

• Communications “blackout” will be invoked following issuance of Final RFP and will continue until contract award.

• All communications with industry concerning this acquisition will then be with the Contracting Officer only.

General Guidance

• Index cards are available at the sign‐in table.

• Questions will be addressed as time permits.

• All questions related to the Draft RFP, Facility Tours or Pre-Proposal

Conference shall be submitted in writing no later than 4:00PM PST on Friday, March 3, 2017 to the ARC Contracting Officer at Veronica.L.Gutierrez@nasa.gov.

• Presentation and all questions and answers will be made available on the Federal Business Opportunities (FBO) website.

• Only answers posted are official.

Procurement Overview

Freedom of Information Act (FOIA) FOIA Requests may be submitted electronically to:

Contract NNA07CA29C/ARC hq-foia@nasa.gov

Contract NNG13HQ01C/GSFC gsfc-foia@mail.nasa.gov

No proprietary information can be disclosed.

NACS Requirements Overview

• The NACS procurement is a consolidation of the following current contracts: NASA ARC, NASA

Supercomputing Support Services (NS3), NNA07CA29C and NASA GSFC, Computational and Information Science and Technology Office and Technical Services (CISTO-SCTS), NNG13HQ01C.

• A seven-month extension to the current NS3 contract is anticipated.

• NACS will provide world-class High Performance Computing (HPC) to all NASA missions in support of mission-driven science and engineering. In addition to supporting the NAS at ARC and the NCCS at GSFC, the NACS may be used to provide advanced computing services to other NASA Centers and Jet Propulsion Laboratory (JPL).

• North American Industry Classification System (NAICS) Code 541513 for Computer Facilities Management Services, with a size standard of $27.5 Million.

Acquisition Description

• Single Award Hybrid Cost Plus Award Fee (CPAF) and Cost Plus Fixed Fee (CPFF) contract consisting of a core contract management services and core technical requirements and Indefinite Delivery/Indefinite Quantity (IDIQ) requirements for additional technical requirements.

• The maximum contract value of the proposed contract is $1.1 Billion. The IDIQ contract line items will have a minimum ordering value of $100,000 and a maximum ordering value of $915 Million.

• Ten-year performance period consisting of a one-year Base Period, and nine one-year Option Periods. A 30-day Phase-in Period will be included in the Base.

• Work is to be performed at NASA Ames Research Center and Goddard Space Flight Center.

General Procurement Information

• This procurement will be conducted through a Full and Open Competition pursuant to FAR and NFS Part 15 - Contracting by Negotiations.

• FAR 52.215-1 and NFS 1815.209 allow for an award to be made without discussions. The Government may award a contract based solely on the initial proposals received. The Government reserves the right to hold discussions if award on the basis of initial offers is determined not to be in the best interest of the Government. If discussions are necessary, then a competitive range will be set and negotiations will commence. The initial proposals to the Government should contain the most favorable terms from a price and technical standpoint.

Proposal Preparation

• In accordance with Section L of the Final RFP and written amendments, if any. Acknowledge all amendments.

• Format

• Hardcopy/Electronic

• Oral/Written

• See Draft RFP L.5, L.6, L.7 and L.8

Proposal Content

• Cover Letter

• Volume I - Mission Suitability

• A. Management Approach

• B. Technical Approach

• C. Small Business Utilization

• Small Business Subcontracting

• Commitment to the Small Business Program

• Volume II – Past Performance

• Questionnaires & Reference List (10 days prior to proposal due date)

• Information Provided by Offerors and Major Subcontractors

• Volume III – Cost

Source Evaluation Process

• In accordance with Section M of the Final RFP and written amendments, if any.

• Three evaluation factors for the NACS Procurement:

• Mission Suitability

• Past Performance

• Cost

• Of the evaluation factors identified above, Mission Suitability is the most important and Cost is more important than Past Performance. Mission Suitability and Past Performance, when combined, are significantly more important than Cost.

Source Evaluation Process (continued) Mission Suitability Factor The following sub-factors and points have been determined for the Mission Suitability Factor.

Mission Suitability Sub-factors Points

Management Approach 500

Technical Approach 400

Small Business Utilization 100

Total 1000

The Mission Suitability Factor is evaluated at the subfactor level and is the only factor scored.

The Past Performance Factor is evaluated and given a Confidence Rating. The Cost/Price Factor is evaluated, but not scored.

Source Evaluation Process (continued)

• Use oral presentations for portions of Mission Suitability and Past

Performance.

• Instructions for Oral Presentation – See L.7

• Within fifteen (15) business days after the closing date of the RFP, each Offeror will be given the date, time and place for its oral presentation.

• The oral presentation will be held at Ames Research Center, Moffett Field, California.

Small Business Utilization

• The Government assessed the appropriate subcontracting goals for this acquisition. The small business goals for this procurement, expressed as a percent of total contract value are as follows:

Total Small Business (SB): 12%

Small Disadvantaged Business (SDB) Concerns 5.0%

Women Owned Small Business (WOSB) Concerns 4.0%

Historically Underutilized Business Zone (HUBZone) Zone 1.0%

Veteran Owned Small Business (VOSB) Concerns 2.0%

Service-Disabled Veteran-Owned Small Business (SDVOSB) Concerns 1.0%

• Large Businesses must submit Small Business Subcontracting Plan. Small Businesses are not required to submit Small

Business Subcontracting Plan; however, Small Businesses are required to indicate the amount of effort proposed to be done by a Small Business either at the prime level or at the first tier subcontract level. All Offerors are required to respond to the Commitment to the Small Business Program.

Acquisition Schedule (Tentative)

Synopsis Issued November 8, 2016 Virtual Pre-Solicitation (ARC) November 28, 2016 Issuance of Draft RFP February 10, 2017 Pre-Proposal/Facility Tour (ARC) February 27, 2017 Draft Questions/Comments March 3, 2017 Facility Tour (GSFC) March 6, 2017 Issuance of RFP (45 days) March 2017 Receipt of Proposals April 2017 Contract Award (Phase-In) January 2018

Statement of Work (SOW)

Overview

NACS Technical Areas

1. Program Management

2. Technical Integration

3. High Performance Computing Environment

4. Networking and Communications

5. User Services

6. Facility Operations

7. Information Technology Security

8. Application Services and Tools

9. Visualization and Data Analysis

10. Modeling and Simulation

11. Data Publication and Distribution

12. Big Data Analysis and Analytics Support

13. Enhanced Collaborations and Visiting Scholars Program

14. Advanced Computing Research, Development and Enhancement

15. Other Advanced Computing System Support

Summary of Technical Areas (1)

1. Program Management: managing the contractor’s support for all administrative and technical activities, providing an interface between contractor management and the Government, and drawing on corporate knowledge and capabilities to support development of innovative solutions for the Government.

2. Technical Integration: conducts or manages any high-level and crosscutting technical activities such as the development and implementation of advanced computing best practices that are designed and optimized for NASA and the individual center environments.

3. High Performance Computing Environment: Provide acquisition support and state-of-the-art management of robust HPC and data environments to support NASA mission requirements for high-fidelity computational modeling, simulation, and data analysis.

Summary of Technical Areas (2)

4. Networking and Communications: network development, engineering, integration, testing, deployment, operations, and performance studies in support of high-bandwidth and specialized HPC and other advanced computing requirements for the NACS facilities.

5. User Services: maintain highly effective and efficient information exchange between users, NACS facilities, and Mission Directorate sponsors about all aspects of the systems and services through a variety of outlets.

• NAS: Includes two 24x7 functions: 1) control room operation, to provide operations and monitoring of the NAS supercomputers, their support systems, and the facility; 2) to provide the initial and primary contact with users of the NAS facility.

• NCCS: Requirement for help desk to be the initial and primary contact with the users of the NCCS facility and to be staffed from 8:00 a.m. to 6:00 p.m. Eastern Time (U.S.)

Monday through Friday.

Summary of Technical Areas (3)

6. Facility Operations: support for physical plant operations and maintenance (O&M), property control, computer system administration for facility staff, website maintenance, and facility multimedia and communications.

7. Information Technology Security: provides comprehensive IT security management, operation, monitoring and safeguards to protect All NACS information systems and IT assets.

8. Application Services and Tools: focuses on enhancing the performance and productivity of NASA’s advanced modeling, simulation, and data analysis applications.

9. Visualization and Data Analysis: develops and employs advanced visualization and data analysis in both interactive and batch environments to enable users to derive increased value from their computations.

Summary of Technical Areas (4)

10. Modeling and Simulation: conducts advanced modeling and simulation analyses requested by NASA programs and projects.

11. Data Publication and Distribution: develops and maintains data publication and distribution services to enable the NACS user community to quickly be able to publish and distribute key science and engineering research products.

12. Big Data Analysis and Analytics Support: advance NACS’ capabilities to support big data analysis and analytics development of methods and tools, continue and enhance collaboration with other NASA organizations involved with big data initiatives, and seek opportunities for outreach to discover new users and create awareness of NACS’ capabilities to support big data analysis and analytics on NASA data.

Enhancements, Efficiencies and Related Innovations

• The Offeror is encouraged to propose enhancements and/or efficiencies that provide new or innovative methods, techniques, or technologies across the entire SOW. For each proposed enhancement and/or efficiency, Offeror shall:

• Describe each proposed method, technique, or technology, explain and quantify (where possible) how it impacts the performance of the SOW, and the benefit to the Government.

• Address any assumptions needed to implement any proposed enhancement, efficiency, or innovation and provide rationale for these assumptions.

• Provide a risk assessment for any proposed enhancement, efficiency, or innovation.

Case Studies

• The Case Studies below are provided to enable the Government to evaluate the Offeror’s overall understanding of the technical and management challenges of operating world-class HPC facilities. The Offeror’s detailed response for each case study shall address the following:

• Provide a narrative that fully describes the Offeror’s understanding and impact to the performance of the NACS contract.

• Provide technical details and representative schedules (if applicable).

• Address unique and critical aspects that might affect performance on the NACS contract.

• Describe the rationale for any assumptions, dependencies, risks and mitigations made by the Offeror.

• Describe any innovations that may apply to the case studies.

Case Study A: Disruptive Technologies

• Disruptive technologies have the possibility of significantly altering the HPC landscape across NACS centers over the course of the contract. The Offeror shall provide its understanding of disruptive technologies and provide a roadmap of potential disruptive technologies that could significantly impact NACS’s ability to provide systems and services to meet NASA’s science and engineering requirements. The Offeror shall describe its rationale for why a technology is considered disruptive, the time frame, impact, and risks on the NACS environment, and recommended activities that should be accomplished over the course of the NACS contract period to better understand, evaluate, or implement relevant disruptive technologies.

Case Study B: Exascale Applications

• As NASA moves toward Exascale computing, applications must also change to take advantage of this massive increase in capability. The Offeror shall describe its understanding of the requirements for applications to take advantage of Exascale computing. In addition, the Offeror shall describe its approach to evaluate an application to run at Exascale to address the question of whether an application can be evolved, as it is currently implemented, or if a refactoring approach is needed to reach the required performance.

Sample Tasks

• The Offeror shall provide a detailed response to each sample task. In addition, the Offeror shall address the following for each sample task:

• Provide a narrative that fully describes the Offeror’s understanding and impact to the performance of the NACS contract.

• Discuss the technical, management, and communication approaches. Address unique and critical aspects that might affect performance on the NACS contract. Provide a detailed schedule that addresses dates for major and minor milestones and deliverables.

• Provide estimated labor categories and labor hours, including the designation and justification of each position to be performed by the prime contractor or subcontractor (only applies to Sample Task A).

• Provide rationale for any assumptions, dependencies, and risks, and risk mitigations.

• Describe any innovations that may apply to the sample tasks.

Sample Task A: Performance Analysis

• The Government believes maintaining the initial two sites (NAS and the NCCS) is in the best interest to

NASA in order to provide some level of redundancy for HPC and to tailor services to meet requirements specific to the mission and users at each site. The government recognizes that improvements can be made to enhance the level of services provided by each site.

• Provide an approach to analyze performance across all NACS sites.

• Provide an adequate justification to show the potential success of the approach, address any assumptions, challenges, risks, and risk mitigation plan.

• Identify areas of the NACS Statement of Work (SOW) whose performance could benefit from proposed enhancements and provide rationale for identifying those areas.

• Recommend specific enhancements to the identified areas of the SOW.

• Provide justification for the proposed enhancements with quantifiable impacts on performance.

• Provide a detailed implementation schedule with milestones, tasks, and deliverables for the recommended enhancements and the methodology for measuring the impact of the proposed enhancements.

• Must provide an estimated level of effort with labor categories and hours

Sample Task B: NACS Operations

• To operate and maintain the operational environments for both the HECC at Ames Research

Center (ARC) and NCCS at Goddard Space Flight Center (GSFC), the contractor shall operate as the procurement agent for both new functionality and operational costs. This task supports both the NAS and the NCCS in procuring essential hardware, software, maintenance, and support services to enable scientific discover and engineering excellence throughout the Agency.

• Mistake in the title – will be changed in the final to “NACS Procurements”

• Will need to provide cost for this sample task, and the budget is provided in the sample task order

• During the contract year, EACH site will typically have the following procurement requirements

» Total of 60 to 70 overall procurements » Small purchases are $10s to $100s » Large purchases can exceed $5M (between 1 and 6 purchases) » No labor is expected for this task

NASA Advanced Computing Services (NACS)

Site Overviews

High-End Computing Capability (HECC) Project NASA Advanced Supercomputing (NAS)

NASA Ames Dr. Piyush Mehrotra

William Thigpen https://www.nas.nasa.gov/hecc/ https://www.nas.nasa.gov/

NASA Center for Climate Simulation (NCCS) Goddard Space Flight Center (GSFC)

Dr. Phil Webster Dr. Daniel Duffy http://www.nccs.nasa.gov/

HEC Program Office NASA Headquarters

Dr. Tsengdar Lee Scientific Computing Portfolio Manager http://www.hec.nasa.gov/

NASA High-End Computing Program

NASA Ames Research Center NASA Advanced Supercomputing

An Overview

NAS Division Organization Division Chief /

Deputy Division Chief Piyush Mehrotra /

John Parks (Acting)

Computational Aerosciences Branch

Chief Cetin Kiris

Advanced Computing Branch Chief

William Thigpen

Computational Physics Branch Chief

Seokkwan Yoon (Acting)

Assistant Division Chief of Operations Ana Grady-Hiser

NAS Project Portfolio

• Engineering Risk Assessment (ERA) - NAS experts in risk modeling and simulation support the ERA project, which provides advanced, physics-based risk analyses to support the development of safe, reliable space launch vehicles and exploration systems.

• Heliophysics Modeling & Simulation (HMS) - The Heliophysics Modeling and Simulation (HMS) team develops high-fidelity modeling and simulations tools for research on, and modeling of, the solar interior and atmosphere.

• NASA Earth Exchange - NAS is a partner in the NASA Earth Exchange project, a collaborative environment that utilizes the Pleiades supercomputer along with a NAS-operated, custom collaboration environment for Earth scientists to gain and share new insights into how the Earth's ecosystems interact and respond to climate variability and change. Current projects include an important study to understand the carbon balance of the U.S. and the North American continent.

• Quantum Artificial Intelligence Lab (QuAIL) - The NAS facility hosts the Quantum Artificial Intelligence Lab, a collaboration among NASA, University Science Research Associates, and Google to explore the potential for quantum computers to tackle computational challenges that are too difficult or impossible for traditional supercomputers to handle.

• Big Data at NAS – NASA’s data from in place assets and simulation results provide challenges that were not seen in the past. The NAS division is working at providing viable solutions to these problems.

• Computational Aerosciences - To develop, enhance, and apply predictive capabilities in computational aerosciences, and perform large-scale simulations that advance the goals of the NASA Human Exploration and Operations and Aeronautics Research and Directorates

• Computational Physics - To develop multi-disciplinary physical models, numerical methods and algorithms, and technologies in support of NASA Science and Space Technology Mission Directorates.

• High-End Computing Capability (HECC) Project – HECC provides the high-end computing resources and expert services to NASA scientists and engineers across all of the Agency's mission organizations.

Engineering Risk Assessment (ERA)

• Provides advanced risk analyses to support the development of complex space launch and exploration systems that are typically difficult to represent effectively using traditional risk assessment methods.

• Quantifies system risks through a combination of probabilistic analyses, physics-based simulations of key risk factors, and failure timing and propagation models

• Develops dynamic, integrated risk models to not only quantify the probabilities of individual failures, but also to learn about the specific systems, identify the driving risk factors, and guide designers toward the most effective strategies for reducing risk.

Simulation of blast wave passing over the Orion crew module during an abort from a vehicle explosion.

ERA Work and Benefits

• First developed and applied through NASA's

Simulation Assisted Risk Assessment (SARA) Project, which generated detailed crew-risk probability estimates for the Ares I Crew Launch Vehicle.

• Performed risk assessments for other potential launch vehicle designs, exploration systems, and missions.

• Contributes to NASA's broader safety policies and procedures by supporting key safety studies, helping to develop effective safety/risk guidelines for future space programs.

• ERA has exceptional potential to benefit the design and evaluation of a wide range of systems and technologies. Its dynamic, multi-fidelity approach provides more realistic representation of failures and their propagation, identifies sensitivities and dependencies within highly complex systems, and most importantly, facilitates risk-informed design throughout every stage of the development process.

Simulation of debris fragmentation resulting from a vehicle explosion to determine probability of debris striking the aborting crew module.

Heliophysics Modeling & Simulation Project

• Develops high-fidelity modeling and simulations tools for research on, and modeling of, the solar interior and atmosphere. The project supports NASA's Living With a Star (LWS) program, which has a goal to provide a predictive understanding of the Sun's system, specifically of the space weather conditions near Earth and in the interplanetary medium.

• Addresses the LWS program's primary goal to develop first-principles-based models for the coupled Sun- Earth system by providing tools to U.S. space weather forecasting organizations, such as the National Oceanic and Atmospheric Administration.

• Take a unique, multidisciplinary approach to deliver real-time models to the scientific and operational communities for forecasting both magnetic fields and velocity fields on the surface of the Sun.

Coronal mass ejection observed by NASA's Solar Dynamics Observatory (SDO), in extreme ultraviolet radiation emitted by ionized helium atoms heated to 80,000 Kelvin. The eruption is caused by a magnetic field that was generated by a dynamo process beneath the visible surface of the Sun. (NASA/SDO)

NASA Earth Exchange (NEX) A collaborative environment that brings scientists and researchers together in a knowledge-based social network along with tools, computing power and data to accelerate research, innovation and provide transparency.

VISION

To provide “science as a service” to the Earth Science community addressing global environmental challenges.

GOAL

To improve efficiency and expand the scope of NASA Earth science technology, research and applications programs.

NEX

Project Manager: Piyush Mehrotra Principal Scientist: Ramakrishna Nemani

NEX Overview

NEX Infrastructure

NEX Software Platform

NEX Science and Applications

Data repository – 1.3PB+ Compute Infrastructure (HPC, GPUs, Shared Memory, Quantum)

Collaboration Portal and Knowledge Base

Science Data Management

Workflow and Process

Management

Outreach and Engagement

Knowledge Management

Cloud Infrastructure (OpenNEX)

Global Science Products Carbon Monitoring Water Management Climate Landcover (1m)

Quantum Computing: D-Wave Two™ System

• Collaboration between NASA / Google /

USRA

• D-Wave 2 Installed at NAS

• Washington processor – 1,097 qubits

(quantum bits – niobium superconducting loops encoding 2 magnetic states)

• Physical characteristic » 10 kg of metal in vacuum at 15 mK » Magnetic shielding to 1 nanoTesla

(50,000x less than Earth’s magnetic field) » Uses 12 kW electrical power

• Focused on solving discrete optimization problems via quantum annealing

Big Data Challenges for NASA Users

• Data Discovery – finding what data is available and where

• Indexing, federated metadata service and semantic reasoning

• Increased WAN bandwidth

• Fault tolerant and resilient hardware/software infrastructure

– Tools/models/algorithms - developing analytics/analysis software at scale

• Mechanisms for sharing software to reduce duplication

• Software for workflow description and management to tie all components together and facilitate re-use

• Analysis/Analy:cs infrastructure – inadequacy of available resources

• I/O infrastructure

• Large memory spaces for in-core analysis

• Support for the heterogeneous resources in an integrated environment:

distributed memory & shared memory systems, hadoop cluster, accelerators, FPGAs etc.

• Data Dissemina:on– difficult to share knowledge across a wider community

• Support for dissemination and sharing of code, data products, results, etc…..

Based on a HECC survey: NAS Technical Report: NAS-2014-02.pdf

Big Data Related Projects @ NAS Applica'ons

Analy'cs

Hardware Infrastructure

! Xeon PHI, GPU, FPGA, Hadoop

Computational Aerosciences

• HEOMD (SLS, MPCV & Commercial Crew): To develop and apply aeroscience modeling capabilities for space vehicle design and analysis, including databases and induced environments for ascent and abort, staging, debris, plume impingement, retro-propulsion, and launch environment. Develop and apply physics based Engineering Risk Assessment (ERA) models for space vehicles.

• ARMD (Fundamental Aeronautics): To advance fundamental aeroscience theory, models, methodologies, codes, and understanding; develop and apply aeroscience modeling capabilities for air vehicle design and performance, efficiency, emissions, noise, and safety analysis.

• The computational aerosciences teams support many NASA and commercial aeronautics and aerospace missions and projects, including:

– Space Launch System, Aquila-II, SWORDS, and BioSentinel

– Commercial Crew Program, Sierra Nevada Corporation, and

SpaceX

– Planetary Defense Project

– Office of Safety and Mission Assurance

– Fundamental Aeronautics Program (AATT, CST, Rotary Wing, ERA)

Gulfstream Landing Gear Noise Prediction

Ascent Aero-database for Space Launch System

Prediction Noise Generation from High-Lift Devices

Shock-Plume Interaction Wind Tunnel Support for CST

Computational Physics

• Solar Stellar Modeling & Analysis: To understand the variability of the

Sun and its influence on space weather and the solar planetary system, and to understand the variability of stars and their effects on the habitability of accompanying planets.

- Customers Heliophysics and Astrophysics (SMD)

• Planetary Atmosphere Simulation: To drive advances in thermophysical chemistry of planetary atmospheres and Earth’s ionosphere, relevant to entry systems and space-weather effects.

- Customers: Game Changing Program - Entry Systems Modeling (STMD)

- Material Response to High-Enthalpy Environments: To develop understanding of material response to high-enthalpy environments at the micro-scale, and develop physics-base material response models for designing thermal protection systems.

- Customers: Game Changing Program - Entry Systems Modeling (STMD)

• Planetary Defense: To develop advanced risk models and simulations to assess and characterize the potential threat due to impacts from various classes of asteroids and other near-Earth objects

- Customers: ARC Planetary Defense Project for the NASA Near-Earth

Object Officeon the surface of the Sun.

Satellite optical communication and CubeSat instruments

Preliminary Cart3D Simulations of Chelyabinsk Meteor

Chemistry products for ionosphere and planetary entry simulations

High End Computing Capability

Project

NASA’s HEC Requirements: Capacity HEOMD (engineering-related work) require HEC resources that can handle large numbers of relatively-low CPU-count jobs with quick turnaround times.

Over 1500 simulations utilized ~ 2 million processor hours to study launch abort systems on the next generation crew transport vehicle

Over 4 million hours were used over a 4 month project to evaluate future designed of the next generation launch complex at the Kennedy Space Center

The formation of vortex filaments and their roll-up into a single, prominent vortex at each tip on a Gulfstream aircraft

NASA’s HEC Requirements: Capability ARMD and SMD (aeronautics and science related work) require HEC resources that can handle high fidelity relatively-large CPU-count jobs with minimal time-to-solution. Capability enables work that wasn’t possible on previous architectures.

NASA is looking at the oceans, running 100’s of jobs on Pleiades using up to 10,000 processors. Looking at the role of the oceans in the global carbon cycle is enabled by access to large processing and storage assets

For the first time, the Figure-of-Merit has been predicted within experimental error for the V22 Osprey and Black Hawk helicopter rotors in hover, over a wide range of flow conditions

To complete the Bolshoi simulation, which traces how the largest galaxies and galaxy structures in the universe were formed billions of years ago, astrophysicists ran their code for 18 straight days, consuming millions of hours of computer time, and generating massive amounts of data

KEPLER

NASA’s HEC Requirements: Time Critical NASA also has need for HEC resources that can handle time-sensitive mission-critical applications on demand (maintain readiness)

ReEntry

Storm Prediction

KEPLER

UAVSAR produces polarimetric (PolSAR) and interferometric (repeat-pass InSAR) data that highlight different features and show changes in the Earth over time

HECC enables the enormous planetary transit searches to be completed in less than a day, as opposed to more than a month on the Kepler SOC systems, with significantly improved accuracy and effectiveness of the software pipeline

HECC Assets

HECC Traditional Computer Floors

PDU

ISU 6

0 2 4 6 8 10 Scale in Feet

Visualization Lab Building 258

AMES RESEARCH CENTER

MOFFETT FIELD, CA 94035

N258 COMPUTER ROOM 125

John Parks Facilities

Chris Henze Visualization hyperwall (128-screen display) hyperwall Visualization

Systems

Updated

June 13, 2014 Updated By

Chris Buchanan

PDU

N258-Rm 125

Current Diagram

Sto rag e

DDN

RAID

SG

I

D R ack

SG

I

D R ack

SG

I

D R ack

SG

I

D R ack

SG

I D

R ack

SG

I

D R ack

SG

I

D R ack

SG

I

D R ack

16,800

6,900

2,700

1,275

2,080

HECC Modular Computer Floors

R&D 088 16,800

AMES RESEARCH CENTER

MOFFETT FIELD, CA 94035

N258 COMPUTER ROOM 230

Updated Updated By

November 7, 2016 Chris Tanner

0 2 4 6 8 10 Scale in Feet

B D F HA C E G

B D F HA C E G

R

S

T

U

V

W

X

NAS HPC Facility Modular Supercomputing Facility Floor Diagram

R

S

V

W

X

T

U

I J K

MSF Diagram

Project Manager FacilitiesNetworksCSS

Notes:

1. xx.

101001002003004005006007008 Broadwell

Bl an k

016 015 014 013 012 011 010 009 102

Bl an k

Broadwell

W at er Co nt ro l W at er

Co nt ro l

El ec tr ic al P an el

Fan Wall Evaporative Media & Filter Wall

Fan Wall Evaporative Media & Filter Wall

User Interface

2800kVA Transformer 13.8KV/415V Switchgear & 5 kVA

Transformer 415V/120V

Pleiades Specifics 161 SGI Racks (7.58 PF; 936 TB; 32,308 SBUs/hr) 158 SGI Altix ICE X Racks (7.21 PF; 931 TB; 32,134 SBUs/hr)

• 26 racks of ICE-X with Intel Xeon processor E5-2670 (Sandy Bridge):

623 TF; 59.9 TB; 3,407 SBUs/hr

• 75 racks of ICE-X with Intel Xeon processor E5-2680v2 (Ivy Bridge):

2.419 PF; 345.6 TB; 13,608 SBUs/hr

• 29 racks of ICE-X with Intel Xeon processor E5-2680v3 (Haswell):

2.004 PF; 267.3 TB; 6,974 SBUs/hr

• 28 racks of ICE-X with Intel Xeon processor E5-2680v4 (Broadwell):

2.167 PF; 129.0 TB; 8,145 SBUs/hr

3 SGI Coyote Racks (371 TF; 5 TB; 175 SBUs) (note, accelerators are not counted in SBU numbers)

• 2 racks of Intel Xeon processor E5-2670 (Sandy Bridge) and Nvidia K40 graphic processors: 296 TF; 4 TB; 117 SBUs

• 1 rack of Intel Xeon processor E5-2670 (Sandy Bridge) and Intel Xeon Phi 5110P accelerator processor: 75 TF; 1 TB; 58 SBUs

Cores

• 22,944 Intel Xeon processors (246,048 cores)

• 64 Nvidia GPUs (184,320 cores)

• 64 Intel Xeon Phi processors (3,804 cores)

Nodes

• 11,376 nodes (dual-socket blades)

• 64 nodes (dual-socket + GPU)

• 32 nodes (dual-socket + dual-Phi)

• 14 Login nodes

Networks

• Internode: Dual-plane partial 11D hypercube (FDR)

• Gigabit Ethernet Management Network

Electra Specifics 16 SGI Racks (1.24 PF; 147 TB; 4,654 SBUs/hr)

• 16 racks of ICE-X with Intel Xeon processor E5-2680v4 (Broadwell): 1.24 PF; 147 TB; 4,654 SBUs/hr

Cores

• 2,304 Intel Xeon processors (32,256 cores)

Nodes

• 1,152 nodes (dual-socket blades)

Networks

• Internode: Dual-plane fully-populated 7D hypercube

(FDR)

• Gigabit Ethernet Management Network

• Metro-X IB extenders for shared storage access

Merope Specifics

56 SGI Altix ICE X ½ Racks (252 TF; 86 TB; 1,792 SBUs)

• 56 ½-racks of 8400EX with Intel Xeon processor E5670

(Westmere): 252 TF; 86 TB; 1,792 SBUs 3,584 Intel Xeon processors (21,504 cores)

• 3,584 six-core Westmere

• 2.93 GHz processors (21,504 cores)

Endeavour Specifics 32 TF constellation-class supercluster

2 SGI Ultra Violet 2 nodes with Intel Xeon E5-4650L

2.6 GHz processors

• One 512-core node with 2 TB globally addressable RAM (Endeavour1)

• One 1,024-core node with 4 TB globally addressable RAM (Endeavour2)

Interconnect

• Intranode: NUMALink-6 (enable large SSI)

• Dual-Plane QDR InfiniBand connectivity into Pleiades infrastructure » 1 connection from each node into IB0 for TCP traffic (pbs, login, …) » IB1 is for I/O traffic to the Lustre file systems.

Endeavour1 has 3 connections and Endeavour2 has 4 connections.

• 10 Gb Ethernet can be used for WAN traffic

Advanced Visualization: Hyperwall and CV Supercomputing-scale visualization system to handle massive size of simulation results and increasing complexity of data analysis

• 8x16 LCD tiled panel display (23 feet x 10 feet)

• 245 million pixels

• Debuted as #1 resolution system in the world

• In-depth data analysis and software

Two primary modes

• Single large high definition image

• Sets of related images (e.g. parameter study)

High-bandwidth to HEC resources

• Concurrent Visualization: Runtime data streaming allows visualization of every simulation time step - ultimate insight into simulation code without increase in traditional disk I/O

• Traditional Post-Processing: Direct read/write access to Pleiades filesystems eliminates nee for copying large datasets

GPU-based computational acceleration R&D for appropriate NASA codes

Storage and Archive

• Lustre File Systems (39.6 PB in 7 file systems)

• DDN

» 14 DDN RAID Systems, 9.9 PB total, 3 facility-wide file systems

• NetApp » 62 RAID Systems, 29.7 PB total, 4 facility-wide file systems

• NFS File Systems

• 3 home file systems 3.7 TB total

• 2 facility-wide scratch file systems 59 TB & 1 PB

• .4 PB for NEX

• Archive System

• 490 PB Maximum Capacity

• 6 T950 Spectra Logic Libraries

HECC Growth

HECC Growth

Largest HECC LINPACK Result

HECC Conducts Work in Four Major Technical Areas

Supercomputing Systems

Data Analysis and Visualization

Application Performance and User Productivity

Networking

Provide computational power, mass storage, and user-friendly runtime environment through continuous development of management tools, IT security, systems engineering

Facilitate advances in science and engineering for NASA programs by enhancing user productivity and code performance of high-end computing applications of interest

Create functional data analysis and visualization software to enhance engineering decision support and scientific discovery by incorporating advanced visualization technologies

Provide end-to-end high-performance networking analysis and support to meet massive modeling and simulation distribution and access requirements of geographically dispersed users

Supporting Tasks Facility, Plant Engineering, and Operations: Necessary engineering and facility support to ensure the safety of HECC assets and staff Information Technology Security: Provide management, operation, monitoring, and safeguards to protect information and IT assets User Services: Account management and reporting, system monitoring and operations, first-tier 24x7 support Internal Operations: NASA Division activities hat support and enhance the HECC Project areas

Resource Utilization

User Location

CT – 6

DC – 6

MA – 56

MD -- 166

NH – 5

NJ – 16

RI – 2

VT – 3

Offshore – 71

United States of America

80°W 75°W50°N 70°W

25°N

30°N

35°N

40°N

45°N

85°W90°W95°W100°W105°W110°W115°W120°W125°W

Tropic of Cancer

Lake Superior

La ke

M ic hi ga n

Lake Huron

Lake Erie

Lake Ontario

ATLANTIC

OCEAN

Gulf of Mexico

PACIFIC

OCEAN

W

E

N

S

20°N

22°N

160°W 156°W 60°N

70°N

50°N 160°W 140°W

Bering Sea

Gulf of Alaska

250 500

250 500

0mi

0km

100 200

100 200

0mi

0km

National boundary State boundary

LEGEND

125 250

125 250

0mi

0km

1 162

Quarterly Utilization Over 10+ Years

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

35,000,000

40,000,000

45,000,000

50,000,000

55,000,000

60,000,000

65,000,000

Q3 F Y20

Q4 F

Q1 F

Q2 F

Q3 F

St an da rd

B ill in g

U ni ts

SOMD

ESMD

NAS

NLCS

NESC

SMD

HEOMD

ARMD

Alloc. to Orgs

75% of Peak Capacity

Aeronautics Support (55,958,567 SBUs) Advanced Air Vehicles # of projects: 142 # of SBUs used*: 16,966,350

✦ HECC is used to develop concepts and technologies for dramatic improvements in the noise, emissions, and performance of transport aircraft.

✦ HECC is used to develop concepts and technologies to increase rotorcraft speed, range and payload, and decrease noise, vibration and emissions.

✦ HECC is used to develop advanced computer-based prediction methods for supersonic aircraft shape and performance and to develop technologies that will help eliminate today's technical barriers (such as sonic booms) to practical, commercial supersonic flight.

✦ HECC is used to develop computer-based tools and models and scientific knowledge that will lead to significant advances in our ability to understand and predict flight performance for a wide variety of air vehicles.

Transformative Aeronautics Concepts # of projects: 45 # of SBUs used*: 26,195,249

✦ HECC is used to develop and utilize Reynolds-averaged Navier-Stokes (RANS) and Large Eddy Simulation (LES) methods, and hybrid RANS-LES techniques to improve calculation methods for propulsion flows dominated by turbulent boundary layers and mixing.

✦ HECC is used to assess natural laminar flow concepts, to elucidate the physics and control of boundary layer transition in swept wing flows and drag reduction concepts for compressible boundary layers.

✦ HECC is used to validate chemistry, chemistry-turbulence and spray models being developed under the National Jet Fuels Combustion program.

Airspace Operations and Safety & Other # of projects: 8 # of SBUs used*: 506,017

� HECC is used for developing reliable computational tools for predicting and analyzing stability & control characteristics of aircraft prior to or while encountering loss-of-control flight conditions characterized by abnormal flight (e.g., stall), abnormal vehicle conditions (e.g., damage, jammed control surfaces), external upsets (e.g., wake vortex, wind shear, gusts), and icing.

� HECC is used to develop methods for computing aerodynamic performance degradation associated with ice accretions on swept wing geometries.

� HECC is used to produce real-time icing impact fields for flight planning and post mission analysis.

Integrated Aviation Systems # of projects: 10 # of SBUs used*: 12,290,951

� HECC is used for accurate prediction of airframe noise from a full scale aircraft and evaluation of flap and landing gear noise reduction concepts in flight environments.

� HECC is used to develop technology for compact, high-power-density electric motors to power an all-electric general-aviation aircraft or helicopter, a hybrid turbine-electric regional airliner or a large transport with many small engines distributed around the aircraft.

� HECC used for parametric studies conducted to optimize size, shape and placement of an array of fluidic actuators for maximizing the lift for control surfaces on an aircraft, which would help reduce the size of control surface and the weight of an aircraft.

*October 1, 2015 to September 30, 2016

Aviation Safety Program The Aviation Safety Program (AvSP), part of NASA’s Aeronautics Research Mission Directorate, helps to develop new ways to achieve exceptional levels of safety for air travel despite increasingly crowded skies and congested airports.

Over the past decade, collaboration between in-dustry and government to proactively identify new risks has led to historically low rates of commercial accidents. But as air traffic volume increases, the vigilance of the aviation community must continue.

That’s why, working with partners from academia and in the public and private sectors, AvSP con-ducts foundational research and develops new technologies to overcome the emerging challeng-es created by the nation’s transition to the Next Generation Air Transportation System (NextGen).

To help provide solutions, the program inves-tigates improvements to increase the inherent safety of aircraft systems and structures, ways to avoid atmospheric hazards, and development of next-generation concepts for on-board and on-ground safety systems. AvSP studies:

systems;

eliminate any potential issues;

systems to improve overall performance;

issues from ever occurring; and related systems.

Images (Clockwise, from top-left) Data Mining: -ing safety issues are already proving useful to commercial airlines. Flight Safety: Subscale models are put through loss-of-control scenarios in NASA wind tunnels to test new recovery techniques. Engine Icing Prevention: Studies are being done into the types of atmospheric conditions that can form ice particles inside engines, leading to power loss. Human-Friendly Flight Decks: NASA uses

Human Exploration and Operations & Safety Support (51,658,239 SBUs)

Surface pressure coefficient with sonic iso-surfaces, from an 11,640-processor computation of the transonic flow about a Space Launch System configuration. The time-accurate computation is being performed to predict unsteady surface pressures and ultimately the development of buffet loads.

� HECC is used to simulate the effect of larger solid rocket boosters and new propulsion systems on the launch facility at Kennedy Space Center, such as investigating whether ignition overpressure waves generated during liftoff are fully suppressed by the existing water suppression system.

� HECC is used to evaluate visiting-vehicle induced loads on the International Space Station (ISS) during mated and rendezvous operations and to evaluate crew Extra-vehicular Activity/Intra-vehicular Activity and attitude control loads on ISS.

� HECC is used in developing a combustion response model to investigate combustion instability in hydrocarbon-fueled rocket engines.

� HECC is used for technology development for entry, descent and landing systems.

*October 1, 2015 to September 30, 2016

Multi-Purpose Crew Vehicle # of projects: 7 # of SBUs used*: 1,636,080

✦ HECC is used to support the creation of hundreds of computational solutions that model the flow field around the Crew Module and Launch Abort System for all flight regimes to be used as input for the aerodynamic databases.

� HECC is used to run computational fluid dynamics simulations to study the aerodynamic and aerothermal environments for the Multi-Purpose Crew Vehicle.

� HECC is used to develop and deploy a prototype system for rapid aerodynamic performance database generation and to use it on real-world problems faced by the Human Exploration and Operations mission directorate.

Space Launch Systems # of projects: 13 # of SBUs used*: 31,423,169

� HECC is used for computational fluid dynamics simulations of Space Launch Systems ascent to assess aerodynamic performance, protuberances, stage separation, and plume effects (such as plume-induced flow separation) for evolving vehicle designs.

� HECC is used for computational fluid dynamics analysis of Advanced Booster development efforts in the combustion stability areas.

� HECC is used for prediction of the launch induced environment for the Space Launch System including liftoff acoustics, ignition over-pressure, separation environments, debris, Launch Pad Abort Environments and hydrogen entrapment.

� HECC is used to simulate tanks and main propulsion system components (including feedlines, valves, manifolds, ducts, and pogo accumulators) for evaluation of criteria such as flow uniformity and component pressure drop.

HEOMD - Space Flight Operation & General # of projects: 32 # of SBUs used*: 11,042,783

NASA Engineering & Safety Center # of projects: 8 # of SBUs used*: 7,556,207

� HECC is used for simulations to provide guidance to the Space Launch System advanced booster designers by providing aerodynamic loading implications for various potential advanced booster geometric configurations.

� HECC is used to improve the capability to predict combustion stability in liquid rocket engines to increase NASA engineers’ capability to more confidently and efficiently identify and mitigate combustion stability issues in engine development programs.

� HECC is used to used for studies of large eddy simulations of oblique-shock / supersonic hot jet interaction, aimed at prediction of plume-induced vibroacoustics.

Science Support (94,488,707 SBUs) Astrophysics # of projects: 100 # of SBUs used*: 37,875,367

� HECC is used by the Kepler mission to find Earth-sized planets around other stars and to fully analyze the Kepler data to find any undiscovered planets still “hiding” in the data.

✦ HECC is used to understand the physics of high redshift galaxy formation and make detailed predictions that can be used to guide NASA observations of the first galaxies.

� HECC is used for quantifying the redistribution of matter in galaxies when supernova energy is deposited; exploring the growth of black holes and the impact of active galactic nuclei on galaxy evolution; and determining whether the ultraviolet light from stars in galaxies can "escape" to re-ionize the universe.

Earth Science # of projects: 136 # of SBUs used*: 18,460,872

✦ HECC is used to combine observational data with numerical simulations of the global ocean circulation to provide vital information for understanding climate change and its impact on land and sea ice, ocean ecology, and the global carbon cycle.

✦ HECC is used for high-resolution cloud resolving model simulations to provide unique and detailed insights into the processes that form tropical clouds and cloud systems, which account for approximately two-thirds of global rainfall.

✦ HECC is used explore the feedback mechanisms between polar ice sheet and atmosphere circulation in order to determine how global temperature changes translate into increased sea level rise.

✦ HECC is used to improve the understanding of the current balance of carbon in the Arctic and to provide a framework for early detection of future carbon destabilization.

Heliophysics # of projects: 100 # of SBUs used*: 19,458,069

✦ HECC is used for modeling solar magneto-convection in order to understand how magnetic fields emerge through the sun’s surface, heat the sun’s outer atmosphere, and produce sunspots, spicules, and flares.

✦ HECC is used for realistic multi-scale simulations to understand the complicated physics of the turbulent convection zone and atmosphere of the sun and for analyzing and interpreting observations from the NASA space missions.

✦ HECC is used to simulate small-scale magnetic fields generated by turbulent dynamo action just beneath the solar surface in order to accurately predict space weather events that impact the Earth environment.

Planet…