PKG00250359-instructions.docx

PREDICTIVE SCIENCE ACADEMIC ALLIANCE PROGRAM III DE-FOA-0002068

FUNDING OPPORTUNITY ANNOUNCEMENT

PROJECT TITLE: ADVANCED SIMULATION AND COMPUTING PREDICTIVE SCIENCE ACADEMIC ALLIANCE PROGRAM (PSAAP III)

FUNDING OPPORTUNITY ANNOUNCEMENT (FOA) Number:

DE-FOA-0002068

U.S. Department of Energy National Nuclear Security Administration Office of Acquisition Management

11:59 PM EST

Registration Requirements

There are several one-time actions applicants must complete in order to submit an application in response to this Announcement (e.g., obtain a Dun and Bradstreet Data Universal Numbering System (DUNS) number, register with the System for Award Management (SAM), register with Grants.gov, or register with another portal as indicated in the FOA). Applicants who are not registered with SAM and Grants.gov should allow up to five weeks to complete these requirements. It is suggested that the process be started as soon as possible.

If an applicant has not already done so, they must:

1. Obtain a DUNS number. DUNS website: http://fedgov.dnb.com/webform.

2. Register with the SAM. SAM website: http://www.sam.gov/. For more information about SAM registration for applicants please refer to: https://www.sam.gov/sam/transcript/Quick_Guide_for_Grants_Registrations_v1.7.pdf.

3. Register with Grants.gov and set up your Workspace. There are multiple steps to this process, for more information about Grants.gov registration for applicants please refer to: https://www.grants.gov/web/grants/applicants/registration.html

For more information about setting up Workspace please refer to:

https://www.grants.gov/web/grants/applicants/workspace-overview.html For more detailed instructions about creating a profile on Grants.gov, refer to: https://www.grants.gov/web/grants/applicants/registration/authorize-roles.html IMPORTANT NOTICE TO POTENTIAL APPLICANTS: When you have completed the Grants.Gov registration process, you should call the Grants.gov Helpdesk at 1-800-518-4726 to verify that you have completed the final step.

4. Register with FedConnect to submit questions. FedConnect website: www.fedconnect.net.

Questions

Questions regarding the content of the funding opportunity announcement must be submitted through the FedConnect portal. You must register with FedConnect to respond as an interested party to submit questions, and to view responses to questions. It is recommended that you register as soon after release of the FOA as possible to have the benefit of all responses. DOE/NNSA will try to respond to a question within 3 business days, unless a similar question and answer have already been posted on the website.

Questions relating to the Grants.Gov registration process, system requirements, or how an application form works must be directed to Grants.gov at 1-800-518-4726 or support@grants.gov.

Application Preparation

Applicants must prepare the application package and application forms from the Grants.gov website: http://www.grants.gov/ Application Submission Applications must be submitted through Grants.gov to be considered for award. You cannot submit an application through Grants.gov unless you are registered. Please read the registration requirements carefully and start the process immediately. Remember you have to update your SAM registration annually. If you have any questions about your registration, you should contact the Grants.gov Helpdesk at 1-800-518-4726 to verify that you are still registered in Grants.gov.

Table of Contents

Section I - FUNDING OPPORTUNITY DESCRIPTION

DESCRIPTION

A. SUMMARY

The Defense Programs’ Office of Research, Development, Test, and Evaluation within the Department of Energy’s (DOE) National Nuclear Security Administration (NNSA), directs research and development activities to maintain the safety, security, and effectiveness of the nation’s nuclear weapons stockpile. This nuclear deterrent remains a vital part of our national security infrastructure. It maintains strategic stability, deters potential adversaries, and reassures our allies and partners of our security commitments. Since 1992, the United States has observed the moratorium on underground nuclear testing while significantly decreasing the size of its nuclear arsenal. National Policy required NNSA and its weapons laboratories to institute the science-based Stockpile Stewardship Program (SSP) to ensure the safety, security, and effectiveness of the stockpile, while allowing the country to pursue a variety of nonproliferation and disarmament goals. The SSP combines sophisticated experiments, highly accurate physics modeling, and improved computational power to simulate and predict nuclear weapon performance over a wide range of conditions and scenarios.

In 1995 the U.S. Department of Energy (DOE) established the Accelerated Strategic Computing Initiative (ASCI), which is now known as the Advanced Simulation and Computing (ASC) program. Its purpose is to bring together the resources of three NNSA Defense Program national laboratories in a consolidated effort to achieve significant advances in computational modeling and simulation capabilities for stockpile stewardship. ASCI recognized that some of its program objectives are only achievable by the establishment of a strong research portfolio of strategic alliances with leading U.S. academic institutions. To help meet the challenge, ASCI’s Academic Strategic Alliance Program (ASAP) was formed in 1997 to engage the U.S. academic community in advancing science-based modeling and simulation technologies. In 2008, the next phase of the alliance program, PSAAP II, continued the academic engagement, adding an emphasis on validation, verification, and uncertainty quantification. The program continued in 2014 with the establishment of PSAAP II that provided for an additional focus on computer science.

As PSAAP II begins to wind down the NNSA Office of Advanced Simulation and Computing (ASC) and Institutional Research and Development Programs, Lawrence Livermore National Laboratory (LLNL), Los Alamos National Laboratory (LANL) and Sandia National Laboratories (SNL), are initiating the next phase of its academic science program, called Predictive Science Academic Alliance Program III or PSAAP III.

This PSAAP III Funding Opportunity Announcement (FOA) is a call for applications to create a program consisting of some combination of the following:

· Multi-disciplinary Simulation Center (MSC) - whose research will focus on scalable application simulations, targeting large-scale, integrated multidisciplinary problems to be carried out on ASC’s unclassified high-performance computing (HPC) systems at LANL, LLNL and SNL that will be made available to the funded PSAAP III “Centers”. An MSC must (1) develop and demonstrate computer science technologies and methodologies that will advance Exascale computing, and (2) demonstrate an integrated, verified, validated predictive simulation with uncertainty quantification. Both (1) and (2) must be demonstrated within the context of the proposed integrated multidisciplinary application. It is expected that an MSC will demonstrate a significant advance in predictive science, in the context of their application. The overall advance should require a combination of significant advances in some Exascale-enabled piece of science, integration science, and computer science, together with wider use of state-of-the-art techniques. The advance would consist of better predictive capability, as manifested by predictions of a wider range of phenomena, with improved predictive accuracy and reduced uncertainty, in comparison to existing capability at the beginning of the project. Integrated system simulation results must be produced each year, beginning in program year two.

· Single-Discipline Center (SDC) - whose research will focus on scalable application simulations, targeting a broad single science/engineering discipline, to be carried out on ASC’s unclassified HPC systems at LANL, LLNL and SNL that will be made available to the funded PSAAP III Centers. The SDC must (1) develop and demonstrate computer science technologies and methodologies that will advance Exascale computing, and (2) demonstrate a verified, validated predictive simulation with uncertainty quantification. Both (1) and (2) must be demonstrated within the context of the proposed problem. It is expected that an SDC will demonstrate a compelling and significant scientific advance in the single discipline, enabled by Exascale computing. The technical advance should ideally represent a qualitative step up in the discipline, as opposed to incremental progress. The overall advance should require a combination of significant advances in some Exascale-enabled piece of science, integration science, and computer science, together with wider use of state-of-the-art techniques. While the research efforts within an SDC may follow several divergent paths under the broad umbrella of the single science/engineering discipline, it is required that all research paths be integrated into a single demonstration problem, for which simulation results must be produced each year beginning in the second year of the program.

· Focused Investigatory Center (FIC) – whose research will be tightly focused on a specific research topic either in one of the disciplines or one or more of the Exascale enabling technologies listed below. FICs will not necessarily have a tie to an application or be required to demonstrate a verified, validated predictive simulation with uncertainty quantification. It is expected that a successful FIC will demonstrate a compelling and significant scientific advance in the single discipline or enabling technology. The technical advance should represent a qualitative step up in the discipline, as opposed to incremental progress.

B. PROGRAM VISION

The vision for PSAAP III consists of participation by leading US universities focusing on three major integrated areas:

· Discipline-focused research needed to further predictive science and enabled by effective Exascale computing technologies[footnoteRef:1]; [1: The phrase “Exascale computing” used herein refers to the next High Performance Computing (HPC) paradigm shift to extreme, heterogeneous, multi-core on-node parallelism – and not necessarily to any hardware or system at such scale.]

· Developing and demonstrating technologies and methodologies to support effective Exascale computing in the context of science/engineering applications; and

· Predictive Science based on verification and validation and uncertainty quantification (V&V/UQ) for large-scale simulations

C. TECHNICAL SCOPE AND TOPICAL RESEARCH AREAS

Predictive Science is potentially applicable to a variety of fields, including nuclear weapons, efficient manufacturing, biological systems, nanoscale material science, organic chemical processes, climate modeling, etc. Success in these simulations requires both software and algorithmic frameworks for integrating multiple disciplines into a single application and adding significant disciplinary strength and depth to make that integration effective. MSCs, SDCs and FICs with interest in disciplinary research should consider advancing one or more of the following disciplines of interest to NNSA’s mission. The following list should not be considered inclusive:

· Hydrodynamics, Turbulence, Transition to Turbulence

· Hypersonics

· Radiation and Particle transport

· Atomic Physics

· Plasma physics, Transition to Turbulent Plasma

· Reactive materials, Energetic materials

· Chemical transformations, Combustion

· Solid-solid phase transitions, plastic flow

· Shock-assisted and shock-induced reactions

· Equations of state and constitutive properties

· Material damage and failure

· Material stability

· Novel materials

· Nuclear properties

· Physics related to nuclear reactors

· Inertial confinement fusion energy systems

· Design of experiments for validation, including surrogate materials and environments

· Engineering mechanics and design

· Molecular dynamics

The following application areas are not primary scientific disciplines relevant to the NNSA mission and are therefore not of interest for this particular FOA:

· Response to natural and man-made threats

· Weather

· Climate

· Science of natural disasters (earthquakes, tsunami, etc.)

· Infectious diseases

· Protein dynamics

· Eco-systems

· Magnetic fusion systems

· Crowd behavior

· Nuclear reactor design

· Bioscience and bioengineering

· Economics and business systems

· Logistics and agency resource deployment

· Internal combustion engines

In addition, successful MSC and SDC applications should conduct computer science research in areas that will contribute to the advancement of Exascale computing technologies and demonstrate the results in the context of the chosen application. The following topics are of particular interest to the NNSA National Laboratories as they move towards Exascale computing. Other topics that will enable the advancements in Exascale computing are encouraged as well. FICs wanting to pursue research in a specific area of computer science are encouraged to choose from the following list of topics:

· Data Analytics for science/engineering applications

· Machine learning (ML) for science/engineering applications

· Quantifying uncertainty in ML

· Neuromorphic computing

· Impact of embedded ML on application performance

· Statistical fusion of simulation and experimental data

· Rigorous math models for data analytics

· Exploration of advanced HPC architectures in the context of the chosen application

· Architecture simulation/emulation

· Performance prediction

· Programming environments and runtime systems

· Composition of libraries, runtimes, programming languages

· Performance portability to insulate users from hardware

· Task based programming

· Workflow automation

· Simulation setup, simulation runs or complex ensemble runs and post processing

· Management of bulk data

· Simulation repeatability, e.g. role of containers

· Dynamic resource management, e.g. UQ pipeline

· Productivity and performance portability

· Programming models and tools

· Techniques for adding new physics

· Environments for efficient development of simulations

· Heterogeneous computing systems

· New approaches to engineering

· Models and processes for the physics and chemistry of additive manufacturing

· Machine learning applied to design

· Algorithms/models

· Novel approaches to coupling multiphysics/multiscale

· Algorithms for increasing performance of HPC systems, e.g. latency hiding, reduction of synchronization, utilization of simultaneous execution

· Support for resilience

· Exposing more parallelism at the cost of algorithm efficiency

· Reduced order models and their use in ensemble analysis

· Stochastic algorithms and adaptive algorithms

While the ASC is extremely interested in pursuing topics related to Machine Learning (ML), ML is neither necessary nor sufficient for an application to be competitive. All applications using ML should be clear about the value it will bring to the science or Exascale research. We will expect you to discuss the specific ML techniques you will use, and the advantages and disadvantages of the approaches chosen. As with other parts of the application having faculty or staff with a proven track record in this area is essential. Just mentioning ML without providing details will weaken rather than strengthen the application.

NOTE: All proposed work to be funded through this PSAAP III program announcement is to be UNCLASSIFIED. No applications for CLASSIFIED work will be accepted.

D. MISSION AREA DESCRIPTION AND REQUIREMENTS

(MSCs and SDCs) Applicants shall be required to work toward solving an integrated/overarching problem by the final year of funding through the advancement, development, and application of science and research to multiscale V&V/UQ challenges. Applicants will be required to demonstrate on available NNSA supercomputers (availability determined based on requirements and ASC program priorities) a full-system prediction within the second year. Centers perform work that will contribute to the development of effective Exascale computational science. They must demonstrate advances in and utilize state-of-the-art V&V/UQ. Applicants may apply as an MSC or SDC.

MSCs select an overarching multidiscipline, multiscale application from science/ engineering disciplines.

SDCs select an integrated multiscale problem or challenge from a single discipline.

FICs Applicants will be required to focus on a specific research topic either in one of the disciplines or one or more of the Exascale enabling technologies listed above. Successful FIC Applicants will be expected to demonstrate a compelling and significant scientific advance in the single discipline and/or an enabling technology. The technical advance should represent a qualitative step up in the discipline, as opposed to incremental progress.

E. MANDATORY SITE VISITS AND COLLABORATION WITH NNSA NATIONAL LABORATORIES

NNSA-funded graduate students at each Center are required to complete a 10 consecutive week visit to one of the three NNSA National Laboratories during their graduate career. During their visits graduate students shall conduct research related to their responsibilities at their home institution. In addition, they shall take advantage of opportunities provided by the Laboratories that expose them to Laboratory research programs.

Additionally, each Center shall implement collaboration among Center participants and employees of one or more of the three NNSA National Laboratories. Each Center may engage in the any of the following types of collaboration, or other similar types of collaboration, with NNSA National Laboratories:

· Workshops

· Symposia

· Campus visits

· Supervising students (serving on doctoral committees)

· Adjunct faculty appointments

· Guest lectures

The NNSA National Laboratories’ participation/collaboration shall be limited to supervising required visits and ancillary activities such as those described above. NNSA National Laboratories shall not be engaged by Centers to perform any portion of the substantive technical work/research authorized in any resulting awards. However, the Centers are encouraged to collaborate with the National Laboratories on ongoing research which is related to, or which overlaps with, research that will be conducted by the Centers.

Section II - AWARD INFORMATION

A. TYPE OF AWARD INSTRUMENT

COOPERATIVE AGREEMENTS

DOE anticipates awarding cooperative agreements under this funding opportunity announcement.

B. ESTIMATED FUNDING

AMOUNT NEW AWARDS

An estimated total of approximately $20M per year, over a five year period, is anticipated to be available for the entire program, subject to annual appropriations. The first funding year will begin in FY20 and repeat, upon a successful annual continuation application, on an annual period thereafter, for a period of five years. The expected amount for an individual award is dependent on the type of center. See the table below for the estimated individual project award funding.

Funding for costs incurred by the NNSA National Laboratories (Lawrence Livermore National Laboratory, Los Alamos National Laboratory, and Sandia National Laboratories) through their participation/collaboration with the Centers shall be provided by NNSA directly to the NNSA National Laboratories through existing channels and shall not be included in the PSAAP III Centers’ budgets.

AMOUNT MULTIPLE YEAR AWARDS

Approximately $20M is expected to be available for new awards in FY 2020 and an additional $80M is expected to be available for awards under this announcement in years FY 2021 through FY 2024. The following is the expected breakdown by Center for years one through five.

* Award size will vary depending on the approved scope of work ** Contingent upon additional years being awarded

AMOUNT APPROPRIATION DEPENDENT

Funding for all awards and future budget periods are contingent upon the availability of funds appropriated by Congress for the purpose of this program and the availability of future-year budget authority.

C. MAXIMUM AND MINIMUM AWARD SIZE

See above table for minimum and maximum funding for each type of Center.

D. EXPECTED NUMBER OF AWARDS

Under this announcement, DOE/NNSA expects to make awards by type for each of the following categories: Multi-disciplinary Simulation Center (MSC); Single-Discipline Center (SDC); and Focused Investigatory Center (FIC). It is anticipated that multiple awards will be made for each type of Center. Each application will be reviewed by a merit evaluation panel and the number of awards will be based on adherence to the requirements as written in this FOA, the alignment to the PSAAP III Program Vision, the technical work proposed, and the adequacy of the team resources. Due to the variance in the scope and funding level for each type of Center it is unknown how many individual awards will be funded.

DOE/NNSA is under no obligation to pay for any costs associated with the preparation or submission of an application.

F. PERIOD OF PERFORMANCE

It is anticipated that awards to MSCs and SDCs will be for five years, along with FICs being funded from one year up to five years.

G. TYPE OF APPLICATION

DOE will accept only new applications under this announcement. Previously funded PSAAP II Centers may apply as well but are required to propose a new and different application problem, that is, a significantly new phenomenon must occur in the new proposed problem.

H. LIMITATIONS ON PSAAP III APPLICATIONS BY SAME ACADEMIC INSTITUTION

DOE/NNSA will not fund more than one MSC or SDC per campus, however DOE/NNSA will accept multiple applications for FICs. In other words an institution may have one MSC or SDC, and one or more FICs.

Section III - ELIGIBILITY INFORMATION

A. ELIGIBLE APPLICANTS

RESTRICTED ELIGIBILITY (DECEMBER 2014)

In accordance with 2 CFR 910.126, Competition, eligibility for award is restricted to a United States academic institution that can grant PhD degrees. (A for-profit or non-profit business entity may be a subcontractor providing labor effort for MSC, SDC or FIC research.) The academic institution must be one that:

a. Admits as regular students only persons having a certificate of graduation from a school providing secondary education, or the recognized equivalent of such a certificate;

b. Is legally authorized within a State to provide a program of education beyond secondary education;

c. Provides an educational program for which the institution awards PhD degrees;

d. is accredited by a nationally or regionally recognized accrediting agency or association, or if not so accredited, is an institution that has been granted reaccreditation status by such an agency or association that has been recognized by the U.S. Secretary of Education for the granting of reaccreditation status, and the U.S. Secretary of Education has determined that there is satisfactory assurance that the institution will meet the accreditation standards of such an agency or association within a reasonable time.

Distinct academic campuses (e.g., that award their own degrees, have independent administrative structures, admissions policies, alumni associations, etc.) within multi-campus systems qualify as separate submission-eligible institutions.

Adjunct professors, visiting professors and post-doctoral fellows are ineligible to be Principal Investigators.

FFRDC/NATIONAL LABORATORIES

Federally Funded Research and Development Center (FFRDC)/National Laboratories Collaboration Support (includes DOE/NNSA National Laboratories and other Federal Agency-Sponsored FFRDCs)

FFRDCs are not eligible for an award under this announcement. However, FFRDCs may participate as collaborators.

FFRDCs are not eligible to receive any direct funding associated with this FOA, except for reimbursement for use of facilities that require “user fees” in accordance with the institution’s written policies or procedures.

To affect this collaboration, a letter from an FFRDC employee who will be collaborating with the applicant must be submitted with the application. This letter must outline their understanding and role in the collaboration effort and indicate that no funds will be provided to the FFRDC from the awarded grant except for reimbursement for use of facilities that require “user fees” as stated above.

If an applicant is selected for award, the FFRDC contractor(s) participating as collaborating partners must receive prior authorization from the cognizant Federal Agency for non-DOE/NNSA FFRDCs and from the cognizant Contracting Officer for DOE/NNSA FFRDCs, subject to the following guidelines:

Authorization for non-DOE/NNSA FFRDCs. If selected for award, the Federal agency sponsoring the FFRDC contractor must authorize in writing the use of the FFRDC contractor on the proposed project and this authorization must be submitted prior to award. The use of a FFRDC contractor must be consistent with the contractor's authority under its award and must not place the FFRDC contractor in direct competition with the private sector.

Authorization for DOE/NNSA FFRDCs. If selected for award, the cognizant contracting officer for the FFRDC must authorize in writing the use of a DOE/NNSA FFRDC contractor on the proposed project and this authorization must be submitted prior to award. The following wording is acceptable for this authorization.

"Authorization is granted for the [Name] Laboratory to participate in the proposed project. The work proposed for the laboratory is consistent with or complimentary to the missions of the laboratory, will not adversely impact execution of the DOE/NNSA assigned programs at the laboratory, and will not place the laboratory in direct competition with the domestic private sector."

If selected for award, the recipient is responsible for making contact with the appropriate DOE/NNSA FFRDC collaborating partner(s) who should in turn obtain prior written authorization from the DOE/NNSA FFRDC Contracting Officer.

Applicant Responsibility. The applicant, if successful, will be the responsible authority regarding the settlement and satisfaction of all contractual and administrative issues, including but not limited to, disputes and claims arising out of any agreement between the applicant and the FFRDC contractor.

B. COST SHARING

COST SHARE 10%

The cost share must be at least 10% of the total allowable costs of the project (i.e., the sum of the Government share and the recipient share of allowable costs equals the total allowable costs of the projects) and must come from non-Federal sources unless otherwise allowed by law. (See 2 CFR part 200 as amended by 2 CFR part 910 (DOE Financial Assistance Regulation) for the applicable cost sharing requirements.) The applying institution will be contributing 10% as cost matching to the NNSA award on an annual basis. The lead institution will be responsible for the cost sharing contributions under this award.

C. OTHER REQUIREMENTS

The Applicant must include in the application a letter of support from the Applicant’s Vice President for Research, or someone in a comparable position, that affirms the proposed Center will comply with the following five performance requirements:

· Every graduate student, who is financially supported under a resulting cooperative agreement, must spend at least one period of ten contiguous weeks collaborating at one of the three NNSA Defense Programs National Laboratories. Associated costs must be included in the proposed budget. PSAAP III funds may only be used for students who are U.S. citizens or non-U.S citizens from non-sensitive countries. However, your university can provide funding support for students from DOE-listed sensitive countries as part of its 10% cost sharing contribution to the PSAAP III Center.

· Every research staff member and post doc (funded half-time or greater), who is financially supported under a resulting cooperative agreement, must spend at least one week collaborating at one of the three NNSA Defense Programs National Laboratories each year. (Associated costs must be included in the proposed budget.) PSAAP III funds may only be used for research staff and post docs who are U.S. citizens or non-U.S. citizens from non-sensitive countries. However, your university can support research staff members and post docs from DOE-listed sensitive countries as part of its 10% cost sharing contribution to the PSAAP III Center.

· The Principal Investigator will host an annual peer review organized by ASC at a mutually convenient time.

· For MSCs and SDCs, the Principal Investigator agrees to provide semi-annual progress reports that describe key scientific, simulation, computer science and V&V/UQ progress, and other issues, in addition to responses to the recommendations of the annual peer reviews. The report will include the Center’s technical goals for the coming year and provide an updated project plan, software plan, and prediction and V&V plan, as appropriate.

· For FICs, the Principal Investigator agrees to provide annual progress reports that describe key advances in their scientific discipline and/or enabling technology development, and other issues, in addition to responses to the recommendations of the annual peer reviews. The report will include the Center’s technical goals for the coming year and provide an updated project plan, as appropriate.

· The Principal Investigator will not submit more than one application as Principal Investigator.

· The academic institutions selected for awards will be held responsible for managing the entire scope, schedule and cost of the project, to include all reporting, per DOE F 4600.2, Federal Assistance Reporting Checklist and Instructions.

Section IV - APPLICATION AND SUBMISSION INFORMATION

A. ADDRESS TO REQUEST APPLICATION PACKAGE

APPLY AT GRANTS.GOV

Application forms and instructions are available at Grants.gov. To access these materials, go to http://www.grants.gov, select "APPLICANTS”, then select “Apply for Grants," and then select "Get Application Package” button. Enter the funding opportunity number (DE-FOA-0002068). Under the PACKAGE tab, select “Apply” under the Actions column, and under Option: 1 Apply Now Using Workspace, select the Login to Apply Now button. Follow the prompts in Workspace to complete the application package. You are encouraged to review the Applicant FAQs in Grants.gov as you prepare and submit your application.

B. LETTER OF INTENT AND PRE-APPLICATION

Letters of Intent and Pre-applications are not required.

C. CONTENT AND APPLICATION FORMS

All applicants including sub-applicants must complete the mandatory forms and any applicable optional forms as provided under Section IV – Application and Submission Information, sub-section D. Summary of Required Forms and Files of this FOA. All documents that are attached as forms must be submitted in Adobe Portable Document Format (PDF) unless otherwise specified in this FOA. Attached PDF files must be plain files consisting of text, numbers, and images without editable fields, signatures, passwords, redactions, or other advanced features available in some PDF-compatible software. Do not use PDF portfolios or binders.

Please note: you may only use the following UTF-8 characters when naming your application attachments: A-Z, a-z, 0-9, underscore (_), hyphen (-), space, period. You must limit the file name to 50 or fewer characters. Attachments that do not follow this rule may cause the entire application to be rejected or cause issues during processing.

To avoid interfering with the selection of peer reviewers, please do NOT include general letters of support for the application or list collaborators that do not have an active part in the project. General letters of support will not be used in making funding decisions.

Optional letters of collaboration for unfunded or funded collaborations may be placed in Appendix 6 (Other Attachments). Letters of collaboration should state the intention to participate, but they should not be written as recommendation or endorsement letters, which are not allowed.

If forms identified as mandatory are not submitted with the application package by both the Lead institution and subawardees, the application package will be considered non-responsive during the initial review, will not be forwarded for merit review, and will be eliminated from further consideration. For additional information, see Section V – Application Review Information, Initial Review Criteria, and Application Award Eligibility.

1. APPLICATION FOR FEDERAL ASSISTANCE (SF 424 R&R) [MANDATORY]

For the Prime applicant, you must create a workspace using Grants.gov Workspace (see https://www.grants.gov/help/html/help/Applicants/CreateWorkspace.htm) to complete the workspace online forms and submit your application workspace package. Forms can either be completed online within a web browser or individual PDF forms from within the Forms tab of the Manage Workspace page can be downloaded and uploaded to Workspace when completed (see https://www.grants.gov/help/html/help/ManageWorkspaces/ManageWorkspaces.htm).

For the Prime applicant, in Grants.gov Workspace, Complete SF-424 Fields First: The forms are designed to fill in common required fields across other forms, such as the applicant name, address, and DUNS number. To trigger this feature, an applicant must complete the SF-424 information first. Once it is completed, the information will transfer to the other forms.

NOTE: If sub-applicants will participate in this award, this form is mandatory and must be submitted for all sub-applicants. See Section IV – Application and Submission Information, sub-section C. 8. R&R Sub-award Budget Form below for sub-applicant instructions. Also, see Section IV – Application and Submission Information, sub-section D. Summary of Required Forms and Files, Table 1 - Prime Applicant Mandatory Forms for additional instructions.

2. RESEARCH AND RELATED OTHER PROJECT INFORMATION [MANDATORY]

Complete questions 1 through 6 and attach files. The files must comply with the following instructions:

For the Prime applicant, the Research and Related (R&R) Other Project Information form is part of the Application Package that must be completed in Grants.gov Workspace. The special instructions for completing this form follow:

· Fields 1 to 6.b: Applicant must complete the entries to these fields.

· Field 7: Applicant must attach completed file for Project Summary/Abstract.

· Field 8: Applicant must attach completed file for Project Narrative plus required appendices.

· Field 12: Applicant must attach completed file for Project Management Plan.

· Field 12: Applicant must attach completed National Environmental Policy Act (NEPA) compliance form in addition to the other forms as referenced below.

3. PROJECT SUMMARY/ABSTRACT (FIELD 7 ON THE FORM)

For the Prime applicant, the project summary/abstract must be submitted and must contain a summary of the proposed activity suitable for dissemination to the public. It must be a self-contained document that identifies:

· the name of the applicant,

· the project director/principal investigator(s),

· the project title,

· the type of center: MSC, SDC or FIC,

· the objectives of the project,

· a description of the project, including methods to be employed, the potential impact of the project (i.e., benefits, outcomes), and major participants (for collaborative projects).

This document must not include any proprietary or sensitive business information as the Department may make it available to the public after awards are made. The project summary must not exceed 2 pages for MSCs and SDCs and 1 page for FICs when printed using standard 8.5" by 11" paper with 1" margins (top, bottom, left and right) and single spaced with font size not smaller than 11 point. To attach a Project Summary/Abstract, click "Add Attachment."

The Project Summary/Abstract file must be saved in PDF format and uploaded to the Grants.gov R&R Other Project Information form, which is part of the Application Package in Grants.gov Workspace. To attach the completed Project Summary/Abstract in PDF format, click the "Add Attachment" button next to Field 7 of the R&R Other Project Information form in Grants.gov Workspace.

4. PROJECT NARRATIVE (FIELD 8 ON THE FORM) FOR MSCs AND SDCs ONLY

The project narrative must not exceed 36 pages total, including cover page, table of contents, charts, graphs, maps, photographs, tables, and other pictorial presentations, when printed using standard 8.5" by 11" paper with 1 inch margins (top, bottom, left, and right) and single spaced with font size not smaller than 11 point. The Scientific and Technical Merit section, including required plans detailed below, should comprise the majority of the narrative, with at least 5 pages each should devoted to the predictive science, Exascale, V&V/UQ, integration, and software plans. The combination of the Management, Collaboration, and Education Pans and Proposed Center Resources should not exceed 8 pages total. Suggested combinations for those sections are either 4 pages for the Management, Collaboration, and Education Plan and 4 pages for the Proposed Center Resources; or 5 pages for the Management, Collaboration and Education Plan and 3 pages for the Proposed Center Resources.

EVALUATORS WILL ONLY REVIEW THE NUMBER OF PAGES SPECIFIED IN THE PRECEDING PARAGRAPH. Do not include any Internet addresses (URLs) that provide information necessary to review the application, because the information contained in these sites will not be reviewed. See Section VIII – Other Information – Part D for instructions on how to mark proprietary application information.

Instructions: The Project Narrative Cover Page, Project Narrative, and Appendices must be saved as one single document in PDF format and uploaded to the Grants.gov R&R Other Project Information form, which is part of the Application Package in Grants.gov Workspace. To attach the completed Project Narrative in PDF format, click "Add Attachment" button next to field 8 of the R&R Other Project Information form in Grants.gov Workspace.

COVER PAGE (not to exceed 1 page)

The project narrative cover page should include the following required components:

· The Funding Opportunity Number of the PSAAP III solicitation: DE-FOA-0002068;

· Project title and type of Center (MSC or SDC);

· Name of Lead Institution;

· Name, mailing address, telephone number, fax number, and electronic mail address of Lead Project Director;

· Name, email address, and telephone number of organization’s business point of contact or Grants office point of contact

· Listing of any national laboratory and point of contact collaborating on the project.

PROJECT NARRATIVE (not to exceed 35 total pages)

The project narrative must provide a detailed description of the following:

· Project Objectives: This section should provide a clear, concise statement of the specific objectives/aims of the proposed project.

· Scientific and Technical Merit: This section should address how the project will advance the current state of science and technology in alignment with the Program Objectives and Mission Areas (Topic Research Areas) in Section I. In accordance with the instructions below, the technical narrative must include a well-integrated set of the following plans: one for predictive science, one for V&V/UQ, one for Exascale, one for Integration, and finally one for software execution. All of these plans must be implemented, updated, and maintained during the project period. These plans must include a list of milestones at various intervals, but no less than yearly. Technical risks and mitigation strategies should not be included in this discussion, but should be addressed in the risks section of the management plan (see below).

· Overarching or integrated problem: Applicants applying as Multi-disciplinary Simulation Centers must select and describe in the project narrative an overarching multidiscipline, multiscale application from science/engineering disciplines, which demonstrates advances in, or utilizes state-of-the-art V&V/UQ and drives contributions to the development of effective Exascale computational science. MSCs must propose significant progress on Exascale cross-cutting issues and in multidiscipline predictive capability. In addition, the application must clearly explain how the selected application will benefit from Exascale computing.

Single-Discipline Centers must select an integrated multiscale problem or challenge from a single discipline that utilizes state-of-the-art V&V/UQ and drives contributions to the development of effective Exascale computational science. Centers must propose significant progress on Exascale cross-cutting issues and advances in single discipline predictive science/engineering. In addition, the application must clearly explain how the selected application will benefit from Exascale computing.

The narrative shall include a discussion of how the center will accomplish a full-system prediction in the second year using available NNSA supercomputer(s) even though such full-system prediction may lack full fidelity. (Availability of the NNSA supercomputers is determined during program implementation, based on Center-requested resources and competing ASC priorities). The Narrative shall also address how the Center will add physics and geometric fidelity and other simulation capabilities to the system and use these to demonstrate a full-system prediction in each subsequent year culminating in a well-defined fifth-year prediction. Additionally, the discussion shall explain how priorities for added physics and geometric fidelity are driven by the Center’s V&V/UQ framework.

· Predictive Science Plan: The applicant must include in the narrative a predictive science plan, which the applicant will be required to update and maintain during the project period. The predictive science plan must be integrated with the V&V/UQ plan (see below), and include milestones that tie demonstration of advances in predictive science capability with full system simulations at various intervals (but no less than yearly beginning with the second year). The plan must also show how science/engineering research will lead directly to the required verified and validated full system predictive simulation.

For an MSC, the narrative must address how advances in predictive science will result in better predictive capability, manifested as predictions of a wider range of phenomena, over wider ranges of space- and time-scales, with improved predictive accuracy and reduced uncertainty, or at least better understood uncertainty, in comparison to existing capability at the beginning of the project. The overall advance may require a combination of advances in some Exascale-enabled science, integration science, or UQ science, together with wider use of state-of-the-art techniques.

For an SDC, the narrative must address how advances in predictive science will result in compelling and significant scientific achievements in the single discipline, enabled by Exascale computing. The technical advances should ideally represent a qualitative step up in the discipline, as opposed to incremental progress. While the research efforts within an SDC may follow several divergent paths under the broad umbrella of the single science/engineering discipline, it is required that all research paths be integrated into a single demonstration problem, for which predictions must be produced each year during the life of the program. The demonstration problem is the vehicle by which predictive simulation progress is verified and validated, and uncertainty quantification is assessed each year.

· Verification & Validation/Uncertainty Quantification Plan: The narrative must include a V&V/UQ plan, complete with milestones that tie demonstration of V&V/UQ capability and progress with full system simulations at various intervals (but no less than yearly beginning with the second year). The Center shall be required to maintain and update the plan during the project period. The plan must show how the V&V/UQ capability will lead directly to the required verified and validated predictive simulation. The Center must integrate V&V/UQ processes into the plan to ensure credibility of the predictive simulation. Thus, the application must include a fully integrated V&V/UQ plan beginning in Year 1 that reflects understanding of V&V/UQ issues and plans for addressing them, including (but not limited to) the following:

· Software code reliability

· Numerical accuracy of the simulations

· Any external targets or requirements for accuracy, and the level of required accuracy

· Major sources of uncertainty in simulations

· Quantification and reduction of those uncertainties

· Major sources of uncertainty in validation data

· Required data and sources for that data

· Source, quality and utility of validation/UQ data Validation and verification of large-scale models and simulation codes, coupled across many orders of magnitude of space- and time-scales and compared to experimental studies, is an important undertaking toward achieving predictive capability. Therefore, it is encouraged that the PSAAP III Center’s work be coupled, if possible, to experimental activities in physical sciences and engineering in progress at the DOE/NNSA Laboratories or elsewhere.

· Exascale Plan: The narrative must include a plan to develop and demonstrate technologies and methodologies that will lead to effective Exascale computing in the context of the science or engineering application/problem selected. The plan must show how Exascale-oriented research and development will lead directly to effective use of emerging HPC platforms, and support the advances in predictive science called for by this FOA.