DE-FOA-0003264.000001.pdf

DEPARTMENT OF ENERGY (DOE)

OFFICE OF SCIENCE (SC)

ADVANCED SCIENTIFIC COMPUTING RESEARCH (ASCR)

ADVANCEMENTS IN ARTIFICIAL INTELLIGENCE FOR SCIENCE

FUNDING OPPORTUNITY ANNOUNCEMENT (FOA) NUMBER:

DE-FOA-0003264

FOA TYPE: AMENDMENT 000001

CFDA NUMBER: 81.049

FOA Issue Date: February 13, 2024 Submission Deadline for Pre-Applications: March 19, 2024 at 5:00 PM Eastern Time

A Pre-Application is required.

Pre-Application Response Date: April 11, 2024 at 11:59 PM Eastern Time Submission Deadline for Applications: May 21, 2024 at 11:59 PM Eastern Time

Amendment 000001 was created to include the webinar information on page i.

Table of Contents

SECTION I – FUNDING OPPORTUNITY DESCRIPTION

SECTION II – AWARD INFORMATION

II.A. TYPE OF AWARD INSTRUMENT

II.B. ESTIMATED FUNDING

II.C. MAXIMUM AND MINIMUM AWARD SIZE

II.D. EXPECTED NUMBER OF AWARDS

II.E. ANTICIPATED AWARD SIZE

II.F. PERIOD OF PERFORMANCE

II.G. TYPE OF APPLICATION

SECTION III – ELIGIBILITY INFORMATION

III.A. ELIGIBLE APPLICANTS

III.B. COST SHARING

III.C. ELIGIBLE INDIVIDUALS

III.D. LIMITATIONS ON SUBMISSIONS

SECTION IV – APPLICATION AND SUBMISSION INFORMATION

IV.A. ADDRESS TO REQUEST APPLICATION PACKAGE

IV.C. GRANTS.GOV APPLICATION SUBMISSION AND RECEIPT PROCEDURES

IV.D. CONTENT AND APPLICATION FORMS

IV.E. SUBMISSIONS FROM SUCCESSFUL APPLICANTS

IV.F. SUBMISSION DATES AND TIMES

SECTION V - APPLICATION REVIEW INFORMATION

V.A. CRITERIA

V.B. REVIEW AND SELECTION PROCESS

V.C. ANTICIPATED NOTICE OF SELECTION AND AWARD DATES

SECTION VI – AWARD ADMINISTRATION INFORMATION

VI.A. AWARD NOTICES

VI.B. ADMINISTRATIVE AND NATIONAL POLICY REQUIREMENTS

VI.C. REPORTING

VI.D. REPORTING OF MATTERS RELATED TO RECIPIENT INTEGRITY AND

PERFORMANCE (DECEMBER 2015)

VI.E. INTERIM CONFLICT OF INTEREST POLICY FOR FINANCIAL ASSISTANCE...42

SECTION VII - QUESTIONS/AGENCY CONTACTS

VII.A. QUESTIONS

VII.B. AGENCY CONTACTS

SECTION VIII – SUPPLEMENTARY MATERIAL

VIII.A. HOW-TO GUIDES

VIII.B. ADMINISTRATIVE AND NATIONAL POLICY REQUIREMENTS

VIII.C. REFERENCE MATERIAL

i

UPDATES AND REMINDERS

RECOMMENDATION

The Department of Energy (DOE) Office of Science (SC) encourages you to register in all systems as soon as possible. You are also encouraged to submit letters of intent (LOIs), pre-applications, and applications well before the deadline.

INFORMATIONAL WEBINAR

SC plans to hold an informational webinar about this FOA on Monday, February 26, 2024 at 2:00 PM ET. Registration instructions and other details will be posted at https://science-doe.zoomgov.com/webinar/register/WN_fj05ahYxSuy96Wm2MYr0gg#/registration.

PROMOTING INCLUSIVE AND EQUITABLE RESEARCH (PIER) PLAN

All new and renewal applications must provide a Promoting Inclusive and Equitable Research (PIER) Plan as an appendix to the project narrative. Please read the instructions in Section IV and the associated review criteria in Section V.

INTERAGENCY FORMATS FOR CURRENT AND PENDING SUPPORT AND BIOGRAPHICAL SKETCHES

Interagency common instructions for preparing current and pending support and biographical sketches are being developed. The Science Experts Network Curriculum Vitae (SciENcv) system at https://www.ncbi.nlm.nih.gov/sciencv/ will be updated to support the forthcoming common instructions and formats. The fillable PDFs provided by the National Science Foundation at https://nsf.gov/bfa/dias/policy/nsfapprovedformats/ are no longer available. When interagency common formats and instructions are promulgated, their use will be required. SC strongly encourages all researchers to use the online SciENcv system to ensure that their documents are prepared in the appropriate format with the least inconvenience.

INDIVIDUALS WHO SHOULD NOT SERVE AS MERIT REVIEWERS

Follow the updated instructions in Section VIII and consider the use of the Collaborator Template available at https://science.osti.gov/grants/Policy-and-Guidance/Agreement-Forms. Do not include this list as part of the biographical sketch.

LIVING WAGES

SC is committed to ensuring that students, trainees, and postdoctoral fellows are paid a fair and equitable wage sufficient to allow a reasonable standard of living. Applicant institutions are strongly encouraged to examine their institutional pay scales to ensure that all personnel earn a living wage. The provision of fellowships, traineeships, stipends, honoraria, subsistence allowances, and other similar payments may be allowable expenses on SC financial assistance awards, per 2 CFR 200.430, § 200.431, and § 200.466. For graduate students, SC considers a reasonable living wage to be an annual income of $45,000, excluding benefits.

https://science-doe.zoomgov.com/webinar/register/WN_fj05ahYxSuy96Wm2MYr0gg#/registration https://science-doe.zoomgov.com/webinar/register/WN_fj05ahYxSuy96Wm2MYr0gg#/registration https://www.ncbi.nlm.nih.gov/sciencv/ https://nsf.gov/bfa/dias/policy/nsfapprovedformats/biosketch.pdf https://science.osti.gov/grants/Policy-and-Guidance/Agreement-Forms ii

SC STATEMENT OF COMMITMENT

The DOE SC is fully and unconditionally committed to fostering safe, diverse, equitable, and inclusive work, research, and funding environments that value mutual respect and personal integrity. SC is committed to advancing belonging, accessibility, justice, equity, diversity, and inclusion across the portfolio of activities we sponsor. SC’s effective stewardship and promotion of safe, accessible, diverse, and inclusive workplaces that value and celebrate the diversity of people, ideas, cultures, and educational backgrounds across the country and that foster a sense of belonging in our scientific community is foundational to delivering on our mission. We are committed to promoting people from all backgrounds, including individuals and communities that were historically underrepresented and minoritized in science, technology, engineering, and math (STEM) fields and the activities we sponsor in recognition of our responsibility to serve the public. We also recognize that harnessing a broad range of views, expertise, and experiences drives scientific and technological innovation and enables the SC community to push the frontiers of scientific knowledge for U.S. prosperity and security. Discrimination and harassment undermine SC’s ability to achieve its mission by reducing productivity, discouraging, or inhibiting talent retention and career advancement, and weakening the integrity of the SC enterprise overall. SC does not tolerate discrimination or harassment of any kind, including sexual or non-sexual harassment, bullying, intimidation, violence, threats of violence, retaliation, or other disruptive behavior at institutions receiving SC funding or other locations where activities funded by SC are carried out. All applicants and collaborators should familiarize themselves with the SC Statement of Commitment available at https://science.osti.gov/SW- DEI/SC-Statement-of-Commitment.

UPDATING YOUR PORTFOLIO ANALYSIS AND MANAGEMENT SYSTEM (PAMS) PROFILE

All applicants are encouraged to update their profiles in the PAMS website at https://pamspublic.science.energy.gov regularly, at least annually, to ensure SC has your most up to date information. The PAMS profile now requires that individuals provide responses to the demographic related fields. SC strongly encourages personnel at applicant and recipient institutions, including Principal Investigators (PIs), Co-PIs, and other Key Personnel, to provide their demographic information. By providing your demographic information, you are assisting with SC’s continued commitment to advancing diversity, equity, and inclusion in its business practices. Alternatively, for information you wish not to disclose, please select, “Do not wish to provide.” Your individual demographic information will not be shared with peer reviewers and the information in your PAMS profile is protected by the requirements established in the Federal Privacy Act of 1974. Aggregate, anonymized demographic information may be shared with confidential review committees who are charged with evaluating the quality and efficacy of SC’s business practices. For example, summary statistics of all applicants to award selections from a particular SC FOA may be reviewed by a Committee of Visitors.

PORTABLE DOCUMENT FORMAT (PDF) GENERATION

The project narrative in an application must be one single machine-readable PDF file that contains the DOE Title Page, project narrative, all required appendices, and other attachments.

https://science.osti.gov/SW-DEI/SC-Statement-of-Commitment https://science.osti.gov/SW-DEI/SC-Statement-of-Commitment https://pamspublic.science.energy.gov/ iii

This single PDF file may not be scanned from a printed document and must be attached in Field 8 on the Grants.gov form. This must be a plain PDF file consisting of text, numbers, and images without editable fields, signatures, passwords, redactions, or other advanced features available in some PDF-compatible software. The project narrative will be read by SC staff using the full version of Adobe Acrobat: Please ensure that the narrative is readable in Acrobat. If combining multiple files into one research narrative, ensure that a PDF portfolio or binder is not created. If creating PDF files using any software other than Adobe Acrobat, please use a “Print to PDF” or equivalent process to ensure that all content is visible in the project narrative. Once a project narrative has been assembled, please submit the combined project narrative file through a “Print to PDF” or equivalent process to ensure that all content is visible in one PDF file that can be viewed in Adobe Acrobat. Review your submission to ensure that blank pages are not present.

https://www.grants.gov/ iv

Checklist for Avoiding Common Errors:

Item Issue Applications Submitted in Grants.gov. Do not submit applications in PAMS or FedConnect.

Grants.gov Submission Ensure that applications are submitted under the correct Opportunity Number.

Standard Form (SF)-424 Research and Related (R&R):

- Attach nothing to field 20

- Attach nothing to field 21

SF-424 Research and Related Other Project Information form:

- Attach the abstract to field 7

- Attach the project narrative, with all appendices, to field 8

- Attach nothing to field 9

- Attach nothing to field 10

- Attach nothing to field 11

- Attach the list of individuals who should not serve as merit reviewers (Collaborator Template) to field 12

- Do not attach other files to field 12

- NOTE: Files attached to field 12 will not be shared with merit reviewers.

Pre-Applications

- Submit your pre-application in PAMS.

- Do not submit your pre-application in

Grants.gov.

- Do not attach your pre-application to the

SF-424 Research and Related (R&R) form.

- Follow the instructions in Section VIII for the preparation of a pre-application.

Page Limits Strictly followed throughout application, including particular attention to:

- Project narrative and appendices

- Biographical sketches

- Data Management Plans (DMPs)

- Promoting Inclusive and Equitable

Research (PIER) Plan

- Letter(s) of Collaboration or Access, if any

Personally Identifiable Information None present in the application Project Narrative Composed of one PDF file including all appendices (bibliography, facilities, equipment, DMP, PIER) v

Item Issue Project Summary / Abstract Name(s) of applicant, PI(s), PI’s institutional affiliation(s), Co-Investigator(s), Co- Investigator’s institutional affiliation(s)

DOE Title Page Follow instructions closely Budget Use current negotiated indirect cost and fringe benefit rates Budget Justification (attached to budget) Justify all requested costs Biographical Sketches Follow page limits strictly and do not include list of collaborators. Attach the biographical sketch to the Senior/Key Person Profile (Expanded) Form.

Current and Pending Support Ensure complete listing of all activities, regardless of source of funding. Attach the current and pending support to the Senior/Key Person Profile (Expanded) Form.

List of Individuals who Should not Serve as Merit Reviews

Attach to field 12 of the SF-424 Research and Related Other Project Information form.

Data Management Plans (DMP) - If referring to an experiment’s DMP, describe the relationship to the proposed research.

- Include a DMP even if no experimental data is expected.

Promoting Inclusive and Equitable Research (PIER) Plan

PIER Plans are required for new and renewal applications.

Institutions capable of being funded through the DOE Field Work System

If National Laboratories and/or DOE sites are permitted to submit under this FOA:

- Do not create new institutions in the PAMS website.

- Submit applications in Grants.gov using the name of the laboratory or site in Field 5 of the SF-424(R&R) application form, not the contractor operating the laboratory or site.

Submissions under this FOA will be evaluated for technical merit, but any resulting funding, work, or awards will be made under the laboratory or site’s contract with DOE. No separate financial assistance awards will be made. No administrative provisions of this FOA will apply to the laboratory or any laboratory subcontractor.

Section I – FUNDING OPPORTUNITY DESCRIPTION

GENERAL INQUIRIES ABOUT THIS FOA SHOULD BE DIRECTED TO:

Technical/Scientific Program Contacts:

Dr. Hal Finkel [Primary] hal.finkel@science.doe.gov

Dr. Steven Lee steven.lee@science.doe.gov

Dr. Margaret Lentz margaret.lentz@science.doe.gov

Dr. Kalyan Perumalla kalyan.perumalla@science.doe.gov

Dr. Robinson Pino robinson.pino@science.doe.gov

Dr. William Spotz william.spotz@science.doe.gov

STATUTORY AUTHORITY

Section 646 of Public Law 95-91, U.S. Department of Energy Organization Act Section 901, et seq. of Public Law 109-58, Energy Policy Act of 2005

APPLICABLE REGULATIONS

Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal

Awards, codified at 2 CFR 200 U.S. Department of Energy Financial Assistance Rules, codified at 2 CFR 910 U.S. Department of Energy, Office of Science Financial Assistance Program Rule, codified at 10

CFR 605

SUMMARY

The DOE SC program in Advanced Scientific Computing Research (ASCR) hereby announces its interest in basic computer science and applied mathematics research in the fundamentals of Artificial Intelligence (AI) for science. Specifically, advancements in this area are sought that can enable the development of:

• Foundation models for computational science;

• Automated scientific workflows and laboratories;

• Scientific programming and scientific-knowledge-management systems;

mailto:hal.finkel@science.doe.gov mailto:steven.lee@science.doe.gov mailto:margaret.lentz@science.doe.gov mailto:kalyan.perumalla@science.doe.gov mailto:robinson.pino@science.doe.gov mailto:william.spotz@science.doe.gov

• Federated and privacy-preserving training for foundation and other AI models for science; and

• Energy-efficient AI algorithms and hardware for science.

The development of new AI techniques applicable to multiple scientific domains can accelerate progress, increase transparency, and open new areas of exploration across the scientific enterprise.

SUPPLEMENTARY INFORMATION

AI is one of the most powerful technologies of our time1 and DOE is at the forefront of research and development in AI technologies for enabling scientific discovery and innovation. Core components of the scientific method remain unchanged: Observation, Hypothesis, Experiments, and Analysis. However, DOE recognizes that abundant sources of data, high-performance computing (HPC) and networking, energy-efficient algorithms, and AI-related technologies can be harnessed to significantly accelerate and expand the impact of scientific research. The breadth of applications spans climate science, cybersecurity and electric grid resilience, biotechnology, microelectronics, disaster response, and beyond. Research to address national priorities will require advances and AI innovations in high-level capabilities such as: monitoring and predicting the onset of real-world anomalies and extreme events; adaptive strategies to control the real-time behavior of complex systems, infrastructure, and processes; approaches for the optimal development and design of physical systems; decision-support for planning, risk, and policy formulation; and tools that synthesize scientific knowledge and accelerate the design, manufacturing, testing, and optimization of new technologies. The focus of ASCR research and development investments is on the underlying approaches for AI-enhanced scientific and engineering capabilities and to significantly transform the scientific method for accelerated discovery and innovation.

Realizing the next generation of AI for science will require innovations in both hardware and algorithms. Future AI-enabled scientific workflows are expected to use Machine Learning (ML) to enhance numerical modeling and data analysis along with technologies that process natural and computer-programming languages. DOE’s exascale supercomputers2 are some of the Nation’s most powerful systems for large-scale AI training and for tasks integrating AI, modeling, simulation, and data analysis. These exascale and future systems complement the vast array of other AI-enabled HPC and edge systems, including automated laboratories and facilities, that will significantly accelerate scientific progress in the coming decades.

DOE’s scientific community has collectively articulated important research directions toward

1 For additional background on the promise and importance of AI R&D, see the OMB/OSTP Memorandum on Multi-Agency Research and Development Priorities for the FY 2025 Budget (August 2023) https://www.whitehouse.gov/wp-content/uploads/2023/08/FY2025-OMB-OSTP-RD-Budget-Priorities-Memo.pdf, and the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 2023) https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ 2 For more information on ASCR’s exascale supercomputers, and other HPC resources, available as national user facilities, see https://science.osti.gov/ascr/Facilities/User-Facilities https://www.whitehouse.gov/wp-content/uploads/2023/08/FY2025-OMB-OSTP-RD-Budget-Priorities-Memo.pdf https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ https://science.osti.gov/ascr/Facilities/User-Facilities realizing the promise of AI for science and other DOE missions in the recently-released AI For Science, Energy, and Security report [1], building on the preceding AI for Science report [2], and complementing the report on Opportunities and Challenges from Artificial Intelligence and Machine Learning for the Advancement of Science, Technology, and the Office of Science Missions [3]. The research directions highlighted in these reports, and others, appear prominently in the National Artificial Intelligence Research and Development Strategic Plan [4]. This FOA addresses a broad spectrum of research priorities described in these documents that are critical to enabling trustworthy AI for scientific applications advancing human understanding and addressing national needs.

Research Areas

Each pre-application and application, as described in Section IV.B.2 and Section IV.D.2 respectively, must identify one primary research area.

Research Area 1: Extreme-Scale Foundation Models for Computational Science

Technical Contacts: All FOA technical contacts.

ASCR sees an opportunity to capitalize on the rapidly-advancing foundation-model techniques, combined with recently-deployed exascale computing resources, to expeditiously jumpstart the impact of foundation models on computational science in order to significantly advance the state of the art in computational science. Accordingly, this research area seeks the creation of a foundation model, or multiple foundation models, for computational science, covering broad areas in computer science and applied mathematics. A foundation model is trained on broad data, generally uses self-supervision for training, and is applicable across a wide range of contexts.

While many well-known foundation models are Large Language Models (LLMs) processing textual data, foundation models can also be created for other kinds of data, or moreover, can be multimodal in nature (processing multiple kinds of data). For the purposes of this research area, a foundation model is expected to have at least tens of billions of parameters. It is expected that the foundation model, or foundation models, will explore innovative next-generation capabilities such as flexible multimodality, advanced external tool integration, sophisticated reasoning and planning, and robust memory. Moreover, given the importance of accuracy and quantified uncertainty in science, the research must address how these factors will be assessed. The proposal must address how the training process, or processes, will minimize risks associated with model misuse, unintentional biases or inaccuracies in model outputs, and other potential undesirable consequences [5]. This should also include addressing the corresponding considerations for data employed in the training process, or processes. For additional background, see Chapter 2, “AI Foundation Models for Scientific Knowledge Discovery, Integration, and Synthesis,” Chapter 12, “Mathematics and Foundations,” and Chapter 14, “Data Ecosystem,” of the AI For Science, Energy, and Security report [1].

ASCR’s exascale supercomputers, Frontier and Aurora, each contain tens of thousands of state-of-the-art graphics processing units (GPUs) and rank among the world’s most-powerful resources for AI model training. Accordingly, proposed efforts are encouraged to make use of ASCR’s exascale resources to create a foundation model, or foundation models, that take advantage of extreme-scale computing to advance the state of the art in foundation-model creation. Note that this does not require that the foundation model, or foundation models, have more parameters than, or use more data than, all other known foundation models – rather, each model is expected to be of sufficient size and trained using sufficient data to achieve the goal to significantly advance the state of the art in capabilities for computational science.

For this research area only, a pre-application in response to this FOA may, at the applicant’s discretion, also include an articulation of likely HPC resource requirements as might be requested from the ASCR Leadership Computing Challenge (ALCC) allocation program3 comprising: the quantity of physical node hours required (not multiplied by any site charging factor) and a detailed summary of the software to be used, the planned production runs, the number of nodes per run, the expected wall clock time required, and how benchmarking was determined4. ASCR is seeking projects that will be ready to use HPC resources starting in 2024 on Frontier, Polaris, Perlmutter-GPU, and/or Aurora5. Following the pre-application review process described in Section IV.B.2, should ASCR encourage submission of an application in response to this FOA, ASCR will also send instructions to the applicant on how to submit a companion application to the ALCC program for the allocation request. The companion ALCC application must meet the content requirements for ALCC program6 but will be due on the same day as the application in response to this FOA. Merit reviewers evaluating the application submitted in response to this FOA will also be provided with, and may also be asked to evaluate, the companion ALCC application. ASCR anticipates providing selection notifications for research proposed in response to this FOA along with selection notifications for any companion ALCC applications. Note that successful applicants will be responsible for submitting any necessary pre-applications and applications to the ALCC program, following the usual procedure for such pre-applications and applications specified on the ALCC website, for the 2025 allocation cycle and allocation cycles thereafter.

Research Area 2: AI Innovations for Scientific Knowledge Synthesis and Software Development

Technical Contacts: Margaret Lentz, Margaret.Lentz@science.doe.gov; Kalyan Perumalla, Kalyan.Perumalla@science.doe.gov; Hal Finkel, Hal.Finkel@science.doe.gov.

The state-of-the-art in knowledge synthesis and programming tools are changing rapidly, fueled by AI Large Language Models (LLMs) trained on text, source code, and other data sources. New AI-driven tools are currently not trustworthy; do not systematically understand mathematical and

3 For more information on the ASCR Leadership Computing Challenge (ALCC) program, see https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC 4 These required elements are intended to mirror the required narrative elements in ALCC pre-proposals, see https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC/Pre-Proposal-Information 5 Project allocations on ALCF’s new exascale-system, Aurora, will begin on January 1, 2025, and will run through the end of the ALCC cycle, June 30, 2025. ASCR is seeking projects that will be ready to use HPC resources starting in 2024, and as a result, expects that requests for resources on Aurora will be accompanied by requests for resources on one or more other machines.

6 See https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC/Proposal-Information for a description of the required elements in an ALCC proposal.

mailto:Margaret.Lentz@science.doe.gov mailto:Kalyan.Perumalla@science.doe.gov mailto:Hal.Finkel@science.doe.gov https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC/Pre-Proposal-Information https://science.osti.gov/ascr/Facilities/Accessing-ASCR-Facilities/ALCC/Proposal-Information physical principles; cannot properly ingest and understand scientific literature and data; and do not produce consistent, verified, uncertainty-quantified, reproducible results. In addition to addressing those challenges, there may be particular advantages in such tools building up knowledge and context over many interactions with a user or group of users. However, incremental training of AI systems over long time horizons, and the representation of knowledge in AI systems robust to changes in the underlying AI models, remain critical challenges.

This research area seeks fundamental advancements in knowledge synthesis and programming tools for science. Moreover, realizing AI systems that can truly understand, and assist with, all aspects of the scientific process requires innovation in many areas, including multimodality, tool use, deeper reasoning and planning, memory, and external interaction. For additional background, see Chapter 2, “AI Foundation Models for Scientific Knowledge Discovery, Integration, and Synthesis,” Chapter 6, “AI for Programming and Software Engineering,” Chapter 12, “Mathematics and Foundations,” and Chapter 14, “Data Ecosystem,” of the AI For Science, Energy, and Security report [1].

Additionally, investigations into AI-driven tools for science should be conceptualized accounting for the iterative and collaborative processes that define modern science and scientific-software development. Accordingly, research proposed in this area is encouraged to address the relevant Priority Research Directions (PRDs) from the Basic Research Needs in The Science of Scientific Software Development and Use report [6], which are PRD 1, “Develop next-generation tools to enhance developer productivity and software sustainability,” PRD 2, “Develop methodologies and tools to comprehensively improve team-based scientific software development and use,” and PRD 3, “Develop methodologies, tools, and infrastructure for trustworthy software-intensive science.”

Methods proposed for investigation should use any appropriate techniques that might be necessary to accomplish their goals, including, but not limited to, machine learning, natural-language processing, formal reasoning, instrumentation, data management, and compiler technology. The sustainability and explainability of scientific software are critically important to the scientific process, and as a result, particular consideration should be given to maximizing the extent to which human programmers understand and/or trust the outputs of these methods.

Research Area 3: AI Innovations for Computational Decision Support of Complex Systems

Technical Contact: William Spotz, William.Spotz@science.doe.gov; Steven Lee, Steven.Lee@science.doe.gov.

Scientific machine learning is a core component of artificial intelligence and a computational technology that can be trained, with scientific data, to augment or automate human skills. The principal focus of this research area is on the use of scientific AI/ML for intelligent automation and decision support for complex systems. This area is defined by PRD 6, “Intelligent Automation and Decision Support” of the Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence report [7]. Foundational research, as described in PRDs 1, 2, and 3 from that report, will be needed for strengthening the mathematical and statistical basis in developing predictive AI/ML-based computational models mailto:William.Spotz@science.doe.gov mailto:Steven.Lee@science.doe.gov and adaptive algorithms for scientific advances. Also, new techniques, software tools, and approaches will likely be needed to reap scientific benefits from the extreme heterogeneity of scientific computing technologies (e.g., processors, memory and interconnect systems, sensors) that are emerging. For additional background, see Chapter 4, “AI-Based Design, Prediction, and Control of Complex Engineered Systems,” Chapter 12, “Mathematics and Foundations”, and Chapter 13, “AI Workflows (Edge, Center, Cloud),” of the AI For Science, Energy, and Security report [1], and other reports highlighting challenges and opportunities associated with scientific decision support, autonomous systems, and digital twins [8-10].

Scientific computing within DOE traditionally has been dominated by complex, resource-intensive numerical simulations. However, the rise of data-driven scientific AI/ML models and algorithms provides exciting opportunities. The term “outer loop” is used increasingly to describe computational applications that form outer loops around a forward simulation.

Examples of outer-loop applications include optimization, uncertainty quantification, inverse problems, data assimilation, and control. Examples of complex systems of interest to the DOE abound, and while it is anticipated that proposed projects will focus on a specific complex system, this research area seeks to advance applied mathematical techniques with general applicability. Applicants must demonstrate that their proposed work is not specific to a particular complex system. It is expected that the proposed projects will significantly benefit from the exploration of innovative ideas or from the development of unconventional approaches.

Proposed approaches may include innovative research with one or more key characteristics, such as asynchronous computations, mixed-precision arithmetic, automatic differentiation, compressed sensing, coupling frameworks, graph and network algorithms, randomization, Monte Carlo or Bayesian methods, probabilistic programming, or other relevant facets.

Research Area 4: Federated and Privacy-Preserving Machine Learning and Synthetic Data Creation

Technical Contacts: Steven Lee, Steven.Lee@science.doe.gov; Kalyan Perumalla, Kalyan.Perumalla@science.doe.gov; Margaret Lentz, Margaret.Lentz@science.doe.gov.

Key challenges in training foundation and other AI models for science, are created by the reality that the data available for training is often sparse and distributed across different computing systems. Sometimes, when there is too little data for training, or the underlying data cannot be used or shared, more-voluminous synthetic data can be created for AI training. Alternatively, there may be sufficient data, but it might be too voluminous to be transferred to a common system for training, or it might be restricted from being aggregated to protect privacy-sensitive or proprietary data, thus requiring federated AI training techniques. The focus of this research area is on federated learning, potentially combined with privacy-preserving techniques, and synthetic data creation to address the aforementioned challenges.

Basic research challenges in federated approaches arise in dealing with the heterogeneous characteristics of edge devices (memory capacity, computing power, network connectivity) and local data (skewed data samples, different modalities such as images, video, text). Another major challenge is the need for improved energy efficiency, scalability, and performance in the federated training process. An additional challenge area is the need for incorporating more robust mailto:Steven.Lee@science.doe.gov mailto:Kalyan.Perumalla@science.doe.gov mailto:Margaret.Lentz@science.doe.gov communication patterns and update strategies from the multiple sources into the global shared model. For additional background, see the AI For Science, Energy, and Security report [1] and the National Strategy to Advance Privacy-Preserving Data Sharing and Analytics [11].

For this research area, the focus for federated learning is on the role and innovative use of federated approaches to train and fine-tune foundation and other AI models for science on massive amounts of multi-modal, distributed, and potentially privacy-sensitive datasets. The emphasis for the above and related challenges is on mathematical, statistical, and scientific computing research and development. Significant advances will be needed in asynchronous, randomized, mixed-precision, and other algorithmic developments, optimization, and numerical analyses that underpin the overall process of building and fine-tuning trustworthy foundation and other AI models for science.

Additionally, challenges relevant to the creation and validation of foundation and other AI models for science might be addressed by innovative methods for the creation of synthetic data sets. Due to the special characteristics of scientific data, novel approaches are needed to address issues such as access, sharing, versioning, control, and evolution of large, verified and/or validated datasets. Moreover, research may investigate new tradeoffs between computation, communication, and storage, such as in terms of algorithmic entropy and overall usage costs, by exploiting the potential for deterministic regeneration of synthetic data via on-demand re-computation.

Research Area 5: The Co-Design of Energy-Efficient AI Algorithms and Hardware Architectures

Technical Contacts: Robinson Pino, Robinson.Pino@science.doe.gov; Kalyan Perumalla, Kalyan.Perumalla@science.doe.gov; Hal Finkel, Hal.Finkel@science.doe.gov.

This research area seeks innovative approaches to energy-efficient scientific AI, including corresponding mathematical paradigms and modeling capabilities capable of predicting the resource efficiency of AI systems, potentially up to the largest scales. Energy-efficient AI technologies, covering both training and inference, and including current technologies as a point of comparison, should be addressed holistically, accounting for energy for parameter representation, model size, and other representational factors, plus computational energy for activations and other inherent operations in the neural networks. Modeling capabilities to address energy efficiency should include both the ability to estimate resource usage relatively cheaply and the ability to perform detailed simulations of next-generation AI systems at extreme scales.

Specific scientific AI use cases must be identified and used to guide the exploration of algorithmic and hardware innovations. The results of research in this area should enable fundamental advances in energy-efficient AI algorithms and also provide knowledge useful for the design of next-generation AI systems for science.

It seems useful to note that, in many respects, biological neurons are significantly more efficient than present-day computational approaches, and it has long been recognized that one factor that is likely key to this energy-efficiency gap is the inherent use of sparsity of connectivity and mailto:Robinson.Pino@science.doe.gov mailto:Kalyan.Perumalla@science.doe.gov mailto:Hal.Finkel@science.doe.gov timing by biological neurons: neurons in the brain are “spiking” and transmit data asynchronously on an as-needed basis to a limited neighborhood. While microelectronics for Spiking Neural Networks (SNNs) have been created, an approach often referred to as neuromorphic computing, and considerable promise exists for scientific applications [12], the practical energy efficiency and scale of AI systems built using such hardware is limited by gaps in our fundamental understanding of how to structure the architectures of SNN implementations, how to encode data in SNNs, and in many cases, how to train SNNs in situ (meaning training on the SNN hardware itself instead of converting for execution on SNN hardware some other kind of neural network trained on some other kind of hardware).

SNNs are not the only kind of AI approach promising significant increases in energy efficiency over modern AI systems. Hardware and algorithms, for example, neuromorphic, utilizing statistical, probabilistic, approximate, sub-threshold, photonic, cryogenic, and other hardware and software approaches remain the subject of important investigations. While, for some of these approaches, our gaps in fundamental algorithmic knowledge might be qualitatively less significant than for SNNs, there remain important unknowns. Moreover, when considering next-generation HPC and AI systems, multiple kinds of technologies could contribute to the overall system implementation. For additional background, see the Reimagining Codesign for Advanced Scientific Computing report [13]; the Neuromorphic Computing, Architectures, Models, and Applications: A Beyond-CMOS Approach to Future Computing report [14]; and Chapter 15, “AI- Oriented Hardware Architectures,” of the AI For Science, Energy, and Security report [1].

Out of Scope

Out of scope are pre-applications and applications that:

• Fail to focus on one of the research areas described above and indicate that research area as described in Section IV.B.2 and Section IV.D.2;

• Fail to focus on fundamental advances in AI for science;

• Fail to describe the kinds of scientific-computing workloads targeted by the proposed investigations;

• Provide discipline-specific and/or application-specific solutions that do not generalize to multiple applications; or

• Focus on the development of applications or approaches for quantum computers.

Out-of-scope pre-applications and applications may be declined without further review.

Annual Meetings

The selected awardees will be expected to attend one annual PI meeting. Applicants should anticipate a need for travel to effectively communicate with other researchers and request appropriate funding in their budgets.

References

1. Advanced Research Directions on AI for Science, Energy, and Security Report. May 2023.

https://www.anl.gov/ai-for-science-report https://www.anl.gov/ai-for-science-report

2. AI for Science Report 2020. February 2020. https://www.anl.gov/cels/reference/ai-for-science-report-2020

3. Opportunities and Challenges from Artificial Intelligence and Machine Learning for the Advancement of Science, Technology, and the Office of Science Missions. September 2020.

https://www.osti.gov/biblio/1734848

4. National Artificial Intelligence Research and Development Strategic Plan 2023 Update. May 2023. https://www.nitrd.gov/pubs/National-Artificial-Intelligence-Research-and- Development-Strategic-Plan-2023-Update.pdf

5. Skating to Where the Puck Is Going: Anticipating and Managing Risks from Frontier AI Systems. October 2023. https://cset.georgetown.edu/publication/skating-to-where-the-puck-is-going

6. Basic Research Needs in The Science of Scientific Software Development and Use:

Investment in Software is Investment in Science. August 2023.

https://www.osti.gov/biblio/1846009

7. Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence. February 2019. https://www.osti.gov/biblio/1478744

8. Automated Research Workflows for Accelerated Discovery: Closing the Knowledge Discovery Loop. 2022. https://doi.org/10.17226/26532

9. Autonomous Discovery in Science and Engineering. August 2021.

https://www.osti.gov/biblio/1818491

10. Foundational Research Gaps and Future Directions for Digital Twins. 2023.

https://www.nationalacademies.org/our-work/foundational-research-gaps-and-future-directions-for-digital-twins

11. National Strategy to Advance Privacy-Preserving Data Sharing and Analytics. March 2023.

https://www.whitehouse.gov/wp-content/uploads/2023/03/National-Strategy-to-Advance- Privacy-Preserving-Data-Sharing-and-Analytics.pdf

12. Neuromorphic Computing for Scientific Applications. November 2022.

https://www.osti.gov/biblio/1928928

13. Reimagining Codesign for Advanced Scientific Computing. April 2022.

https://www.osti.gov/biblio/1822199

14. Neuromorphic Computing, Architectures, Models, and Applications. A Beyond-CMOS Approach to Future Computing. December 2016. https://www.osti.gov/biblio/1341738

Open Science and Data Management

SC is dedicated to promoting the values of openness in Federally supported scientific research, including, but not limited to, ensuring that research may be reproduced and that the results of Federally supported research are made available to other researchers. These objectives may be met through any number of mechanisms including, but not limited to, data access plans, data sharing agreements, the use of archives and repositories, and the use of various licensing schemes.

Guidance for Application Budgets and Costs

All costs requested in a budget must adhere to standard requirements for all Federal awards:

• Costs must be reasonable, using a prudent-person standard. (2 CFR 200.404), https://www.anl.gov/cels/reference/ai-for-science-report-2020 https://www.anl.gov/cels/reference/ai-for-science-report-2020 https://www.osti.gov/biblio/1734848 https://www.nitrd.gov/pubs/National-Artificial-Intelligence-Research-and-Development-Strategic-Plan-2023-Update.pdf https://www.nitrd.gov/pubs/National-Artificial-Intelligence-Research-and-Development-Strategic-Plan-2023-Update.pdf https://cset.georgetown.edu/publication/skating-to-where-the-puck-is-going https://cset.georgetown.edu/publication/skating-to-where-the-puck-is-going https://www.osti.gov/biblio/1846009 https://www.osti.gov/biblio/1478744 https://doi.org/10.17226/26532 https://www.osti.gov/biblio/1818491 https://www.nationalacademies.org/our-work/foundational-research-gaps-and-future-directions-for-digital-twins https://www.nationalacademies.org/our-work/foundational-research-gaps-and-future-directions-for-digital-twins https://www.whitehouse.gov/wp-content/uploads/2023/03/National-Strategy-to-Advance-Privacy-Preserving-Data-Sharing-and-Analytics.pdf https://www.whitehouse.gov/wp-content/uploads/2023/03/National-Strategy-to-Advance-Privacy-Preserving-Data-Sharing-and-Analytics.pdf https://www.osti.gov/biblio/1928928 https://www.osti.gov/biblio/1822199 https://www.osti.gov/biblio/1341738

• Costs must be allocable, related to the particular Federal award. (2 CFR 200.405),

• Costs must be allowable under the relevant Federal cost principles. (See 2 CFR 200.420 and following), and

• Costs must be consistently treated, whether they are paid for with Federal funds or institutional funds. (2 CFR 200.403(c)).

Allowable costs may include the following, subject to the applicable cost principles:

• “Buying out” faculty time dedicated to teaching or administrative responsibilities,

• Support for administrative personnel dedicated to the proposed activity,

• Support for professional development, training, mentoring of students and junior researchers,

• Travel to meet with collaborators at other institutions and relevant DOE/NNSA national laboratories, including costs for internships at the national laboratories; or to attend one or more science team, user facility, scientific conference, workshop, or professional society meetings relevant to the proposed research; or for the conduct of off-site research,

• Fringe benefits, which must be paid in accordance with an institution’s negotiated rates agreement, institutional policies, and the individual’s appointment,

• Temporary dependent-care costs incurred during travel,

• Membership costs in relevant professional societies, including both scientific societies and those dedicated to research administration,

• Instrumentation required to conduct proposed research,

• Equipment (items with a useful life of more than 12 months and a per-item acquisition cost of more than $5,000) required to conduct proposed research,

• Purchase of equipment, modification of equipment, provide services necessary to enable work to be carried out by project personnel with a disability,

• Stipends and benefits for students and post-doctoral researchers, recognizing their dual nature as both trainees and employees,

• Participation in standards development relevant to the proposed research, including travel and membership costs,

• Salary support to cover time to participate in outreach for recruitment, internships, and training events, science team meetings, partnership development, or information gathering, and

• Other direct costs, e.g., materials and supplies such as office supplies, desktop or laptop computer, and/or software licenses that are directly necessary to enable the proposed activities.

Multi-Institutional Teams

SC uses two different mechanisms to support teams of multiple institutions.

COLLABORATIVE APPLICATIONS

Teams of multiple institutions may submit collaborative applications. Each submitted application in such a team must indicate that it is part of a collaborative project/group. Every partner institution must submit an application through its own sponsored research office. Each multi-institutional team can have only one lead institution. Each application within the multi-institutional team, including the narrative, starting with the title page, and all required appendices and attachments, must be identical with the following exceptions:

• Each application must contain a correct SF-424 (R&R)7 cover page for the submitting institution only.

• Each application must contain a unique budget corresponding to the expenditures for that application’s submitting institution only.

• Each application must contain a unique budget justification corresponding to the expenditures for that application’s submitting institution only.

• Each application must contain a Project/Performance Site Location(s) form for the submitting institution and its subawards.

• Each application must include a list for the Identification of Merit Review Conflicts for the submitting institution’s senior/key personnel and its subawards’ senior/key personnel.

• Each application must include a Research and Related Senior/Key Person Profile (Expanded) form with the biographical sketches and current and pending support from that institution’s senior/key personnel. The applicant leading the multi-institutional team must include biographical sketches and current and pending support from each institution’s senior/key personnel.

Our intent is to create from the various applications associated with a multi-institutional team one document for merit review that consists of the common, identical materials combined with a set of detailed budgets from the partner institutions. Each team member’s application must contain the same project title. Team members may use Grants.gov Workspace(s), webforms, and their system-to-system services in any combination.

SUBAWARDS8

Multi-institutional teams may submit one application from a designated lead institution with all other team members proposed as subrecipients. Institutions submitting applications as part of a collaborative project/group are also permitted to propose subrecipients.

DOE/National Nuclear Security Administration (NNSA) National Laboratories9, other Federal agencies, and another Federal agency’s FFRDCs10, if participating in a team led by another institution, must use the collaborative application process described above and may not be proposed as subrecipients when requesting funding more than or equal to the applicable floor specified in Section II.C. Note that the value of any such proposed subaward may be removed from any such prime award: DOE may make separate awards to Federally affiliated institutions.

These organizations are usually funded directly by DOE and therefore must submit their own

7 The Standard Form 424 (SF-424) family of forms is used to apply for Federal financial assistance through https://www.Grants.gov. The Research and Related (R&R) set of forms is used by the Office of Science.

8 Subawards are made to subrecipients. Both terms are defined in 2 CFR 200.1 (https://www.ecfr.gov) 9 The phrase “National Laboratories” is used broadly to encompass DOE/NNSA laboratories and sites capable of performing the work described in this FOA and capable of receiving funds through the DOE Field Work System.

10 An authoritative list of all Federally Funded Research and Development Centers (FFRDCs) may be found at https://www.nsf.gov/statistics/ffrdclist/ https://www.grants.gov/ https://www.ecfr.gov/ https://www.nsf.gov/statistics/ffrdclist/ applications as collaborative applications when requesting funding more than or equal to the applicable floor specified in…