Amendment 0002_BAA-NOAA-KMNLP-2024_7June2024_FINAL.pdf

A Study to Determine Natural Language Processing Capabilities with the NCCF Open Knowledge Mesh

(KM/NLP)

Broad Agency Announcement

Amendment 0002, BAA-NOAA-KM/NLP-2024

June 7 May 30, 2024

Office of System Architecture & Advanced Planning (OSAAP)

National Environmental Satellite, Data, and Information Service (NESDIS)

National Oceanic and Atmospheric Administration (NOAA)

Department of Commerce (DOC)

Table of Contents

1. OVERVIEW 4

GENERAL INFORMATION 4

NAICS Code 4

Agency Name 4

Development Opportunity Title 4

Announcement Type and Date 5

Proposals Due Date 5

Eligible Offerors 5

FFRDCs 5

Government Entities 5

Program Details and Areas of Interest 5

Award Information 5

2. RESEARCH AND DEVELOPMENT TECHNICAL DESCRIPTION 5

Background and Overview: 5

Illustrative Example 6

Domain Assessment 10

Solution Intent 10

Provisions and Expectations 11

Scope and Objective 12

General Guidance 12

Scope Intent 13

Technologies Under Test 13

Study Options 14

Planning Considerations 18

Solution Constraints 19

Expected Outcomes 19

Technical Requirements 20

What Does Success Look Like? 22

Agile Implementation Details and Coordination with Government Team 24

Coordination Requirements 25

Communications Requirements 25

Period of Performance 26

Deliverables 26

Reporting Content 29

3. PROPOSAL GUIDELINES 31

3.1 ELIGIBILITY INFORMATION 31

3.2 SUBMISSION PREPARATION 32

3.2.1 Quad Chart Guidelines 33

3.2.2 Proposals Guidelines 34

3.3 EVAULATION INFORMATION 36

4. AGENCY CONTACTS 37

Appendices 38

Appendix A 38

Appendix B 44

1. OVERVIEW

This is a Broad Agency Announcement (BAA) posted in accordance with Federal Acquisition Regulation (FAR) Subpart 35.016. The United States Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), National Environmental and Satellite, Data, and Information Service (NESDIS), Office of Systems Architecture and

Engineering (SAE) is requesting the generation, assessment, and delivery to NOAA, of innovative and meritorious studies that examine, deliver, and iteratively demonstrate independently validatable, and working models, features, and capabilities related to the development and management of an enterprise information processing system backed by an interoperable knowledge graph ecosystem (‘knowledge mesh’) of process-contextualized, semantically enabled facts, exposed as a unified access interface (envisioned as foundation model), that is capable of dynamically leveraging APIs using the knowledge mesh capability for the purpose of answering highly contextual queries from a number of personas representing a diverse swath of NOAA users. This BAA is exploratory only and currently there is no planning for activity beyond a study and demonstration.

This announcement shall be posted for a total of twenty-eight (28) calendar days, and all proposal submissions are due by 3:00 PM Eastern Daylight Time (EDT), June 12th, 2024.

NOAA encourages interested offerors to review the BAA and submit any questions prior to the

Community Day scheduled for May 22, 2024. Additional questions arising from topics covered during the Community Day or Vendor 1-on-1 sessions shall be submitted in writing to the

Contract Specialist no later than 3:00 PM EDT, May 24, 2024.

**The program offices and technical points of contact may not be contacted directly and will refer all inquiries to the Contracting Officers and Specialists. Additional submission instructions are provided at Section 3.**

GENERAL INFORMATION

NAICS Code

The North American Industry Classification System (NAICS) code for this BAA will be 541715

– Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology).

Agency Name

United States Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), National Environmental and Satellite Data, and Information Service

(NESDIS), Office of Systems Architecture and Engineering (SAE)

Development Opportunity Title

Knowledge Mesh / Natural Language Processing https://www.interaction-design.org/literature/topics/personas

Announcement Type and Date

Broad Agency Announcement (BAA-NOAA-KM/NLP-2024); posted May 15, 2024

Proposal Due Date

June 12, 2024, 3:00 PM EDT

Award Information

The targeted budget for this BAA is a combined total of $6M over two years for all awardees.

NESDIS envisions (3-5) awards for projects under this BAA, with each one having a duration of up to 24 months, with a technical report as the final deliverable as well as intermediate progress reports. The government may award zero, one, or multiple contracts under this BAA, depending on the applicability of the Proposals to the technical need and congressional appropriations.

Subject to the availability of funding, offerors whose Proposals receive a favorable evaluation may be awarded.

This BAA should not be construed as a commitment or authorization to incur costs in anticipation of a grant, cooperative agreement, other transaction, or contract. Contracts, grants, cooperative agreements, interagency agreements, or other transactions and agreements may be awarded, as appropriate. At this time, it is anticipated that each project award will result in a firm-fixed-price contract.

The Government is not responsible for any costs incurred in the preparation of a submission in response to this BAA.

RESEARCH AND DEVELOPMENT TECHNICAL DESCRIPTION

Background and Overview:

Introduction

The National Oceanic and Atmospheric Administration (NOAA) has identified a need to improve and expand the open discovery, access, use, reproducibility, and trust[1,2,3,4,5,6] of its environmental intelligence - both in terms of process and information - in order to facilitate key strategic initiatives, including the advancement of social environmental equity[7,8,9,10,11] and the enhancement of weather and climate resilience across communities and natural ecosystems.[12,13,14,15]

This BAA is intended to help address the identified need by funding innovative and meritorious studies that promise to examine, deliver, and iteratively demonstrate independently validatable and working models, features, and capabilities related to the development and management of an enterprise information processing system backed by an interoperable knowledge graph ecosystem (‘knowledge mesh’) of process-contextualized, semantically enabled facts, exposed as a unified access interface (envisioned as foundation model), that is capable of dynamically https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-access-Memo.pdf https://opendatahandbook.org/guide/en/what-is-open-data/ https://en.wikipedia.org/wiki/Open_science https://en.wikipedia.org/wiki/Open_access https://opensource.org/ https://www.ecfr.gov/current/title-36/chapter-XII/subchapter-B/part-1220 https://www.noaa.gov/regional-collaboration-network/noaas-climate-and-equity-roundtables https://www.noaa.gov/topic-tags/equity https://www.fisheries.noaa.gov/feature-story/noaa-fisheries-releases-final-equity-and-environmental-justice-strategy#%3A~%3Atext%3DOur%20goals%20under%20the%20strategy%2Cmission%20work%20with%20demonstrable%20progress https://hazards.colorado.edu/research/weather-ready-research/open-data-workshop https://www.gfdl.noaa.gov/core-values/#%3A~%3Atext%3DGFDL%20is%20a%20part%20of%2Cjustice%2C%20belonging%2C%20and%20excellence https://coast.noaa.gov/funding/ira/resilience-challenge/#%3A~%3Atext%3DAbout%20the%20Challenge%2CApproximately%20%24575%20million%20is%20available https://www.whitehouse.gov/wp-content/uploads/2023/09/National-Climate-Resilience-Framework-FINAL.pdf https://www.sustainability.gov/pdfs/doe-2022-cap.pdf https://www.noaa.gov/interagency-ocean-policy/2022-sost-opportunities-and-actions-roundtable/digital-twins-to-support-ocean-science-and-technology leveraging Application Programming Interface (APIs) using the knowledge mesh capability for the purpose of answering highly contextual queries from a number of personas representing a diverse swath of NOAA users.

In part, this BAA represents an idealized follow-on to the Earth Observation Digital Twin (EO-

DT) BAA. For purposes of this study, many of the intended outcomes of the EO-DT ecosystems are assumed to be achieved and available for use, and the efforts here are intended to be spent demonstrating how an idealized EO-DT may be managed and exploited for the benefit of the

NOAA user community at large. Therefore, part of the work involved in solutions developed under this BAA will involve using open, free, and publicly available data and services from

NOAA[16,17,18,19,20,21] (and in certain scenarios, other federal agencies with complementary data and services) to mock process and metadata records according to formats prescribed, and then leveraging the mocked records to demonstrate capabilities of offeror-crafted solutions. While the fact that this BAA is related to the EO-DT is important context information, it should be understood that no explicit knowledge of the EO-DT is required to participate in this BAA.

This BAA is exploratory only and currently there is no planning for activity beyond a demonstration.

Illustrative Example

To briefly illustrate the capability this BAA is intended to enable, imagine a setting of the coastal zone of Louisiana, with all of its biodiversity, rich economic activity, highly coupled physical and ecological systems, infrastructure, and population diversity.

Through multiple process channels, NOAA currently senses, ingests, stores, and contextualizes

(represents/augments/quality controls/etc.) a variety of raw data and metadata from various sources in this geographic area - examples include satellite and in-situ sensor arrays, dive and cruise campaigns, scientific studies, and data purchases that measure sea, estuary, river, land, air, weather, biological, ecological, climate, and social parameters.

NOAA then produces additional value-added information products, models, and services based on this raw data (and also derived products based on other derived products). This often involves fusing data of different source and/or character, filtering data, representing data, extracting features from data, modeling new derived parameters, adding narrative context, and so on.

All of these NOAA information products are then published for dissemination to a diverse user base via various access channels. The types of users that consume NOAA data from these channels are varied, and include groups such as scientists, engineers, consultants, fishermen, city planners, individuals, and other government agencies. Each of these user groups likely have different intended uses for the information, and might have more than one intended use depending on context.

In the traditional access model currently employed across NOAA, each product or product family https://www.interaction-design.org/literature/topics/personas https://www.nesdis.noaa.gov/events/digital-twin-earth-observations-eo-dt-using-artificial-intelligence https://www.nesdis.noaa.gov/events/digital-twin-earth-observations-eo-dt-using-artificial-intelligence https://www.nesdis.noaa.gov/events/digital-twin-earth-observations-eo-dt-using-artificial-intelligence https://www.ncei.noaa.gov/cdo-web/ https://data.noaa.gov/onestop/ https://coast.noaa.gov/dataviewer/%23/ https://www.ncei.noaa.gov/access/search/index https://www.noaa.gov/information-technology/open-data-dissemination https://wpo.noaa.gov/empowering-open-science-unveiling-wpos-system-for-public-access-to-research-knowledge-spark/ https://data.gov/ is typically served either through its own tailored web portal interface or within a portal serving some logical grouping of similar products (e.g. satellite data in CLASS). In general, the multitude of dissemination channels (and the information systems that back them) are not connected to each other, and the access they provide is limited in scope - formats, methods of access, quality of access, inquiries supported, ways of presentation, methods of retrieval, etc. This condition represents a significant barrier in achieving stated NOAA priorities.

Unlike the current access model, the desired Knowledge Mesh outlined in this BAA will support a wider diversity of users in the ways they ask to be supported through a single interface. The envisioned interface will support contextual multi-modal user interaction by integrating services, data, and information from NOAA and non-NOAA sources. Some hypothetical examples of these users and questions may include:

● A fisherman asking - Where should I fish today for tuna? What are some of the challenges I might face?

● A city planner asking - What are some of the flood risks I should consider when building a new parking garage near Jackson Square? What about flooding risk trends?

● An ecologist asking - How healthy is the tuna population? What environmental challenges are they facing right now? How healthy are their direct predator populations?

● An insurance consultant asking - What are the chances of a major hurricane in the next 10 years? What areas of New Orleans have the greatest flood risk under expected conditions?

● A coastal scientist asking - What are some of the active in situ data collection efforts in the

Mississippi estuary?

● A systems engineer asking - What are some coverage gaps for temperature data, and what sensor networks do we have that could be used to fill those gaps?

● An agency policy analyst asking - What other government agencies are using data on crab populations in the Gulf, and in what products?

● A climatologist asking - Please summarize the trends in sea surface temperatures around the near-coast gulf waters over the past several decades, including a descriptive text, representative visualization, related NOAA data, and published reports?

There might also be users of the system that ask broader questions that cross the geographic boundary, such as:

● A social scientist asking - What economic sectors in this region are outperforming those in the Chesapeake Bay?

● A student asking - What are the important components of a coastal zone?

Further, scientists or machine-based systems might further use the system to ask more general questions in order to leverage the system API and responses in their own processes or research.

These sequences of inquiry might look something like https://www.avl.class.noaa.gov/saa/products/welcome%3Bjsessionid%3DA1141CB689CD313E96EA070F00532D60

● Can you give me all the records for the estuary model dataset X from September 1986 to

January 2020 in parquet format?

● Can you describe the schema of these files?

● Can you write me a code snippet that can open and extract column A and B from these files?

All of these questions and inquiries represent capabilities desired from solutions developed through this BAA. As part of its consideration during each unique inquiry, the responding system should be able to decide where and how to retrieve information - it may have the information handy in a trained model; it might need to leverage more recent information captured in the semantic fact table; it might need to call out to a known API to generate an answer; or it might need to perform a combination of one or more of these techniques, potentially in an iterative manner. The sample questions provided here are representative of the types of inquiries that offerors may provide in their plan, although it is ultimately up to the offeror to describe the specific types of inquiries that they intend to demonstrate in solution. Generally, as a course of any inquiry, the system should enable its users to explore given answers in terms of provenance and preservation description -asking contextual questions like

● How accurate is this response or prediction?

● Who produced this data? Who owns the process(es)?

● What processing steps went into generating this result?

● Who should I talk to, to learn more about this product?

● What other products might I care about?

● What other types of users have asked about this topic?

● How was uncertainty measured here?

● How confident is the result?

● What software can I use to work with this data?

Offerors should identify how they will leverage the knowledge mesh data model - an RDF encoding of the core Information Object and Information Package data structures of the Open

Archival Information Systems (OAIS) - to support comprehensive probing of provenance and lineage during user inquiry.

Another important criteria for offerors to consider when developing Proposals is that as a federal science organization with an authoritative mandate, NOAA must be able to provide users with free, confident and factual information. It is not enough to rely on generative AI derived results without the ability to qualify with facts. NOAA’s reputation as an authoritative provider of trustworthy information must be maintained. The proposed solution must provide factual grounding to the user alongside results or on demand, depending on use context (the needs of a user within a particular inquiry). As the system responds to inquiries like those posed in the example above, it is important that uncertainty (i.e., the ‘goodness’ and confidence of a result and its components) be transferred back to the user. This is another area in which the interplay between the knowledge mesh (with its ability to persist facts in an interoperable standardized reference https://public.ccsds.org/pubs/650x0m2.pdf model) and the access interface will be critical.

In light of these conditions, Proposals should describe in their solution strategies on how they will qualify, quantify, store, combine, and communicate uncertainty when computing results, with the understanding that uncertainty is highly contextual and may take different forms depending on the specific inquiry and character of response. Offerors should therefore carefully consider the scope of their proposal, in terms of focus, depth, and breadth of proposed study.

Technical Interest

Toward achievement of the desired capability, NOAA is interested in studies that propose solutions integrating proven or nascent advanced artificial intelligence (AI) and machine learning (ML) technologies in the areas of natural language processing (NLP), networked semantic knowledge graphs (KG), graph neural networks (GNN), transformer-based foundation models (FMs) encompassing large language models (LLMs), as well as knowledge fusion techniques that may include prompt engineering approaches such as Retrieval Augmented Generation (RAG), or federated KG query approaches such as virtualization/materialization.

Use of these advanced AI/ML technologies in areas such as web search, healthcare, finance, e-commerce, legal, education, supply chain, emergency response, and a rapidly expanding list of additional domains, have demonstrated a broad potential to revolutionize the way information is processed, understood, and utilized. Exploration at NOAA is intended to help facilitate maturation of the next generation unified data and information system by identifying and demonstrating innovations that leverage, integrate, and disseminate NOAA information for purposes of broadly improving enterprise, constituent, and environmental outcomes in ways exemplified by the provided illustration and other contextual examples found in this BAA.

It is important to emphasize that in order to achieve all intended outcomes, proposed solutions must promise to exemplify open science and open source principles by ensuring that developed solutions and all of their components - including datasets, models, libraries, references, tools, software, etc. - may be published publicly and openly, and not be restricted from public use by

NOAA or its stakeholders.

During the performance period, delivered solutions are expected to be regularly validated by NOAA designated experts in terms of user experience and domain correctness through a NOAA interface, implying that solutions should provide a publicly (or NOAA) accessible endpoint so that this endpoint can be tested in the user experience and validation system context.

Additionally, in order to provide the best possible chance for adoption and implementation at NOAA, solutions must be developed in an easily portable manner, generally following SQALE principles, using transferable and standardized technologies. Practically this might include use of

Web 3.0 standards; OCI containers; Object Storage, JSON-LD, Parquet, NetCDF, COG, and Zarr data/metadata formats; APIs that leverage the OpenAPI spec; OGC standards; open, transferable protocols for technologies like triple stores and vector databases; standard vocabularies and thesauri[22,23,24,25,26]; non-commercial, open source base models; and small, lightweight, functional libraries.[27,28,29,30,31] https://en.wikipedia.org/wiki/Natural_language_processing https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Graph_neural_network https://en.wikipedia.org/wiki/Foundation_model https://en.wikipedia.org/wiki/Foundation_model https://en.wikipedia.org/wiki/Large_language_model https://en.wikipedia.org/wiki/Prompt_engineering https://en.wikipedia.org/wiki/Prompt_engineering#Retrieval-augmented_generation https://en.wikipedia.org/wiki/Prompt_engineering#Retrieval-augmented_generation https://kg-construct.github.io/rml-core/spec/docs/ https://open.science.gov/ https://opensource.org/ https://en.wikipedia.org/wiki/SQALE https://www.w3.org/RDF/ https://opencontainers.org/ https://en.wikipedia.org/wiki/Amazon_S3 https://json-ld.org/ https://parquet.apache.org/ https://www.unidata.ucar.edu/software/netcdf/ https://www.cogeo.org/ https://zarr.dev/ https://www.openapis.org/ https://www.ogc.org/standards/#%3A~%3Atext%3DOGC%20Standards%20are%20the%20glue%2Cup%20the%20process%20of%20innovation https://www.w3.org/TR/sparql11-query/ https://en.wikipedia.org/wiki/Vector_database https://schema.org/ https://cfconventions.org/ https://bioportal.bioontology.org/ontologies/SWEET https://niemopen.org/ https://www.earthdata.nasa.gov/learn/find-data/idn/gcmd-keywords https://huggingface.co/ https://rdflib.readthedocs.io/en/stable/ https://pytorch.org/ https://spacy.io/ https://github.com/morph-kgc/morph-kgc https://github.com/langchain-ai/langchain

Domain Assessment

Affordances that have been normalized by cloud computing over the past decade were largely unavailable when the current generation of data management systems were designed and deployed. While revolutionary at the time, an ecosystem characterized by disparate and disconnected Spatial Data Infrastructures (SDIs) now acts as a collective barrier to achieving the large degree of integration needed to power the next generation of data driven information systems. Challenges and benefits in overcoming the SDI landscape have been recognized by national and global leaders in the sciences, and coordinated change efforts are now underway.

As examples of these efforts, the United Nations Committee of Experts on Global Geospatial

Information Management (UN-GGIM) laid out a comprehensive global vision and transition plan to a Geoverse; the National Science Foundation (NSF) has initiated a national effort to build a prototype Open Knowledge Network (proto-OKN) based on Web 3.0 standards;

organizations such as the Committee on Earth Observation Satellites (CEOS) and the Group on Earth Observations (GEO) have developed interoperability frameworks; the Open Geospatial

Consortium (OGC) has led new cloud ready standards development and testbeds; and governments around the world have begun actively coordinating on standards and best practices for enabling federated digital twin and open science ecosystems grounded in principles of holistic FAIR (findable, accessible, interoperable, and reusable), CARE (collective benefit, authority to control, responsibility, ethics), TRUST (transparency, responsibility, user focus, sustainability and technology), and AI-readiness.[32,33] Solutions developed within this BAA will help shape, verify, and validate practical implementation of these pillars, principles, and frameworks at NOAA.

Solution Intent

Capitalizing on emergent domain capabilities, the National Environmental Satellite, Data, and Information Service (NESDIS) has been actively developing an open knowledge mesh capability within the NESDIS Common Cloud Framework (NCCF). Implemented as a Web 3.0 standards based, event driven, process-API oriented, federated digital twin framework[34,35,36,37], the knowledge mesh is intended to serve as an enterprise-wide participatory orchestration engine for publishing, sharing, and running models, processes, and tasks, while automating boilerplate burdens of full-provenance surveillance and orchestration infrastructure. See figures 1 through 9 in Appendix A for an illustrated overview of the foundations of the knowledge mesh implementation and how these concerns are addressed.

This knowledge mesh framework is intended to enable the combination of physical, phenomenal, social, biological, environmental, economic, and user access processes so that this data can be easily used together to discover new patterns and knowledge. It is also intended to help organize different types of information so that all of NOAA’s stakeholders - including scientists, engineers, decision makers, and the public - can understand and use the information in new ways. The design of the mesh is further intended to support automated full-provenance capture of process-oriented data; the mesh captures and holds semantically enabled information about how it runs microservices, libraries, models, etc., generally called ‘user business logic’ or

UBL, including:

● Narrative and semantic metadata descriptions of executable resources.

https://ggim.un.org/meetings/GGIM-committee/12th-Session/documents/Future_Geospatial_Information_Ecosystem_Discussion_Paper_July2022.pdf https://ggim.un.org/meetings/GGIM-committee/12th-Session/documents/Future_Geospatial_Information_Ecosystem_Discussion_Paper_July2022.pdf https://www.nsf.gov/pubs/2023/nsf23571/nsf23571.htm https://ceos.org/document_management/Virtual_Constellations/LSI/Meetings/LSI-VC-12/Presentations/CEOS%20Interoperability%20Framework%20Discussion%20Paper.pdf https://qa4eo.org/ https://www.ogc.org/initiatives/t-19/ https://ditto-oceandecade.org/ https://science.nasa.gov/researchers/open-science/ https://www.go-fair.org/fair-principles/ https://www.gida-global.org/care https://www.nature.com/articles/s41597-020-0486-7 https://github.com/esipfed/data-readiness/ https://bipartisanpolicy.org/explainer/ai-ready-open-data/ https://ceos.org/document_management/Working_Groups/WGISS/Meetings/WGISS-52/2.Wednesday/2021.10.20_10.50_NOAA_NESDIS%20Cloud%20Archive%20Pilot%20(NCAP).pdf https://ui.adsabs.harvard.edu/abs/2022AGUFMIN42B0330B/abstract https://esto.nasa.gov/files/AIST/ESDT_ArchitectureFramework.pdf https://2022esipjulymeeting.sched.com/event/12esa/toward-a-digital-twin-for-earth-system-overview-and-enabling-technologies https://digitaltwin.stcenter.net/index.php/digital-twins-webinar-series-2-concepts-practices-and-vision/

● What semantically classified input values are provided at runtime.

● What semantically classified output values are produced at runtime.

● What entity, representation, or dissemination information models are produced (where all information models are based on a generic, recursive, upper level archetype ontology).

● What task contexts they have been used in (where tasks are units of UBL and semantic storage patterns).

● What process contexts they have been used in (where processes are acyclic digraphs of tasks).

This design satisfies a multitude interests and requirements, including the critical ability to ensure easily reproducible scientific results; meeting agency oversight requirements for data stewardship deriving from NARA, PARR, and other authorities; and in conjunction with the dynamic event driven ability of the system to execute arbitrary workflow through data, enabling advanced AI tools to learn how to construct and run processes ‘on the fly’.

In the longer term, agency-wide adoption and exploitation of the knowledge mesh is intended to allow NOAA to participate in larger efforts and collaborations, like the NSF proto-OKN and broader international Digital Twins ecosystems, by acting as a unified network node in broader ecosystem contexts. As it matures, the knowledge mesh should be richly linked to other sources of information from around the semantic web, such as resources, references, data, and metadata provided by partner agencies. This represents progress toward achieving a system characterized by five star linked open data, an original motivating factor in developing the world wide web.

Solutions sought by this BAA are expected to leverage identified technologies, including AI and

ML, to develop and demonstrate models, methods, and processes that support use and management of the knowledge mesh. In general, the NCCF knowledge mesh aligns with NOAA

Strategic Objectives including those related to cloud and data management, as well as the NOAA AI Strategic Plan, and is a collaborative effort between NESDIS Systems Architecture and Engineering (SAE), the NOAA Center for AI (NCAI), and the National Centers for

Environmental Information (NCEI), with support from others around NOAA.

Provisions and Expectations

Offerors will be provided access to:

● Python Packaging Index (pip) access to the knowledge mesh implementation and its API for constructing knowledge mesh resources and submitting them to the processing system. It is important to note that the API can be generally used to construct local resources, but to use the processing component aspect of the system for general deployment of a working copy of the knowledge mesh, the Offeror will need an

Amazon Web Services (AWS) account. This is not a requirement of solicitation response.

● A copy of the existing knowledge mesh metadata as an archive based on a bucket of https://www.archives.gov/ https://www.ngdc.noaa.gov/parr.html https://5stardata.info/en/ https://www.nesdis.noaa.gov/about/our-offices/systems-architecture-and-engineering http://noaa.gov/ai https://www.ncei.noaa.gov/ denormalized JSON-LD files to illustrate how system metadata is organized.

● Additional specifications and other documentation around how the knowledge mesh are constructed and populated.

● Example inputs for specifically identified and selected tasks such as narrative-based data transfer instructions and AI-ready maturity matrices, as requested by Offerors, for planning for or executing particular proposal aspects.

● Links and guidance on preferred community vocabularies and licenses as requested by offerors.

Offerors will be expected to:

● Identify capabilities for study and development, select use cases, identify available and acceptable datasets and services to leverage. Mock solution-specific metadata related to processes, tasks, patterns, templates, and records in the format prescribed by the existing knowledge mesh, using the API as necessary.

● Manage technology and implementation choices, infrastructure procurement, and broad solution implementation.

● Demonstrate conformance and interoperability with the terms listed above - primarily centered around Open Science, Open Source Software/Data/Models, non-commerc ia l models, containerization of solutions.

● Regularly demonstrate working solutions, following an agile project plan, providing accessible endpoints for NOAA experts to perform validation.

● Respond to emergent directions from NOAA in terms of project plan as the project progresses, following an agile development model.

Scope and Objective

Through this call, NOAA is seeking to fund joint partnerships with other U.S Government agencies, academia, and/or private industry, versed or interested in applying semantic knowledge graph, natural language processing, graph neural network, and foundation model capabilities and techniques to explore enhancing the NOAA/NESDIS mission of processing integrated data from ingest to dissemination. The objective of the study will be the generation, assessment, and delivery to NOAA of processes, software, and information models that demonstrate solutions around one or more areas of knowledge mesh construction, grooming, and exploitation.

General Guidance

Processes described as part of study solutions should be designed in a way that aligns them with the ideals described in the background and overview of the BAA, which in part, practically means:

● Broadly leveraging community standards, such as OGC data transfer and format standards, W3C semantic web standards, and targeted semantic metadata vocabulary standards like GCMD, SWEET, Schema.org, Science-on-Schema.org, WORMS, NIEMOPEN, and others, depending on the scientific domains the proposal addresses.

● Conforming use of solution within a process/event-oriented architecture that decouples microservice computation from information model storage, ensuring reuse of both aspects.

● Meeting legal licensing requirements that would allow solutions to be leveraged in an open knowledge mesh context including a marketplace and role-based access

(this depends on solutions ensuring they meet the open science/open source/non-commercial criteria well described in the background section of this BAA).

Scope Intent

The scope defined in this solicitation is intended to help NOAA to gain a better understanding of the optimal applications of these technologies in the context of its needs and current activities and then develop prioritization and integration paths with an eye toward next generation information systems enablement, ultimately with a goal of satisfying identified NOAA mission needs.

This BAA intends to focus primarily on examining tasks related to knowledge mesh construction, grooming, and exploitation, with an emphasis on exploitat ion technologies centered around the user interface, as well as ensuring validity of results via uncertainty quantification/qualification, along with factual grounding, and federation with other systems.

This solicitation prefers development patterns that implement decoupled, microservice-oriented tasks that produce easily consumable information models based on known vocabularies, so that they can be composed in process contexts and be standardized across the semantic web; offerors should suggest specific scenarios, datasets, and approaches for application, keeping in mind the framing of the background and overview section of this BAA.

The Offerors’ use of additional federal data that is currently available publicly, at NOAA and other government agencies, is anticipated to be critical for success for certain study areas, especially around demonstrating holistic system use and government federation beyond NOAA.

Technologies Under Test

A fundamental objective of this research and development effort is to advance the understanding and integration of specific technologies and their uses, including foundation models (meant to include large language models) and innovative access tools within the context of the described knowledge mesh; an overarching goal is to enhance the capabilities of the NOAA) NESDIS through the effective utilization of cutting-edge technologies, including:

● Model Integration, including Language and Foundation Models (technology baseline)

○ Investigating and implementing state-of-the-art language models that can comprehend and generate domain-specific content relevant to NOAA's mission and data.

https://www.earthdata.nasa.gov/learn/find-data/idn/gcmd-keywords https://github.com/ESIPFed/sweet https://schema.org/ https://github.com/ESIPFed/science-on-schema.org https://www.marinespecies.org/index.php https://www.marinespecies.org/index.php

○ Developing methods to integrate these language models seamlessly into existing workflows, facilitating improved information retrieval, analysis, and communication (e.g. with RAG technologies applied to the underlying knowledge mesh persistence layer).

○ Developing strategies for sustaining and improving language models based on knowledge mesh use and feedback.

○ Semantic Mesh Enrichment, Management, and Interlinking (technology maturation)

○ Exploring interoperability and scalability of the knowledge mesh, ensuring seamless collaboration and information exchange across different units within NOAA.

○ Researching techniques for semantic enrichment of data and documents within the knowledge mesh to enhance context-aware understanding.

○ Establishing methodologies for interlinking diverse datasets and information sources, fostering a cohesive and comprehensive knowledge environment.

○ Exploring techniques for uncertainty quantification, automated management, and sustainability.

● Tool development for user-friendly interaction (technology integration)

○ Creating user-friendly tools that leverage the integrated language models and knowledge mesh, promoting efficient and intuitive interaction for NOAA stakeholders.

○ Prioritizing development of tools that enhance data accessibility, analysis, and decision-making processes for a wide range of users, including scientists, researchers, and operational staff.

Growth in NOAA’s understanding of these technologies through projects funded by this BAA is intended to be developed within the framework of the areas outlined in the section below.

Generally, the solutions developed under this BAA must align with tenets of public Open Science[37,38,39], constraints of which are described and referenced in depth in the background and overview section of this BAA.

Study Options

Knowledge mesh work can be divided into three separate phases: construction, grooming, and exploitation. We are interested in stand-alone solutions for each phase, or solutions that address multiple phases.

As part of their study, offerors should develop processes, tasks, and models that use the previously outlined technologies to address multiple aspects related to the mesh in one or more of the following phases:

● Phase A: Knowledge Mesh Construction - capabilities related to automating ingest, https://www.unesco.org/en/open-science?hub=686 https://nasa.github.io/Transform-to-Open-Science/about/ https://open.science.gov/#%3A~%3Atext%3DOpen%20Science%20is%20the%20principle%2Ccollaborations%2C%20reproducibility%2C%20and%20equity enhancement, or quality control of information meant to come into or actively coming into the system. This could include items such as:

○ Automated creation of runnable process documents (knowledge mesh defined process templates) from narrative-based data transfer instructions.

■ E.g., the system might receive a narrative request to set up a process for a new data stream, such as a wetland study. The request describes how the data will arrive, what it contains, how it should be quality checked, what other things it might relate to, where it needs to be delivered, and the things that need to be extracted from it or otherwise represented in metadata. Based on this, the system creates a system-readable process definition document which can then be executed automatically to retrieve the data, process it, and disseminate it. This process definition may go through human review and acceptance before activation. As NOAA receives both one-off and recurring requests, both cases are interesting to study.

○ Automated quality scoring of process data (patterns, entities, and information models) through population of standard analysis ready, AI ready, and data maturity matrices.

■ E.g., a new recurring system process for Uncrewed Systems (UxS) data might be submitted for activation. This process might include in its process graph a transformation task that assesses the received data and populates a quality matrix. When activated, as data flows through the system, the data should be assessed for quality and the matrix template be completed to produce a full record. This record creation might involve checking the parameter values received to make sure that the data conforms to the semantic labels and classes that are claimed on each parameter. This might also involve checking whether certain values are within tolerance, or whether or not all aspects of a submission are present.

This should facilitate uncertainty scoring.

■ E.g., a new entity pattern or template for a satellite data granule stream might be submitted. This submitted resource flows through a core resource deployment process, where it should be checked for validity. In this case, the semantics of the pattern or template - the classes, labels, descriptions, and references should be checked against the core reference model categories and against the rest of the resource motif to validate whether or not the resource is of high quality or not. The quality score should be recorded as part of the resource metadata during deployment. In case the system identifies potential low quality, the system should alert for manual review. The system record may store scores as part of the uncertainty criteria.

○ Entity patterning and templating against existing resources to provide suggestions to users during resource creation.

○ E.g., as pattern and template resources are created, users should be able to submit them to the system for checking against other system pattern and template resources. The system checks should identify similar resources already existing, parts of resources (specific substructures) that may be important to include, and terms that maybe should implement certain standard namespaces, classes, or labels.

○ Entity extraction and persona or use-based semantic information model population.

■ E.g., as processes are triggered, models might automatically extract important entities listed in the semantic template, populating field values dynamically. This is situationally dependent on the entity template and data stream, so solutions developed here should demonstrate generalization and wide applicability. One specific illustration of this might be as benthic surveys come into the system, secondary observations, like bleaching observations, may be captured in ancillary and freeform text fields. The entity extraction work would be to take these freeform text fields, identify valuable parameters like bleaching, and populate them as semantic metadata fields so these can be searched as facts and accessed.

■ E.g., as processes are deployed and used, the system should be able to create and populate information models that help answer questions from scientists that want to solve new problems like how suitable is model X, Y, or Z to answering my specific problem"

● Phase B: Knowledge Mesh Grooming - capabilities related to continual sustainment of the knowledge mesh resource in terms of optimization, standardization, and quality control. This could include items such as:

○ Analysis of existing entities and information models to identify opportunities for semantic standardization.

■ E.g., system processes should, at regular intervals, review existing system resources, particularly in pattern and template namespaces, looking for opportunities in ontological alignment in terms of field classes and namespaces, sending alerts and notifications to the resource POCs when potential optimizations are found with suggested improvements or courses of action for resolution.

○ Analysis of existing entities and information models to identify opportunities for model normalization.

■ E.g., a system process should, at regular intervals, review existing system resources, particularly in pattern and template namespaces, looking for opportunities to merge patterns or templates together with other similar resources (e.g. multiple paleo datasets), sending alerts and notifications to the resource POCs when potential optimizations are found with suggested improvements or courses of action for resolution.

○ Analysis of resources being marked as low quality.

■ E.g., a system process should, at regular intervals, review existing system resources, particularly in pattern and template namespaces, looking specifically at access side (inquiry) processes, in order to identify processes or process components that regularly demonstrate low satisfaction scores or low utilization scores. Notifications should be sent out for investigation that include comparative analysis against other similar resources.

○ Analysis of systems for dead or improper linked data.

■ E.g., system processes should, at regular intervals, crawl existing resources, looking for dead and/or low-confidence relationship scores in semantic field values. When found, dead links should be removed, corrected, and/or noted for notification. Mismatches or low confidence in semantic correlations should be notified for review with suggestions for more appropriate reference resources.

● Phase C: Knowledge Mesh Exploitation - capabilities related to the (foundation model based) access side interface of the system. This could include items such as:

○ Creation of conversationally-based interfaces for use in storefront discovery, supporting a variety of multi-modal uses, user personas, and contextual questions including, but not limited to, identification of data-of-interest; probing data-of-interest to identify provenance information and related context; using context to make on-demand predictions. See Appendix B for examples and suggestions of specific user personas, typical questions, and capabilities that are intended to represent the type of success criteria NOAA is seeking to demonstrate this capability.

■ E.g. Taking an open source, non-commercial pre-trained base transformer model; mocking data in the area designated for study, like the coastal zone, so that it fits within the mesh orientation; performing transfer learning on the model against the mesh by harvesting multiple dimensions (testing use of semantic classes/namespaces, of string-held narratives, of linked data); fine-tuning the model to develop question-answer interface;

implementing a RAG interface and endpoint to demonstrate how the system may leverage the base model, the domain information, access to the normalized mesh, and use of the process dimension in the mesh to call known APIs as needed.

○ Integration of cross-interest NOAA data with links to non-NOAA information and services, demonstrating an ability to support use of federated multi-agency information in answering research inquiries. Integration of non-NOAA data and API services is restricted to federal partners including USGS, NASA, USDA, EPA, DOI, DOC, NSF OKN participating agencies like NREL, NIH, NIJ, or data otherwise held at data.gov. Including these sources should support demonstration of results that merge both archival and real-time factual data of wide variety and source, to answer questions, make predictions, or run simulations that rely on cross-agency collaboration in areas that may include topics like tracking changes https://www.usgs.gov/ https://www.nasa.gov/ https://www.usda.gov/ https://www.epa.gov/home https://www.doi.gov/ https://www.commerce.gov/ https://new.nsf.gov/funding/opportunities/building-prototype-open-knowledge-network-proto https://www.nrel.gov/ https://www.nih.gov/ https://nij.ojp.gov/%3Cfront%3E to land use, ecosystem or human health, monitoring natural disasters, addressing infrastructure planning, or predicting agricultural yields. In this topic, please note that the intent is not to create new data products, but to leverage existing APIs and information.

■ E.g. Augmentation and expansion of the solution in the above bullet by including acceptable partner data and API services (USGS, NASA, USDA, EPA, DOI, DOC, NSF OKN participating federal agencies like

NREL, NIH, NIJ, or data otherwise held at data.gov) to demonstrate evolution of the system toward a federated unified search network. This may involve additional technologies like API service use, testing of legal interoperability and uncertainty quantification for remote APIs, or virtualization of queries against disparate knowledge graphs.

Note that each of these phases and their associated capabilities represent a potentially significant amount of work, and are not necessarily coupled with capability needs within other phases (although they may be tied together). Offerors should be careful to specify which phase they are attempting to address, being mindful of resource commitments over the period of performance, as well as how the solution of one phase may facilitate solution of other phases.

It is recommended that offerors consider each phase roughly aligned to the technology description outlined in the technologies under test section. Offerors may choose to submit comprehensive study plans, multiple phases plans, or singular phase plans, depending on how the offeror imagines fulfilling the call. In all cases, the offeror should call out in their Proposal how the technologies under test relate to the study phases.

Planning Considerations

Note that due to the knowledge mesh requirement, there may be a significant period of discovery and acclimation in terms of data mocking, and this should be considered and clearly discussed when planning solutions.

Proposals that demonstrate clear storytelling, linking feature dependencies, and scaffolding from feature to feature, are highly desired. This is true in whatever form offerors might deliver solutions (either single or multiple phases).

Offerors should also carefully consider the larger semantic context and sentiment of what is being sought in this BAA. The most pressing need for NOAA in meeting its strategic goals is to demonstrate how a knowledge mesh might support a multi-modal and multi-contextual interface, and how this interface might leverage the underlying knowledge mesh and its capabilities to ensure factual correctness, good performance, and data ‘freshness’ (how up to date data is).

Proposals that center on these topics, while also tying in solutions around system sustainability

(grooming) and growth (construction), are desired.

Finally, it is worth highlighting that for all phases, but particularly phase C, the Proposal will be judged on the plans for satisfying a wide variety of specific users - represented as Personas in Appendix B -…