SOW_Draft_21AUG24.docx

"Source Selection Information—See FAR Subparts 2.101 and 3.104-4"

DRAFT

Statement of Work - (Aug 21, 2024) NOAA National Weather Service Commercial Data Program

1.0 Background

In 2008, the National Academy of Sciences (NAS) released Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks. This NAS report provides a framework for filling gaps in the Federal Government’s meteorological, hydrologic and related environmental observing capabilities, to enable significantly improved forecasts and warnings, especially for high-impact, local-scale weather events. The framework specifies that NWS and the Federal Government should effectively leverage observing networks that are owned and operated by nonfederal parties (e.g., the states, academia, and the private sector), in addition to deploying new federal systems.

In response to the NAS, the National Weather Service (NWS) began the National Mesonet Program (NMP) in Fiscal Year 2009. The program initially served as a mechanism for purchasing data from a small number of nonfederal surface observing networks (“mesonets”) located mostly in the south-central U.S, operated by state governments, and one private sector company providing nationwide surface coverage. The NMP has since evolved into NWS’s primary program for purchasing nonfederal data from a wide variety of observing networks consisting of various observing systems technologies and platforms, spanning local to global in coverage, and extending beyond the original meaning of “mesonet.” This mix includes: networks operating in all 50 states providing surface meteorological observations and subsurface soil moisture/temperature; ground-based remote sensing providing vertical profiles of wind, temperature and humidity; aircraft-based observations providing vertical profiles; commercial radiosondes (“weather balloons”); commercial radars filling gaps in NWS’s federal Next Generation Weather Radar (NEXRAD) 88-D network; near-shore coastal/marine observations; and ship-based observations and buoys on the high seas proving surface meteorology and oceanic variables (e.g. wave heights/spectra and Sea Surface Temperature).

This growth in recent years from a program primarily procuring surface meteorological data into a broad-based commercial observations program has resulted from legislative direction, including in annual Appropriations and The Weather Research and Forecasting Innovation Act of 2017. More recently, legislation reauthorizing the 2017 Act authorizes the Commercial Data Program (CDP) consisting of several program elements, provides a standalone authorization for the NMP itself, and codifies the NMP as the flagship and largest component of the CDP, both in terms of the authorized budget and the variety of observing systems from which the program is directed to procure commercial data.

With this Statement of Work (SOW), NWS and its CDP-NMP seeks to procure data from non-federal networks covering an expansive range of geographic/observable domains and types of observing systems/networks, including but not limited to those listed above.

1.1 Extending NWS’s Observing System Capabilities and Filling Gaps

The CDP enables NWS to extend the reach of its own federal observing capabilities by leveraging observations from nonfederal networks whose observing platforms are deployed at significantly closer spacing (smaller distance between platforms) than NWS’s own platforms. The CDP further facilitates access to specific types of networks and observations that do have analogous federal capabilities. All of these various types of non-Federal platforms allow NWS to observe meteorological, hydrological, oceanic, and other environmental quantities with much greater reach, expanse, and precision than would otherwise be possible with NWS’s federal systems and platforms alone.

For all types of observations that are part of the broader CDP-NMP mix, NWS will leverage only those networks that aspire to national and international standards for siting, operations, and provision of high-quality data that is suitable to both NWS applications and the breadth of applications of observations across the broader weather enterprise.

1.2 Ingest, Processing, Dissemination: Relevant Systems and Applications

For most of the 15-year history of the NMP, the federal Meteorological Analysis and Data Ingest System (MADIS) has been the primary conduit for delivery and receipt of non-federal data provided by NMP partners. MADIS is an operational NWS ingest and dissemination system that provides a pathway into NWS for data streams from many non-Federal networks, and serves as a repository for data and metadata for partner networks. MADIS collects non-Federal observations and their metadata in real time, and makes the information accessible to NWS field offices that use these observations in the forecast and warning process, and to downstream forecast and warning systems, such as data assimilation and numerical weather prediction. MADIS also provides data access services for users who are external to NWS, in the broader weather enterprise, including many nonfederal partners.

With this SOW, and in light of the NMP having evolved into and being directed to serve the primary NOAA/NWS CDP program element spanning a large variety of observing systems types, NWS seeks to broaden the scope and mix of systems and applications relevant to the ingest, processing, dissemination, and application of these nonfederal data sources.

2.0 Scope of Work

The NWS seeks commercial pathways analogous to MADIS for meeting data-related requirements heretofore met by interfacing with and delivering data to the federal MADIS system. Such analogous pathways should serve key NWS applications that have been highly dependent on the federal MADIS system including, but not necessarily limited to, the Advanced Weather Information Processing System (AWIPS) and the Multi-Radar Multi Sensor System (MRMS). Analogous commercial pathways are sought to provide the pathways heretofore provided by the federal MADIS, processing and providing data streams to the NWS Gateway, and to downstream applications highly reliant on nonfederal data streams.

This SOW does not extend to operation of the core MADIS functionality itself nor to IT requirements, but only to services provided to the government via alternative pathways inherent in the provision of the data itself.

This SOW provides requirements for the prime contractor to serve as a liaison between all private, state, and University mesonets. It is expected that the prime contractor will form numerous subcontracting relationships to meet the requirements of this SOW, particularly in the domain owners and operators of nonfederal observing networks that are the sources of the observational data and related requirements. The prime will hold the contract with the government and ensure requirements are met, and ensure coordination among the subcontractors to that end, and for providing the interface between the government and the subcontractors for administrative functions such as payment and invoicing.

List of Current NMP Partners

3.0 Requirements

The contractor shall meet all Requirements defined below for an entire Ordering Period as directed in individual Delivery Orders.

3.1 Data-Related and Services Requirements

NWS requires the contractor to provide the data from various platforms and to provide services as described and referenced in Section 1.0 Background:

3.1.1 Surface Meteorology over Land: In-Situ meteorological observations (wind, temperature, moisture, pressure, precipitation) from fixed-location platforms primarily in the lowest 10 meters of the atmosphere, and, where available at these observing stations, soil moisture, soil temperature, solar radiation, and road-surface temperature.

3.1.2 Ground-Based Vertical Profilers Utilizing Remote Sensing: Temperature, moisture, wind speed and direction, and other related meteorological fields and associated parameters, through the vertical extent of the troposphere, including at multiple heights and derived with fixed-location ground-based remote sensing technologies (“vertical profilers”);

3.1.3 Aircraft-Based Observations: Vertical profiles of temperature, moisture, wind speed and direction, and related fields, on ascent and descent paths of aircraft in the vicinity of airports and en route data, as available, including from in situ sensors mounted on aircraft and derived methods.

3.1.4 Rawinsondes and Uncrewed Aerial Systems: Vertical profiles of temperature, moisture, wind speed and direction, and related fields, through the depth of the troposphere to the maximum altitudes possible from commercially operated “weather balloons” and Uncrewed Aerial Systems (UAS). Note: UAS as defined here does not include long-range, fixed-wing aircraft providing targeted surveillance missions, but smaller UAS, typically of rotary wing design, providing vertical profiles at a fixed location in the horizontal.

3.1.5 Marine and Oceanic: Observations of surface meteorology and ocean surface variables from fixed and drifting platforms including but not limited to commercial ships and buoys.

3.1.6 Gap Filling Radars: All available data including reflectivity, velocity, derived data and fields, from X-, S-, or C-Band radars providing coverage where known gaps exist in the NWS federal radar network.

3.1.7 Remote and Unique Data: Data from surface mesonets as generally defined in Requirement 2.1.1 that are sourced from platforms in data voids and thus highly valuable to the NWS forecast and warning mission, and from remote regions outside the continental United States, requiring unusually complex and high-cost maintenance procedures.

3.1.8 Additional data from networks that are not already providing data to NWS via the CDP-NMP as defined above, and in accordance with the direction of the aforementioned Weather Act and future legislative direction for the CDP to expand the program to new data types, observing domains, and platforms.

3.1.9 Observational data services, including collection, formatting, metadata handling, and provision to the government of observations that are obtained via smaller and publicly available networks that are not directly affiliated by subcontract.

3.1.10 Services provided from commercial pathways analogous to MADIS for provision of nonfederal data to the NWS Gateway, AWIPS, MRMS, and other systems as directed and appropriate for a given data stream, including support services for integration, processing, provision, and utilization of data in those downstream systems. Where appropriate, includes a helpdesk function for users of data analogous to what has historically been provided by the NWS “MADIS Helpdesk.”

3.1.11 Services that may take the form of individual “pilot projects” supporting NWS in the evaluation of new data stream for application in its forecast and warning mission and relevant systems and processes.

3.2 Platform Requirements

If a contactor provides any observational data from a given platform, all available environmental variables that were available at the time of subcontracting shall be made available from that platform, consistent with all performance requirements below for availability and frequency.

3.3 Geographical Requirements

In general, for a given nonfederal network providing data, data is required extending to the full horizontal and vertical domain of the network and its observing range, with specific quantities including number of platforms and data volumes as otherwise specified in individual delivery orders.

3.4 Performance Requirements

The contractor shall:

3.4.1 Coordinate and establish subcontracts with networks that use standards to site and operate observation stations;

3.4.2 Maintain detailed metadata that address the attributes, formats, and standards described in the references in Appendix A. Metadata attributes for individual networks and types of observing systems platforms may be further specified by the government in individual Delivery Orders as applicable.

3.4.3 Maintain systems and pathways with 99% uptime substantially similar to the federal MADIS system to process and deliver data to the NWS Gateway, AWIPS, MRMS, and other systems and users as appropriate in coordination with the government and consistent with Requirement 2.1.9 above.

3.4.4 Ensure that individual networks shall have no less than 95.0% availability for all potential reports averaged across all stations over the period of a month, with individual reports provided with a nominal frequency of one report every 5 minutes;

3.4.5 Provide the government with physical access to a network’s individual station and data processing facilities within 5 business days of the government’s request, to enable the government to audit for compliance with requirements.

3.4.6 Provide quarterly status reports to the government, describing the progress and status in fulfilling data and performance requirements.

3.4.7 Provide notification to the government when systemwide outages (defined as an outage across the entirety of a contributing network) of more than 1 hour occur.

3.4.8 Provide notification to the government on data distribution categories corresponding to Appendix B (Data Distribution Categories) for each network.

3.5 Quality Assurance

3.5.1 The contractor shall designate, in writing, an employee who shall serve as the contract point of contact for matters involving quality and performance or nonperformance of the required work stated in this PWS. The employee(s) shall be qualified and fully competent with full authority to act for, and on behalf of, the contractor as necessary to ensure that the required work is performed in accordance with the standards contained within this SOW.

3.5.2 The contractor shall coordinate with all non-federal commercial data and mesonet partners to ensure all data being provided is received by the Government as specified within this SOW. The contractor be responsible as the direct Government communicator for all subcontract efforts.

3.5.3 All commercial and mesonet data shall be transmitted to the NOAA MADIS database. The data collected at all sites must be directly comparable so that the data can be used in models and forecasting efforts. In the event of error of transmitting, or if MADIS is inaccessible, the contractor will be responsible for notifying MADIS (madis-support@noaa.gov) and the Contracting Officer Representative.

Appendix A *References

Fair Weather: Effective Partnerships in Weather and Climate Services; The National Academies; 2003.

Enhancing Metadata Available from MADIS for the National Mesonet; Michael F. Barth, NOAA/ESRL/GSD, Boulder, CO; and P. A. Miller and D. Helms.

Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks; The National Academies; 2009.

Guide to Meteorological Instruments and Methods of Observation (WMO-No. 8):

http://www.wmo.int/pages/prog/www/IMOP/publications/CIMO-Guide/CIMO_Guide-7th_Edition-2008.html

Guide on the Global Observing System (WMO-No. 488) http://www.wmo.ch/pages/prog/gcos/documents/gruanmanuals/GOS/488_Guide_2007.pdf

Observing Urban Weather and Climate Using 'Standard' Stations; T. R. Oke Initial Guidance to Obtain Representative Meteorological Observations at Urban Sites, Tim R. Oke http://www.wmo.int/pages/prog/www/IMOP/publications/IOM-81/IOM-81-UrbanMetObs.pdf

ISO 19115-1 Environmental Information Metadata Standards:

http://www.iso.org/iso/catalogue_detail.htm?csnumber=26020 http://www.wmo.int/pages/prog/www/WDM/reports/ET-IDM-2001.html

UNIDATA Metadata Conventions for NetCDF Climate and Forecast (CF) Data:

http://www.unidata.ucar.edu/projects/THREDDS/GALEON/netCDFprofile-short.htm

Metadata Encoding Formats: OGC XML/SML – Sensor Model Language:

http://www.opengeospatial.org/standards/sensorm

Integrated Ocean Observing System (IOOS) Data Management and Communications (DMAC) metadata standards:

http://marinemetadata.org/references/ioosdmacet

*Additional References will be added to cover new observing systems types and platforms listed in the Section 2.1. Most of the references above pertain to traditional surface “mesonets.”

Appendix B

Data Distribution Categories

For each network that provides data to NWS, a data rights/distribution category is assigned, indicating which types of users may have access to data from the network. The numbers below have their origin in MADIS and the categories of users who have rights to data access for a given network's assigned number, and the ability to do so when logged onto the MADIS user interface. For many networks, this category is "4," meaning that only NOAA accesses and uses the data and it is not distributed to parties external to NOAA.

Please note: the MADIS system is a repository for many sources of data that are unrelated to the National Mesonet Program. Some sources are networks owned and operated by NOAA itself, for which the notion of data rights is not applicable (hence Category 1, "N/A").

The following Data Distribution Categories are described below:

1) N/A (discontinued category)

2) Distribution to government, research, and education organizations;

3) Public - full distribution;

4) Distribution to NOAA only.

5) Distribution to NOAA, U.S.DOT (specifically FAA, and FHWA) and NOAA or DOT specified Federally Funded Research and Development Centers only.

6) Distribution only to NOAA and other WMO Members (the “National Meteorological and Hydrological Services” of other nations) in accordance with WMO Resolution 40.