NOAA-AGO-WAD-SRW-25-01-Industry Q-A.pdf

1. Do you have any template for submitting a proposal? I found the “Response Requirements” in

“REQUEST FOR INFORMATION AND VENDOR DEMONSTRATIONS AUTOMATED IMAGE

ANALYSIS SOFTWARE” document. Is it the only information about the desired proposal format?

No mandatory template required. The response requirements include the minimum requested information but vendors are free to provide more input.

2. Should we provide “Written confirmation of the organization’s registration in SAM” in the proposal or can it be postponed?

Proof of SAM registration is not required to respond to this RFI.

3. Could you please point us to any publicly available datasets for Fish, Aerial/Mammals, Benthic projects? We are especially interested in a dataset with stereo data and satellite imagery support. Do you have a document which describes data annotation requirements for such datasets (good and bad samples to annotate images, required labels and attributes)? It will help us better understand requirements for these projects.

Unfortunately, one of the issues is that the labeled image libraries are generally firewalled behind laboratory networks and security. Thus the data are generally inaccessible for more broad application and use. Some smaller efforts have been made to provide access via portals such as GitHUB, however in general those are limited by size and are not always straightforward (SEAMAPD21 dataset). One goal of the OSI working group is to work towards increasing accessibility by directly serving this data in centralized repositories and linking to portals such as FathomNet. To date software to handle stereo-camera systems, calibrations, and work flow has not been available. So there are no existing labeled image libraries with stereo-imagery available. None of the projects funded in this SI deal with Satellite imagery, however the Remote Sensing SI is funding projects that deal with that kind of data and we will advertise this broadly to our

Remote Sensing collaborators to ensure communication.

4. Could you please share some most important open-source components which are used by

Fish, Aerial/Mammals, Benthic, and Plankton projects? For example, deep learning frameworks, ML/DL models (e.g. YOLOv8, mmDetection, etc.), data annotation tools, etc. Thus we will understand which integrations are critical to implement.

CoralNet, VIAME, YOLO, onefishtwofish (sp?), TagLab, EventMeasure, Sebastes, GoogleCloud, MetaShape, ArcGIS. Bounding box, segmentation and keypoint models in standard formats, and with easily reparameterized output. Stereo vision libraries.

Multispectral imagery. Would prefer if contractors could work together or with these projects to standardize inputs/outputs to facilitate interoperability and reusability.

https://github.com/SEFSC/SEAMAPD21 https://fathomnet.org/fathomnet/#/

5. Is there an incumbent currently performing this work? If so, who?

No, this is a new requirement and there is no existing award

6. Are you seeking both technology and services?

Both, as the anticipated work includes development

7. Where is the data stored—on servers, cloud infrastructure, etc.?

Both - some projects have imagery stored locally on servers while others have cloud storage.

8. Do you currently have any metadata? If so, where is it stored?

Metadata is going to be project specific and the scope across participating projects is large. For instance metadata likely exists across a range of needs from planning, to acquisition, data processing, POC’s, and tracking information and across the survey types (i.e. fish, plankton, benthos, mammals/aerial). Given the scope of OSI, this is best handled on a project by project question and probably best handled during industry days qa/qc.

9.How do you define success, and what is your timeline for achieving it?

We envision a suite of cloud-based automated image analysis pipelines across a range of resource survey applications (e.g. fish, mammals/aerial, plankton, benthos). Image acquisition > upload to cloud > label process > model development > human-in-the-loop

QA/QC > output to analytical software. Two existing Quarto style documents highlight some recent fish examples including SEFSC and SWFSC Cloud based pipelines.

https://sefsc.github.io/SEFSC-VIAME-Web-GCP/ https://www.viametoolkit.org/

10.Does this technology need to run in an air-gapped environment?

While we do not have high security requirements, it is the case that many of our users operate “in the field”, where internet connectivity may not be available. Furthermore, in future, we will want to deploy some of the developed models (but not annotation software) on “the edge”, which may mean underwater and so without inter or intra net connectivity.

Certain types of data in NOAA are proprietary and would need secure environments in which to label, develop, and process information. For instance, for electronic monitoring efforts of commercial fishing activity are proprietary and need to be protected. Most of the resource monitoring efforts that are independent of commercial activity (e.g.

Fisheries Independent Monitoring) there are fewer restrictions on data accessibility.

11. Do vendors have to be Small Businesses to respond to this RFI?

No! We are looking for feedback from all potential solutions and industry partners.

12. Is it NOAA's goal to label their imagery themselves using commercial platforms, or to leverage industry's existing labeling resources with input from NOAA experts?

As with other other answers the sweep of projects funded here have a range of entry points from beginners to experts. Some of these projects will have limited personnel and thus a labeling resource would be useful. Other projects have image libraries stood up, so more help might be necessary on model creation and validation. Thus the needs will vary by project and this is best dealt with during industry day qa sessions.