NFFM73102301304KAW SOW.pdf

DEPARTMENT OF COMMERCE (DOC)

NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION (NOAA)

NATIONAL MARINE FISHERIES SERVICE (NMFS)

NORTHEAST FISHERIES SCIENCE CENTER (NEFSC)

Image Collection, Processing, and Machine Learning to Support Northeast Clam Survey Programs

STATEMENT OF WORK

NF-FM7310-23-01304

C.1.0 GENERAL

The Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), National Marine Fisheries Service (NMFS), Northeast Fisheries Science Center (NEFSC) is seeking a contractor to provide and install an image collection system capable of recording from a conveyor belt on the contracted clam survey platform, ESS Pursuit, a commercial clam fishing vessel. The Contractor shall be responsible for the installation of equipment, removal of equipment, equipment maintenance, field support, coordination and meeting with NOAA staff, image review, data analysis, image data set curation, and assist with report writing. The image collection and processing will be the first step in this project. The Contractor shall also develop software solutions that more closely align the video review process currently occurring as part of electronic monitoring (hereafter EM) programs with machine learning based video analysis.

Specifically, we seek a contractor to develop software and documentation in an open manner that both allows for the application of machine learning algorithms to video footage and allows for the further development of annotated training data sets. The goal of this work is to produce tools that can be used on future clam surveys and potentially contribute to EM deployment in the commercial clam fishery. The Contractor personnel will be working under the guidance of NMFS staff, and with project partners in the northeast region.

The project will include project start-up, project scoping, data collection, data analysis, algorithm development, the production of software, and the creation of accompanying documentation, and report writing periods. The project is expected to begin in August of 2023 with the coordination and installation of image collection equipment on the ESS Pursuit in order to collect high quality images of the clam specimens processed by the clam survey. This will include follow-up meetings to coordinate the planning and specification of technical requirements. This will also entail the development of a project plan as well as a project timeline and governance structure. The Contractor will be responsible for developing image collection methods which minimize the impact of image data collection on vessel crew, NMFS survey staff, and shipboard resources. The Contractor will be responsible for pairing the collected image data with species records from the survey data collection system and producing a dataset appropriate for use in machine learning applications. The processing and data annotation of the electronic data will primarily be the responsibility of the Contractor; however, NMFS may assist or review the data as needed. The Contractor will be responsible for developing algorithm based tools for detecting catch items, estimating the length of these items, and performing species identification via machine learning that are suitable for processing video data from clam fisheries in the northeast region. All data collected will be the property of the Government. The Contractor shall work with NMFS staff to develop an image library that can be of use to the broader scientific community.

The Contractor will be expected to leverage existing image libraries for machine learning algorithm and model development if possible. Additionally, the Contractor will be encouraged to build upon existing open source software tools. The development of software solutions that both implement machine learning models and allow for the generation of additional training data will primarily be the responsibility of the Contractor; however, NMFS will assist and provide feedback on development at regular intervals. All software, models, and data (including annotations) that are generated by the project will be the property of the Government. The Contractor shall work with NMFS staff to develop software solutions that can be of use to the broader scientific community.

This development aligns with NMFS’s longer term goal of implementing machine learning in fisheries monitoring to aid in the development of resources with which to enhance the efficiency and accuracy of automated species identification.

The information gained through this study will enhance EM in New England and Mid-Atlantic by providing the means to make EM video review more efficient and thereby a more viable option for collecting fisheries data.

C.2.0 BACKGROUND

C.2.1 Overview

This project will fund placement of cameras and image recording equipment onboard the vessel contracted to conduct the 2023 Northeast Fisheries Science Center (NEFSC) clam survey. The survey is conducted from, and operates like, a typical commercial clam fishing vessel. Survey catches will be recorded and the images stored for analysis. Machine learning will be employed to train image recognition algorithms in recognizing and differentiating the two primary clam species captured in the federal clam fishery and provide the shell length of each individual. Clam species composition and shell lengths are individually measured as part of survey operations; the measured lengths and species identification can be compared to the results of automation to estimate the skill of competing recognition algorithms.

The motivation for this proposal is two-fold. Industry stakeholders are encountering mixed surfclam and quahog catches at increasing rates. Regulations restrict them from landing mixed catches, which leads to both avoidance of desirable fishing grounds and at sea discarding. Managers and stock assessment scientists are interested in measuring both the frequency and composition of mixed catches. In addition, port sampling costs are increasing rapidly, leading to a reduction in the number of commercial length samples available to the stock assessments for each species. This commercial length data is important to both stock assessments since commercial ages are not sampled. If this project is successful, the techniques developed on the survey might be employed in the clam fishing fleet, greatly increasing the availability of catch at length data from both the survey and commercial fishery.

The clam survey is operated annually in August and covers a portion of the Atlantic surfclam or ocean quahog stock area, encompassing the northern Mid-Atlantic Bight and Georges Bank in USA territorial waters. The survey is conducted by randomly assigning survey stations to individual clam strata (Figure 1) and fishing these using a commercial clam dredge.

The dredge is towed for 5 minutes and all Atlantic surfclam and ocean quahog caught are counted, with a subset measured for length and various biological samples taken.

Figure 1. Atlantic surfclam stock area with strata labeled.

C.2.2 Image Library Development

C.2.2.1 Specifics of clam handling and data acquisition

Clams are processed in a streamlined and efficient manner on the survey vessel. Typically, catch moves from the dredge to a sorting device (shaker table) that separates shell and sand from the catch. After being “cleaned” the clams are passed along a conveyor belt (Figure 2) to the scientific staff. Ideally, cameras would be placed above the conveyor belt where clams can easily be photographed. After being photographed, clams will be sampled per standard survey protocols. This includes measurement of all clams when the catch is relatively small and should allow for validation of EM catch composition estimates.

Figure 2. Looking down on the conveyor belt on the ESS Pursuit.

C.2.3 Examples of open source software related to EM

There are a growing number of open source software options in the EM sphere. These include software that is designed for recording and reviewing EM video and sensor data. Perhaps the most notable example comes from the combined efforts of Chordata LLC, Saltwater Inc, and Sea State Inc (see: https://pt.chrdta.com/em/). Open EM is another similar EM review software that is open source (see: https://github.com/openem-team/openem). These tool sets include software solutions that allow for the full lifecycle of EM data collection (from shipboard data acquisition to a review user interface). Additionally, open source software for image and video annotation is also being produced. These are generally distinct software packages that aim to generate data sets on single video streams (see:

https://github.com/cvisionai/Tator, https://github.com/opencv/cvat).

The ideal product of this work would be software that could ‘plug-in’ with existing EM review software. The NMFS data model requires that a 3rd party provider is responsible for the initial review of video data, therefore the end product of this contract ideally would be a product that could interface with the current software used by EM service providers.

These providers generally stream video and allow for review of these data via web portals.

C.2.4 Machine Learning and Image Analysis in the Northeast

There are a growing number of groups interested in applying a machine learning approach to EM nationally. Much of the innovation in this realm is being driven by collaborations between NOAA staff and researchers from the University of Washington. In New England there have been a couple of projects that have tried this using data collected as part of regional EM development. These include, but are not limited to: a recent Kaggle competition organized by the Nature Conservancy (TNC)1; second Kaggle competition aimed at identifying individual right whales organized by the NEFSC2. Additional projects underway utilize data currently being collected as a part of EM pre-implementation monitoring (collaborations between TNC, UMass Dartmouth's School for Marine Science & Technology (SMAST), and a private company CVision Consulting).

In addition to these ongoing projects, a large library of images to train machine learning models was identified as a regional priority and NOAA strategic initiative. Because existing data sets were seen as not sufficient to train models to distinguish the species found in the region, a project was funded to do just this using the NMFS bottom trawl survey.

Training an AI to distinguish the primary commercial clam species should be relatively simple in comparison to fish identification. The species are morphometrically distinguishable (Figure 3) and are also typically (but not always - at least to the human eye) different colors. Measurement of shell length may be somewhat more challenging as the orientation of the clam on the belt can be suboptimal.

1 Please see the Kaggle winnners: http://drivendata.co/blog/fish-winners/ and general competition description:

https://www.drivendata.org/competitions/48/identify-fish-challenge/ 2 Please see this right whale competition: https://www.kaggle.com/c/noaa-right-whale-recognition https://pt.chrdta.com/em/ https://github.com/openem-team/openem https://github.com/cvisionai/Tator https://github.com/cvisionai/Tator https://github.com/opencv/cvat http://drivendata.co/blog/fish-winners/ https://www.drivendata.org/competitions/48/identify-fish-challenge/ https://www.kaggle.com/c/noaa-right-whale-recognition

Figure 3. The shells of the two primary commercial clam species, demonstrating their different shapes and typical colors.

C.3.0 OBJECTIVES

C.3.1 Image Library

NEFSC is proposing to collect images on the 2023 clam survey. Through this project (as well as other ongoing regional projects), NMFS intends to eventually meet the following strategic goals:

● Enhance the efficiency and accuracy of survey and fishery data collection

● Reduce the reporting burden on fishermen

● Reduce the overall cost of fisheries monitoring

This research project aims to develop a library of individual clam specimen images with co-registered specimen information. The aim is to collect as much of this data as possible (as many images and paired metadata as possible), while impacting the survey operations as little as possible. To achieve these objectives, NMFS has identified the following Contractor specific project deliverables:

● Through an iterative process, develop the means for image data collection. In collaboration with NEFSC, make sure the process is amenable to NEFSC survey staff.

● Test camera image collection systems to ensure proper function and image quality standards are maintained for machine learning. These will include making sure lighting is adequate regardless of ambient illumination and that focal distance is adequate for capturing the entire conveyor belt.

● Develop protocols for camera maintenance and failure (including necessary redundancies to ensure data generation for the duration of the project period).

● Perform installation of a camera system capable of collecting data in an optimal position above the conveyor belt on deck, in order to allow for individual length estimation.

● Collect high quality images and associated data for as large of a proportion of the specimens as possible during clam survey operations.

● The images should be clear, with minimal glare. The animals should ideally be exposed to the camera in a manner that allows for individual length estimation.

● The images should be stored on the vessel until they can be sent to data warehousing facilities.

● Develop local infrastructure to support data collection between survey cruise segments.

● Develop a process with which to pair collected image data with specimen data collected.

● Work with NEFSC to package data for image analysis.

● Develop necessary licensing documentation to make both image and specimen data widely available.

C.3.2 Algorithm Development

NEFSC is proposing to develop algorithms and software solutions to help ease the burden of EM video review. The machine learning components are expected to utilize the existing image data sets and if feasible work with existing open source EM review software (e.g., https://pt.chrdta.com/em/). These software solutions produced by this work are expected to be open source. Through this project (as well as other ongoing regional projects), NMFS intends to eventually meet the following strategic goals:

● Enhance the efficiency and accuracy of survey and fisheries data collection

● Reduce the reporting burden on fishermen

● Reduce the overall cost of fisheries monitoring

● Expand upon current image library and algorithm development

This software development project aims to develop novel software and algorithm solutions for EM. The goal is to leverage existing data sets as much as possible for algorithm development, while also highlighting future image training set needs. An additional focus is to utilize open source software to bring together EM video review and machine learning data set annotation. To achieve these objectives, NMFS has identified the following Contractor specific project deliverables:

● Annotate sufficient images for training artificial intelligence in discriminating between ocean quahogs and

Atlantic surfclams.

● Either develop or leverage existing machine learning software to differentiate between ocean quahogs and

Atlantic surfclams using the annotated training images.

● Either develop or leverage existing machine learning software to estimate the lengths of individual clams in images.

● Develop workflow and procedure for taking packaged images through species identification and length estimation algorithms and producing counts and catch at length data for use in future analysis.

● Automate the above workflow to the extent possible.

● Document products and appropriate performance metrics in a final report that allows for further development.

C.4.0 SERVICE REQUIREMENTS

C.4.1 Project Phases

C.4.1.1 Image Library Project Setup: a period of up to two (2) months (depending on the date of the contract award and the dates that the clam survey is scheduled) involving the installation requirements described in Sections 4.2 (Project Initiation), 4.3 (Camera and Equipment Installation), 4.4 (Servicing of Equipment Aboard Vessels, and Field and Project Support) and 4.5 (Data Curation and Library Development). The activities provided in Section 4.4 relevant to this phase https://pt.chrdta.com/em/ are specifically related to establishing local capacity and infrastructure to provide and support field services. The activities in Section 4.5 relevant to this phase include development of a project plan. Other activities include clarifying project roles and responsibilities and formulating the project team.

C.4.1.2 Algorithm Development Project Setup: a period of up to three months (depending on the date of the contract award) involving project scoping, consultation and development of algorithms and software plan requirements described in Sections 4.2 (Project Initiation). Additional tasks related to establishing project roles and pre-emptive work needed to complete sections 4.3 (Algorithm Development), 4.4 (Bringing Machine Learning to EM Review), 4.5 (Using EM Review to Develop Machine Learning Training Data), and 4.6 (Final Report and Other Documentation).

C.4.1.3 Image Library Data Collection Period: (initiated approximately two (2) months post contract award and coordinated with clam survey dates) involving the service requirements described in Sections 4.4 (Servicing of Equipment Aboard Vessels, and Field and Project Support), 4.5 (Data Curation and Library Development), and 4.6 (Removal of equipment from the survey vessels).

C.4.1.4 Algorithm Development: Algorithm and software development period (initiated approximately three (3) months post survey legs) involving the development of algorithms in Sections 4.3 (Algorithm Development), 4.4 (Bringing Machine Learning to EM Review), and 4.5 (Using EM Review to Develop Machine Learning Training Data).

C.4.1.5 Analysis and Reporting: two (2) month period (initiated twelve (12) months post contract award) involving the service requirements described in Section 4.7 (Final Report and Data Set Finalization).

C.4.2 Project Initiation

C.4.2.1 The Contractor shall participate in regular contact with appropriate NMFS staff during equipment installations, data retrievals, equipment service events, algorithm and software planning phases, or as required. Frequent and consistent communication with NMFS staff will help build a good working relationship, reinforce commitment (NMFS staff and Contractor), and will serve to address any outstanding questions or concerns the vessel crew and NMFS survey staff may have. The Contractor and NMFS will also use these opportunities to describe project goals, provide feedback on equipment performance, evaluate the progress towards project objectives since the last meeting, provide feedback on the state of software and algorithm development, and convey study results. Contact will be conducted in this manner to ensure that clear software requirements are developed and to ensure that the ultimate goal of this project is clear to the Contractor. The Contractor will track all contacts and document the purpose of each event in a study service tracking sheet, which will be accessible to NMFS.

C.4.2.2 The Contractor shall also participate in occasional study coordination and communication meetings with NMFS staff, vessel crew, and potentially other stakeholders throughout the course of the contract. Meetings will be led, organized, and scheduled by NMFS. Outreach may include equipment testing/demonstration components of the image collection system, software demos, sharing algorithm source code, and detailed discussions about algorithm development and improvements.

C.4.2.3 The Contractor shall engage in regular communication (bimonthly or more frequently as required) with NMFS staff (in person or by conference call) to review monitoring objectives, data requirements, project plans, project deliverables, project spending and budget plans (quarterly), and staffing and resources.

C.4.2.4 The Contractor shall lead equipment development (with the assistance of NMFS). Specifically, the Contractor shall perform an iterative process of equipment testing and coordination with NMFS staff. Equipment must be developed and installed such that the current sampling processes are maintained. Equipment must be developed such that redundancies ensure adequate data collection. The Contractor shall provide a detailed description of their image collection system design and its components in their quote and as part of their reference sheet. The Contractor must also include a detailed description of how the images will be paired with survey specimen data. The Contractor shall provide a protocol for troubleshooting common system problems.

The Contractor shall lead the planning phases of algorithm and software development (with the assistance of NMFS).

Specifically, the Contractor shall perform an iterative process of coordination with NMFS staff to ensure that the algorithms and software being developed will have the greatest utility feasible given the existing constraints (data, funding, and time available). Algorithms must be developed and implemented such that they provide data comparable to that currently recorded by humans doing EM review. The Contractor shall provide a detailed description of their algorithm and software development plan and its components in their quote and as part of their reference sheet.

Additionally, the Contractor shall lay out the general plan for software development that specifies key attributes of the software including where the software will run (in the cloud or locally). Finally, the Contractor should explicitly outline how the accuracy of algorithms will be assessed by users in applications where they are applied.

C.4.3 Camera and Equipment Installation

The Contractor shall provide a camera system necessary to capture specimen images at-sea. The Contractor shall also have adequate spare or backup equipment available locally in case of a system failure or equipment malfunction. The vessel installation locations shall be determined through consultation with NMFS staff and vessel crew. Preparation, development, and deployment of camera systems may need to be done with limited access to the survey vessel as the vessel's schedule and commercial fishing commitments may prohibit extensive access prior to installation (for example, there may only be access to the vessel for one to two days prior to deployments).

The camera system supplied by the Contractor shall be approved by NMFS staff as appropriate quality for the development of the image library. Suggested guidelines include a raw image format or a jpeg compression that avoids the loss of image quality (not H.264) as well as at least 2/3 size image sensors (2.3-5 megapixels [MP]) to enhance overall data density. Specimen lighting must also be approved by NMFS and interested parties as appropriate for the ultimate goals of the study. Maintaining as consistent lighting as possible is desirable as variable lighting introduces complexities into the machine learning process. Equipment used for specimen imaging and illumination should be constructed in such a manner that they are robust and able to withstand spray with salt water (e.g., at least IP65 rated or equivalent). Data storage must also be approved by NMFS staff and interested parties as appropriate for the study and compliant with IT infrastructure. All equipment must be ruggedized to withstand the conditions present in catch processing locations.

In considering camera systems for this project the Contractor must pay close attention to the above clam processing section of the background (section C.2.2). Specifically, the positioning of the cameras and the frequency of image capture must be tuned so as to capture images at the rate that specimens are processed (which can occur rapidly).

Positioning of the clams on the conveyor may pose a challenge if the catch is large as clams may pile up and may orient vertically such that species identification and shell length measurements are challenging. Camera placement, lighting and image quality should seek to mitigate these effects to the extent possible.

C.4.4 Servicing of Equipment Aboard Vessels, and Field and Project Support

Prior to commencement of the active data collection component of the project, the Contractor shall set up local capacity to install equipment and provide field support to the survey vessel. The field support shall include; equipment installation, equipment or system troubleshooting, data retrievals (by means of in-person retrieval), and equipment removal. The Contractor shall provide a local service technician who is capable of providing all servicing events and needs on the short time frames associated with the beginning and end of NMFS survey legs. Survey legs have very rapid turnaround times typically, usually less than 12 hours. The data retrievals and equipment assessment shall occur after each survey leg no more than four (4) hours after trip landing if possible. The Contractor shall present a formal plan on servicing, local capacity, and response time to address any equipment issues in their quote.

The Contractor shall create and maintain a study service tracking log to include all development, installation, service events, data retrievals, equipment or system troubleshooting and equipment removals. If equipment servicing or troubleshooting occurs the Contractor shall note the issue, length of time to fix the issue, and how the issue was rectified in the service tracking log. The tracking sheet shall be updated daily and the Contractor shall provide full access to NMFS. The Contractor shall also track all system failures (any instance where the equipment is not functional) and report all system failures to NMFS as soon as they are discovered.

The Contractor shall install, test, and initialize all the study equipment on the NMFS survey vessel. The field service technician may be required to go out to sea on a trip with the vessel to understand sample processing operations and evaluate vessel configuration to meet project goals. Should the field service technician be required to go to sea they may be required to assist with clam processing and other cruise duties. In the rare event that the ESS Pursuit be unable to operate such that the cruise is delayed or potentially conducted on another fishing vessel the Contractor must be prepared to deploy after a delay on a similarly sized fishing vessel.

It is important to note that the deployment of the image collection equipment will likely have to occur over a very short period of time (potentially as little as hours immediately prior to the cruise). This is because very limited time is available for even the survey crew to put out equipment (likely between one and two days). These logistic challenges need to be met by the Contractor.

The Contractor shall provide instruction on the use and general care of the EM equipment to the vessel’s crews. The Contractor shall provide a detailed description of what is involved in the setting up and removal of equipment to NMFS.

The Contractor shall develop a vessel reference sheet which provides the description of the equipment, outlines NMFS crew responsibilities, and includes instruction on the use and care of the equipment. The vessel reference sheet is required prior to the first deployment of the equipment.

The Contractor shall develop a clear protocol for image and specimen data collection and pairing. The protocol shall outline suggested specimen handling protocols and camera equipment configurations being used throughout the project.

The study protocol document is a communication tool used to ensure that NMFS staff, Contractors, field technicians, and project coordination staff know what their roles are for a successful project implementation. The protocol document will include at a minimum; general image collection procedures, camera system configuration, operator responsibilities, protocols for data retrieval, specimen handling protocols, general equipment information, basic camera and data system troubleshooting, project objectives and contacts, and any other pertinent information.

The Contractor’s quote shall include suggested solutions or arrangements to address power issues and data storage issues. Because the purpose of this study is to collect a large number of high quality images, data storage must be sufficient to ensure data collection over the full course of a survey leg. Data storage decisions must be approved by NMFS staff and not impact the IT infrastructure of the survey vessel. The Contractor shall coordinate and determine a schedule for equipment installation on each vessel and will take into account NMFS staff and the survey vessels’ schedules and concerns. Again, the installation and maintenance of the study equipment shall not impact the operations of the survey crew or vessel in any substantive way.

C.4.5 Data Curation and Library Development

The aim of this study is to collect as many annotated images as possible, thus the Contractor shall ensure data collection is maximized during the study period. This includes maximizing the number of images of unique specimens as well as maximizing the connection to relevant collected specimen information.

The Contractor shall curate the image and specimen data sets to remove poor quality images, or images for which specimen data are extracted from the final data set. The process for this curation will be documented so as to communicate the methods to NMFS and potential data end users. A key component in this data curation will be pairing the collected image data with specimen information collected. Matching may be achieved via a number of means, however image data must be collected such that it is time stamped and could be paired with specimen data via the matching of timestamps (should other methods fail).

During the project period NMFS will provide the Contractor with exact specifications for the final data set, but generally it will be a set of images and data labels suitable for machine learning applications. The Contractor shall provide NMFS with the highest quality digital images appropriate for later manipulation. The Contractor shall provide NMFS with data summaries from captured data and provide complete access to all data (both specimen and image data) for NMFS review. It is preferred equipment developers and data processing technicians have experience with machine learning and automated image processing or plan to partner with a vendor with this experience. Summary data provided to NMFS and images submitted for NMFS review purposes under this contract will be the property of NMFS.

The Contractor will work with vendors to determine the best licensing for this data set to encourage use by parties interested in developing algorithms for automated image analysis. Additionally, the Contractor shall determine an appropriate and effective means for distributing the large data set. NMFS must approve all licensing decisions and dissemination means.

C.4.6 Removal of equipment from the survey vessel Contractor shall be responsible for removing the study equipment from the vessel at the end of the project. The Contractor shall remove all evidence of the equipment installation and care must be taken to return the vessel to the state it was in prior to the installation of the study equipment. The removal shall not unduly delay the survey vessel from departing for other cruises.

C.4.7 Algorithm Development

The Contractor shall develop algorithms using industry standard open source tools to meet the needs of EM video review. These algorithms will meet the goals identified through consultation with NMFS during the project initiation (4.2). Examples of tasks that may be required include (but are not limited to): 1) species identification 2) the estimation of length of each individual, and 3) counts of each species in each completed tow. The Contractor shall follow the best open source software development protocols including modern version control. The algorithms will be made such that they can be updated as simply as possible as new data becomes available. Utilization of modern machine learning techniques is essential for this task.

C.4.8 Final report

The Contractor shall provide NOAA Fisheries with a draft and the final report on this project in a format to be approved by NMFS. The draft report is due one (1) month prior to the final report. The draft shall allow NOAA Fisheries to provide comments and edit text. The final report, incorporating changes suggested by NOAA Fisheries, shall be clear, straightforward and will incorporate an overview of the research project, analysis of the research data, algorithm developed, the software produced, lessons learned and next steps in using automated image processing to improve the efficiency of fisheries monitoring, lessons learned, and next steps in using automated image processing to improve the efficiency of fisheries monitoring.

The final report shall be a narrative of the operational components and processes for the study. Specific data analysis and reporting included in the final report shall include but is not limited to the following:

● Document and detail all operational components (e.g., data management and storage, data retrievals and management, equipment maintenance and supporting field services, data processing, onboard methodologies, feedback mechanisms, etc.), necessary to support data capture during the study.

● Document procedures (list of necessary tasks to complete processes), description of protocols (training, skill sets, etc.), data quality procedures and considerations, and timeframes and logistical considerations associated with data collection;

● Identify any significant issues or concerns with image collection on the survey vessel, including: storage concerns, difficulty pairing specimen and image data, as well as any other logistical considerations that impact image quality.

● Document “best practices or best recommendations” for a successful and efficient means to collect images during a survey cruise. The “best practices” information shall be supported by data collected, analyzed, and products developed as part of this study. Supporting data shall be utilized to collaborate and reinforce “best practice recommendations” and shall be incorporated in a “lessons learned” format. This information shall include methods tested and evaluated (efficiencies) as part of this study, including elements such as: data alignment, modified catch handling, distinguishing between discarding events, methods to address undetected discard events, techniques for industry reporting, etc. The Contractor shall also incorporate as applicable “lessons learned” from other pilot studies or operational programs which they have conducted or are associated with;

● Additionally, identify and document how this type of data set can be leveraged to analyze data collected from fishing vessels. Specifically, how camera angle and lighting impact the data set and the next steps in moving toward using this technology for further research and monitoring.

● Summarize all data collected. Summarized information shall include material such as details of collected data (e.g., number of survey legs, estimated total numbers of images captured, and number of images per species in the full data set).

C.4.9 Ownership of materials

All reports, summaries, and information collected or developed during this project and submitted to NMFS as required, and collected image and sensor data, will be the property of NOAA Fisheries and will be maintained onsite by the NMFS. All data will be delivered in an accessible manner to NMFS. The data collected throughout this project may be used for outreach purposes, training (video auditors, NMFS staff, observers, etc.), and presentations as needed. The

Contractor will be required to store all (specimen and image) data for three (3) years from the project start date.

C.5.0 SCHEDULE OF DELIVERABLES

Following is a schedule of all deliverables, including administrative deliverables, required during the period of performance of this contract:

Item Description Quantity Due Date Deliver To:

Reference

1 Development and installation of image data storage equipment (able to collect images at all four stations)

1 units Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TECHNICAL

POINT OF

CONTACT

(TPOC)

Section C, C.4.1.1

2 Local infrastructure to support vessel installation, servicing, and program support

1 1 month after award TPOC Section C, C.4.4

3 Vessel outreach while in port

As needed As required TPOC Sections C, C.4.2.1, C.4.2.2

4 Program outreach meetings

As needed As required TPOC Section C, C.4.2.2

5 Planning document for algorithm and software development

1 units Up to 3 months after award

TPOC Section C, C.4.2.4

6 Schedule for algorithm and software delivery of final data summary

1 Up to 3 months after award

TPOC Section C, C.4.2.4

7 Develop equipment protocols

1 per sampling station

Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Section C, C.3.0, C.4.4

8 Processing and data matching

As needed As data becomes available

TPOC Section C, C.4.5

9 Spending and project budget reports

1 Quarterly over the course of the contract

TPOC Section C, C.4.2.3

10 Study service event log and reports

1 Daily access (via email, google sheets, or similar product)

TPOC Section C, C.4.2.1, C.4.4

11 Report all system failures

1 As soon as possible TPOC Sections C, C.2.1 C.3.1 C.4.3

12 Machine learning algorithms

1 By project conclusion TPOC Section C, C.4.7

13 Software for using algorithms to review EM footage

1 By project conclusion TPOC Sections C, C.4.8 C.4.9

14 Inventory of all code libraries, third party software tools, algorithms, models, scripts, etc that are required to run along with license type and versions (like “Google TensorFlow, v2.1, GPL”) should be included

1 At project completion TPOC Section C, C.4.8

15 Software and algorithm documentation and algorithm/software evaluation

1 By project conclusion TPOC Sections C, C.3.1 C.4.2.1 C.4.8 C.4.9

16 Software demos and other collaborative meetings

As needed As required TPOC Section C, C.4.2.2

17 Detailed software requirements for future developers

1 By project conclusion TPOC Section C, C.4.9

18 Project meetings with

NMFS

1 Likely every week or more frequently if required

TPOC Sections C, C.4.2.1 C.4.2.2

19 Study protocol reference sheet as well as other material to orient vessel crew

1 per sampling station

Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Sections C, C.4.2.4 C.4.4

20 Data retrieval (per leg basis)

As needed Within 4 hours of survey leg landing

TPOC Sections C, C.4.2.1 C.4.4

21 Instructions and schedule for collection of data and retrieval of hard drives

1 Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Sections C, C.4.2.1 C.4.4

22 Detailed description of equipment installation and removal vessel requirements (potential and actual)

1 Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Sections C, C.3.1 C.4.1.1 C.4.2.1 C.4.3 C.4.4 C.4.6

23 Experimental plans for data collection

1 Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Sections C, C.4.1 C.4.5

24 Tracking sheet for data inventory

1 Daily access (via email, google sheets, or similar product)

TPOC Sections C, C.4.2.1 C.4.4

25 Schedule for data processing and delivery of final data summary

1 Up to 2 months after award (or prior to the beginning of the August 2023 Survey, whichever occurs first)

TPOC Section C, C.4.5

26 Removal of all equipment

1 Study completion TPOC Sections C, C.4.1.3 C.4.4 C.4.6

27 Draft Report 1 1 month prior to due date for final report

TPOC Section C, C.4.10

28 Final Report 1 At project completion TPOC Section C, C.4.10

29 Delivery of all raw data and final data products

1 2 months following completion of project

TPOC Section C, C.4.11

30 Storage of data (sensor and image)

1 For three (3) years from project start date

TPOC Section C, C.4.11

C.6.0 PLACE OF PERFORMANCE

Northeast Fisheries Science Center (NFSC) 166 Water Street Woods Hole, MA 02543

C.7.0 PERIOD OF PERFORMANCE

The period of performance is 12 months following date of award.

C.8.0 GOVERNMENT POINT OF CONTACT

The government TPOC:

Daniel Hennen

(508) 495-2398 Daniel.Hennen@noaa.gov mailto:Daniel.Hennen@noaa.gov