Attachment_10-Data_Quality_Monitoring_and_Improvement_Measures.pdf

NEFSC Fisheries Sampling Branch Policies Data Quality Monitoring and Improvement Measures The National Marine Fisheries Service (NMFS) Northeast Fisheries Science Center (NEFSC) Fisheries Sampling Branch (FSB) collects, maintains, and distributes data for scientific and management purposes in the northwest Atlantic Ocean. FSB manages three separate but related observer programs: the Northeast Fisheries Observer Program (NEFOP), the Industry Funded Scallop (IFS) Observer Program, and the At Sea Monitoring (ASM) Program. Here, "observers" refers to any observer/monitor working for the FSB.

Monitoring Data Quality:

FSB considers data quality a critical component of the mission of all observers and observer programs. Poor quality data may result in unreliable or biased estimates of fishery activity and reflects negatively on the program. The Data Quality Lead, Data Editors, and other FSB staff are tasked with ensuring that all data collected by FSB observers is of the highest quality possible.

When an observer's data collection or performance does not meet FSB standards, one or more of the following actions may be taken.

Shadow Trips:

If an observer's data quality is questionable (e.g., poor organization, difficulty sampling catch, etc.), a shadow trip may be requested. The Data Quality Lead will communicate the need for a shadow trip to the observer service provider, and assist in scheduling the trip. An experienced FSB staff member would accompany the observer on a trip (typically a single-day trip on a specified gear type) to review and assess their methods. Refer to the Shadow Trip Program document for more details.

Priority status:

Priority status is used when there are concerns about an observer’s understanding of key concepts. It is an opportunity to provide feedback to the observer before the error is repeated on subsequent trips. Priority status must be approved by the Data Editor Lead, who will communicate with the observer and service provider about the reason(s) for priority status and the expected length of priority status. If trip-by-trip reviews are required, the observer may not redeploy on another trip until the priority trip has been fully edited and debriefed. Trips will be edited immediately, and the observer will be debriefed as soon as possible. The Data Editor Lead will send out status emails after each trip, as well as the final notification when an observer goes off priority status. Priority status may apply to only specific trip types or circumstances (e.g., gear type, trip duration, incidental take of a protected species, etc). In those cases, the observer may redeploy on back-to-back trips until the specified trip type or circumstance occurs.

If at the end of the priority period (maximum five trips), the observer has not rectified the issues noted, or has another egregious error, the observer will be reevaluated and may be recommended for probation status. If the observer successfully completes their priority trips with no further problems, they may resume taking trips in the standard manner. If the issues noted are repeated

Attachment 10

EA133F-17-RP-1824

SECTION J

within three months from the completion of their priority trips, the observer will be reevaluated and may be recommended for probation status.

Probation:

Probation status is used when there are serious data quality concerns or a serious breach of protocols. Probation status is the last chance for an observer to show improvement before decertification. Probation status must be approved by the Data Quality Lead, who will write a specific plan for the probation period. The plan must be approved and signed by the Branch Chief, and electronic and hard copies will be submitted to the observer service provider. A minimum of three trips will be planned, and specific trips (e.g., gear type, trip duration) or other measures (e.g., trip-by-trip approval, submitting notebooks, in-person debriefings, etc) may be required. If the probation plan requires trip-by-trip reviews, the observer may not redeploy on another trip until the probation trip has been fully edited and debriefed. Trips will be edited immediately, and the observer will be debriefed as soon as possible. The Data Quality Lead will send out status emails after each trip, as well as the final notification at the completion of the probation period.

If at the end of the probation period, the observer has not met the requirements of the probation plan, the probation period may be extended (maximum five trips), or the observer may be recommended for decertification. That determination will be made by the Data Quality Team, and all recommendations for decertification must be approved by the Branch Chief. If the observer successfully completes their probation trips with no further problems, they may resume taking trips in the standard manner. If the issues noted are repeated within three months from the completion of their probation trips, the observer will be reevaluated and may be recommended for decertification.

An emergency probation may be invoked at anytime if there was reason to believe some egregious violation of procedures, protocols, or Standards of Conduct had been committed. This action will be carried out by the Branch Chief following discussions with the Data Quality Team, including the Data Quality Lead. The Data Quality Lead will notify the Observer Contract COTR, the observer provider Program Manager, and the observer of their decision in writing.

Typically, this is a short-term probation pending an investigation as to the nature of the offense.

Decertification:

If there is no discernible improvement in the observer’s performance during the probationary trips, the observer will be decertified by the Branch Chief. Decertification will be communicated verbally and in writing by FSB staff to the Observer Contract COTR, the observer service provider Program Manager, and the observer.

An observer will automatically be decertified for failure to conform to the Standards of Conduct signed during observer training. Providing false statements to the government is illegal. An observer will be decertified if they falsify data. Falsification is defined as: The act of deliberately or knowingly fabricating data collected during observed fishing trips, this includes intentional recording of inaccurate data, intentional omission or deletion of data, intentional plagiarism, or, in general, the selective alteration of data.

The decertification procedure is subject to appeal in writing from the observer to the Branch Chief. The Branch Chief’s determination of decertification is final. Decertification would disqualify an observer from re-applying to work as an FSB observer in any program.

Attachment 10

EA133F-17-RP-1824