B08_Attach_B_QASP.pdf

Quality Assurance Surveillance Plan (QASP)

For Great Lakes Acoustic Fisheries Data Collected by Unmanned Surface Vehicle (USV)

1. Vision To provide the highest quality data about Great Lakes pelagic prey fish abundances and biases associated with traditional vessel-based fisheries assessment.

2. Mission The primary responsibility of the US Geological Survey (USGS) is to assist natural resource and land managers by providing them with sound biological information and assistance in applying the information to their needs. The primary means of gathering this information is by using scientific methods applied to monitoring resources and conducting experiments. The USGS Great Lakes Science Center is responsible for implementing a science program focused on assessment of prey fish abundances and other research that supports decision making by Great Lakes state, tribal, and provincial fisheries management agencies.

3. Purpose This Quality Assurance Surveillance Plan is a government-developed document used to determine if the contractor’s performance meets the performance standards contained in the contract. The QASP establishes procedures on how this assessment/inspection process will be conducted. It provides the detailed process for a continuous oversight process:

• What will be monitored

• How monitoring will take place

• Who will conduct the monitoring

• How monitoring efforts and results will be documented

The contractor is responsible for implementing and delivering performance that meets contract standards using its Quality Control Plan. The QASP provides the structure for the government’s surveillance of the contractor’s performance and their Quality Assurance/Quality Control (QA/QC) actions to assure they meet contract standards. It is the government’s responsibility to be objective, fair, and consistent in evaluating contractor performance.

The QASP is not part of the contract nor is it intended to duplicate the contractor’s quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on contractor performance.

The government may provide a copy of the QASP to the contractor to facilitate open communication. In addition, the QASP should recognize that unforeseen or uncontrollable circumstances might occur that are outside the control of the contractor.

Bottom line, the QASP should ensure early identification and resolution of performance issues to minimize impact on mission performance.

4. Authority Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the contracting officer or a duly authorized representative.

5. Roles and Responsibilities The following personnel shall oversee and coordinate surveillance activities.

Program/Project Manager (PM) or Functional Services Manager (FSM) – The PM/FSM provides primary program oversight, nominates the COR, ensures the COR is trained before performing any COR duties and supports the COR’s performance assessment activities. While the PM/FSM may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, they are not empowered to make any contractual commitments or any contract changes on the government’s behalf.

Contracting Officer (CO) – The CO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The CO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Assigned CO: Irma Suryaty Organization or Agency: USGS Telephone: 303-236-9318 Email: isuryaty@usgs.gov

Contracting Officer’s Representative (COR) The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance.

The QASP is the primary tool for surveillance of the contractor’s quality program and help the COR to document contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Contractor Representatives

The following employees of the USGS serves as the contractor’s Program Manager for this contract.

Contractor Program Manager – Peter Esselman Telephone: 734-214-7267 Email: pesselman@usgs.gov

COR – Rhonda Aton Title: Administrative Operations Assistant Telephone: 734-214-7205 Email: raton@usgs.gov

6. Performance Requirements and Method of Surveillance

6.1. Contract Surveillance

The goal of the QASP is to ensure that contractor performance is effectively monitored and documented. The COR’s contribution is their professional, non-adversarial relationships with the KO, PM and the contractor, which enables positive, open and timely communications. The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of contractor performance against contract requirements. The COR uses the methods contained in this QASP to ensure the contractor is in compliance with contract requirements. The COR function is responsible for a wide range of surveillance requirements that effectively measure and evaluate the contractor’s performance. Additionally, this QASP is based on the premise that the contractor, not the government, is responsible for management and QC/QA actions to successfully meet the terms of the contract.

6.2. Surveillance Matrix

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance and frequency the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance.

The Government performs surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The Performance Requirement Summary (PRS) should be used to form the foundation of the COR’s inspection checklist.

6.3. Performance Rating Definitions

The performance ratings below reflect definitions at FAR 42.1503 Table 42-1. The COR will use these rating to evaluate the quality of contractor’s performance.

Performance Rating

Definition

Exceptional Performance meets contractual requirements and exceeds many to the government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

mailto:raton@usgs.gov

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

Note 1: Plus or minus signs may be used to indicate an improving (+) or worsening (-) trend insufficient to change the evaluation status.

Note 2: N/A (not applicable) should be used if the ratings are not going to be applied to a particular area for evaluation.

Appendix 1 - Surveillance Matrix

Statements Standards/AQLs Inspections Ratings C.3.1 USV sensor integration

C.3.1.1 The acoustic system used by each USV(s) must consist of a gimbled four-sector split-beam echosounder with a resonant frequency of 120 kHz, a circular beam width of 7 degrees, and a depth rating of at least 20m. Prior to the data collection mission, the Contractor shall collect echosounder test data with each USV in water depths of at least 50 m and provide these data to GLSC for quality verification.

Deliverables:

A01 Pre-mission quality test datasets A02 Post-mission datasets

Noise during science mission at 1m less than or equal to -134 dB to allow discrimination of individual organisms with a target strength > - 66 dB in 150 m of water

AQL: Noise level achieved 97% of the time while operating

What: Raw acoustic data from split beam echosounder How: 100% review of echosounder test data prior to mission and post-mission data.

Continuous observation Customer feedback trouble reports Who: Program Manager, and COR Incentive: Deduction of mission day fees in proportion to number of hours that data failed to meet the -134 dB at 1 m standard (excepting 3% of the mission duration).

C.3.1.2 The 120 kHz split-beam echosounder will be calibrated using the standard

System calibrations should collect sufficient data to measure on-axis sensitivity and variations in beam pattern following methods described

What: .raw files, Echoview files, and summary reports from calibration process.

How: 100% review of echosounder calibration data prior to mission and post-mission data.

sphere method under the in-person or remote observation of staff scientists from GLSC.

Deliverables:

A03 Emailed calibration data prior to mission start and after mission end.

in Foote et al. (1987) and ICES Cooperative Research Report No.

326. To measure on-axis sensitivity, a minimum of 1,000 targets should be collected from a standard sphere within 0.5 degrees of beam center.

To measure beam pattern consistency, an additional 500 targets should be collected from a standard sphere in each split-beam quadrant, distributed evenly within the half-power beam angle.

Therefore, a single calibration routine should include a minimum of 3,000 pings. Calibration data should be collected pre- and post- mission for each collection ensemble in a freshwater environment. Calibration summaries should be performed using Echoview's calibration assistant.

AQL: Acceptable accuracy will occur when the post-calibration mean compensated TS falls within +/-0.1 dB of the standard sphere with no concerning deviations in beam pattern.

Customer feedback trouble reports Who: Program Manager, and COR Incentive: Delay of mission and re-calibration until calibration is deemed sufficient.

C.3.1.3 The contractor shall ensure that onboard sensors in addition to the acoustic

Sensors are demonstrated to be functioning and accurate.

What: Pre-mission verification data Post mission dataset How: 100% review of sensor data prior to mission and post-mission data.

system (see C.3.1.1) are calibrated and perform to standard. These include sensors for wind direction and speed, PAR, cameras, conductivity, temperature, depth, dissolved oxygen, fluorometrically inferred chlorophyll a, barometric pressure, sea surface temperature, GPS position, and inertial measurement (roll/pitch/yaw).

Deliverables:

A04 Post-mission datasets

AQL: GPS and inertial navigation systems function 99% of the mission duration; other sensors function 75% of the mission duration.

Customer feedback trouble reports Who: Program manager, and COR Incentive: Deduction of mission day fees in proportion to number of hours that data failed to function excepting 1% of GPS and inertial data, and 25% of data from other sensors (not including acoustic system which is addressed in 3.1.1).

C.3.2 Completion of USV science mission

C.3.2.1 The USV(s) shall be commanded to follow paths between a series of waypoints provided by GLSC to sample fisheries and near-surface atmospheric conditions in open waters. Chosen paths and locations shall be adapted during the mission, with up to two adjustments per day during normal business hours (including weekends) to accommodate science needs.

Contractor provides two daily opportunities to adjust USV routes within normal business hours for duration of mission.

AQL: Service level achieved 95% of the mission duration

What: Twice-daily input on USV tasking during business hours How: Ongoing Customer feedback trouble reports Who: Program Manager, and COR Incentive: Deduction of mission day fees in proportion to number of days that twice-per day updates were not available excepting 5% of mission duration.

Piloting shall be done remotely, ensuring the USVs do not interfere with other vessel traffic or endanger human life, while also meeting the science goals of the mission.

3.2.2 One USV will be launched, operated for 56 days of data collection, and recovered on each of Lake Erie and Lake Michigan/Huron (two USVs for a total of 112 mission days) collecting data continuously when USV power allows.

Deliverables:

A02 112 mission days of data provided in post-mission datasets

USVs fulfill full mission requirement.

AQL: 100% of committed mission days (112 days)

What: Raw acoustic data from split beam echosounder How: 100% review of post-mission data.

Customer feedback trouble reports Who: Program Manager, and COR Incentive: Deduction of mission day fees in proportion to time that Contractor failed to collect data.

3.2.3 The Contractor shall provide a user portal for real-time display of the USV(s) position(s) as well as plotted data for the full duration of scientific missions.

Deliverables:

A05 Online, web-accessible user portal

USVs fulfill full mission requirement.

AQL: 95% of committed mission days (112 days)

What: Access to near-real-time portal showing mission progress and recent data.

How: Continuous observation during mission Customer feedback trouble reports Who: Program Manager, and COR Incentive: Deduction of mission day fees in proportion to number of hours that user portal was not available (excepting 5% of the mission duration).