B08_140G0223Q0112_Attachment_3_SAMPLE_Mineral_Audit.pdf

PERFORMANCE-BASED WORK STATEMENT

MINERALS LABORATORIES AUDIT

Task I: Quality Management System Audit of USGS Mineral Resources Program Laboratories

1. Summary

Onsite audits of the Minerals Resource Program (MRP) laboratories and their compliance with the Energy and

Minerals Mission Area (EMMA) Quality Management System (QMS) shall be conducted. The Contractor shall perform the audits based on the requirements in the current controlled version of the EMMA Quality Assurance

Manual (QAM), the QMS Quality Assurance Requirements (QARs), and MRP laboratory Standard Operating

Procedures (SOPs).

The audits will commence following approval of the task order. Laboratories to be audited are in Lakewood, CO, Reston, VA, Menlo Park and Stanford, CA, Spokane, WA, Tucson, AZ, and Anchorage, AK. On-site audits shall take place during the time agreed to by the Contractor and the USGS Technical Liaison. The contractor’s representative and the USGS Technical Liaison will agree to a final schedule based on estimated number of days for each location. Laboratory audits use a set of checklists that are applied uniformly at each location based on

QMS requirements.

2. Requirements

The Contractor shall provide auditing services in accordance with Indefinite Delivery/Indefinite Quantity (IDIQ)

Contract Number 140G0223DXXXX to complete Task I described herein. Contractor auditing personnel’s qualifications and experience shall be in accordance with the base IDIQ contract requirements. The Contractor shall conduct on-site audits for each location described herein. If there are extenuating circumstances for a particular laboratory where an on-site audit cannot be performed, these situations shall be discussed during the task order kick-off meeting and arrangements will be made for the Contractor to perform a virtual audit. The Contractor shall report the results of the audits as identified herein.

2.1 Task Order Kick-off Meeting

Contractor shall schedule and attend an initial task order kick-off meeting with USGS Technical Liaison and the

QMS Manager. Other individuals from the USGS management may attend the meeting, the Contractor will be notified of attendees prior to the meeting. The goals of the task order kick-off meeting are to discuss the logistics for each audit location (audit date, audit length, labs to be audited, safety concerns for each site, and auditor needs at each site). The task order kick-off meeting shall be scheduled no later than fifteen (15) days after task order is awarded.

The task order kick-off meeting may be accomplished in-person or virtually. Outcomes from that meeting include a tentative schedule for each location and a tentative schedule for the entire audit. The Contractor and the USGS

Technical Liaison will agree to a final schedule for all locations. All on-site audits shall be completed within 16 weeks of the initial audit. As a result of the kick-off meeting, the Contractor shall provide a draft schedule and meeting minutes to the Government for review within ten (10) days following the completion of the kick-off meeting. After Government review of the draft schedule, the Contractor shall incorporate feedback, revise as needed, and provide a final schedule to the Government within fifteen (15) days of the first on-site audit.

2.2 Attestations for EMMA QMS Documents

The Contractor personnel performing audits shall read and be familiar with the EMMA QAM and QARs. Copies of the QAM and the QAR documents listed in Table 1, as well as the EMMA QMS attestation form, will be provided electronically by the USGS within ten (10) days following the task order kick-off meeting. QARs will not exceed 20 pages (+/- 10 pages) in length. The Contractor shall complete the EMMA QMS attestation form for each Contractor personnel performing audits. Attestation forms must be submitted to the COR/TL at least ten (10) days prior to performing on-site laboratory audits.

Table 1. EMMA QMS Quality Assurance Requirements and Glossary.

QAR # Title

01 Quality Assurance Requirements and Standard Operating Procedures

02 Document Control

03 Method Requirements, Development, and Validation

04 Maintaining Laboratory Equipment

05 Sample Receipt, Login, and Storage

06 Traceability

07 Calibration

08 Quality Control and Quality Control Charts

09 Data Recording, Reduction, Review and Reporting

10 Chain of Custody

11 Corrective Action

12 Analytical Balances

13 Volumetric Dispensing Devices

14 Reporting Quality Issues

15 Demonstration of Capability

16 Audits and Reviews

18 Notification of Quality Issues After Data Release

19 Training

20 Work Hold Procedures for Data Quality Issues

21 Management

22 Scientific Notebooks and Logbooks

23 QMS Change Review Council

Glossary Glossary

2.3 Review of Technical and Support Standard Operating Procedures

The Government will provide an electronic copy of the Technical and Support Standard Operating Procedures

(SOPs) from the laboratories to be audited to the Contractor within ten (10) days following the task order kick-off meeting. A preliminary list of SOPs to be audited is provided in Table 2 (list may not be complete – titles may change and additional documents (+/- 10 documents) to those listed below may be added for review; documents also may be removed entirely from the list). The final list of SOPs will be provided electronically to the Contractor within ten (10) days following the task order kick-off meeting. SOPs will not exceed 20 pages (+/- 10 pages) in length. The Contractor will not be required, in reviewing Standard Operating Procedures, to review method references.

The Contractor shall review each SOP on the final list to ensure it provides enough detail so that a similarly trained analyst could perform the process and that it meets the requirements in the QAM and QARs. The Contractor shall provide comments for each SOP, as appropriate, in the electronic SOP document. The final comments for each SOP may be submitted as each is completed or in batches. All comments must be submitted to the USGS no later than the date required for the Final Audit Report (see Table 3).

Table 2. Standard Operating Procedures for Laboratories to be Audited

Science

Center Location Standard Operating Procedures

ASC Anchorage, AK MRP-AFLINC-SOP-01 Microthermometry of Fluid Inclusions Using A Gas-Flow Stage

ASC Anchorage, AK MRP-ASCGEOL-SOP-01 Alaska Science Center Geology Laboratory Grain Mount

Preparation

ASC Anchorage, AK MRP-ASCGEOL-SOP-02 Alaska Science Center Geology Laboratory Mini Lathe

ASC Anchorage, AK MRP-ASCGEOL-SOP-03 Alaska Science Center Geology Laboratory Polisher

ASC Anchorage, AK MRP-ASCRP-SOP-01 Mechanical Processing And Cleaning Of Geologic Samples

G3 Denver, CO MRP-ACHEM&SC-SOP-01 Sample Login and Storage

G3 Denver, CO MRP-ACHEM&SC-SOP-02 Sample Preparation

G3 Denver, CO MRP-ACHEM&SC-SOP-03 Crushing Preparation

G3 Denver, CO MRP-ACHEM&SC-SOP-04 Grinding Preparation

G3 Denver, CO MRP-ACHEM&SC-SOP-05 Sieving and Splitting

G3 Denver, CO MRP-ACHEM&SC-SOP-06 Drying and Ashing

G3 Denver, CO MRP-ATEAL-SOP-05 Perchloric Hood Cleaning

G3 Denver, CO MRP-BMRES-SOP-01 Powder X-ray diffraction of environmental materials

G3 Denver, CO MRP-BMRES-SOP-03 Loss on Ignition and Fusion

G3 Denver, CO MRP-BMRES-SOP-05 Anaerobic Chamber

G3 Denver, CO MRP-BMRES-SOP-07 Preparation of solid samples for XRD

G3 Denver, CO MRP-DML-SOP-02 Sample Preparation for Electron Microscopy- Uncontrolled version

G3 Denver, CO MRP-DML-SOP-03 Scanning Electron Microscopy and Microanalysis

G3 Denver, CO MRP-DML-SOP-04 Electron Probe Microanalysis

G3 Denver, CO MRP-G3APL-SOP-01 SPEX 8000 Mixer/Mill

G3 Denver, CO MRP-G3APL-SOP-02 Bico Disc Pulverizer

G3 Denver, CO MRP-G3APL-SOP-03 Frantz Magnetic Separator

G3 Denver, CO MRP-G3APL-SOP-04 Wilfley Table

G3 Denver, CO MRP-G3APL-SOP-05 Heavy Liquid Separation

G3 Denver, CO MRP-G3APL-SOP-06 Sieve Shaker

G3 Denver, CO MRP-G3APL-SOP-07 Rock Splitter

G3 Denver, CO MRP-G3APL-SOP-08 Diamond Pacific Rock Saw

G3 Denver, CO MRP-G3APL-SOP-09 Sturtevant Jaw Crusher

G3 Denver, CO MRP-G3APL-SOP-11 Lapidary Wheel

G3 Denver, CO MRP-G3APL-SOP-13 Heavy Mineral Separation Method for Geologic Materials

G3 Denver, CO MRP-G3DAGL-SOP-01 Argon Geochronology

G3 Denver, CO MRP-G3DIAL-SOP-03 Raw count analysis by LA-ICP-MS

G3 Denver, CO MRP-G3DIAL-SOP-04 Sample Preparation for Transmission Micro-FTIR Analysis

G3 Denver, CO MRP-G3DIAL-SOP-05 Transmission Micro-FTIR Analysis of H2O and CO2 in Igneous

Glass

G3 Denver, CO MRP-G3GCCR-SOP-01 Determination of d18O and d2H values of water by cavity ringdown spectroscopy

G3 Denver, CO MRP-G3ICL-SOP-01 Procedure for cleaning Teflon beakers and jars

G3 Denver, CO MRP-G3ICL-SOP-02 Titrating and adjusting the concentration of mineral acids

Center Location Standard Operating Procedures

G3 Denver, CO MRP-G3ICL-SOP-03 Making Sub-Boiling Double-Distilled Acids

G3 Denver, CO MRP-G3MRL-SOP-01 Powder X-ray Diffraction Using the Panalytical Diffractometer

G3 Denver, CO MRP-G3MRL-SOP-02 Particle Size Analysis by Laser Diffraction

G3 Denver, CO MRP-G3MRL-SOP-03 Particle Size Separation by Dry Sonic Sifting

G3 Denver, CO MRP-G3MRL-SOP-04 Clay Mineral Preparation for X-ray Diffraction Analysis

G3 Denver, CO MRP-G3MRL-SOP-05 Elemental Survey Data by Handheld X-ray Fluorescence Analyzer

G3 Denver, CO MRP-G3PL-SOP-01 U-TH-PB Geochronology by LA-SC-ICPMS

G3 Denver, CO MRP-G3PL-SOP-02 Operating the Selfrag Pulse Power Fragmentation Unit

G3 Denver, CO MRP-G3SC-SOP-01 Care of Platinum Ware

G3 Denver, CO MRP-G3SC-SOP-02 Potentiometric pH Determination Of Aqueous Samples And Extracts

G3 Denver, CO MRP-G3SC-SOP-04 Cleaning Labware for Multiple Methods of Elemental Analysis

G3 Denver, CO MRP-G3SEAL-SOP-01 Analysis of Anions in Water by Ion Chromatography

G3 Denver, CO MRP-G3SEAL-SOP-02 Alkalinity

G3 Denver, CO MRP-G3SEAL-SOP-04 Mercury in Aqueous and Geologic Materials by Continuous

FlowCold Vapor-Atomic Fluorescence Spectrometry

G3 Denver, CO MRP-G3SEAL-SOP-05 Arsenic and Selenium by Continuous Flow-Hydride

GenerationAtomic Fluorescence Spectrometry

G3 Denver, CO MRP-G3SEAL-SOP-06 Dissolved Organic Carbon

G3 Denver, CO MRP-GRMPL-SOP-01 Preparation Of Geologic Reference Materials For Whole Rock

Analysis

G3 Denver, CO MRP-GSIL-SOP-01 Determination of d13C, d15N, or d34S by isotope-ratio-monitoring mass spectrometry using an elemental analyzer

G3 Denver, CO MRP-GSIL-SOP-02 Precipitation of aqueous sulfate as BaSO4 for determination of d34S

G3 Denver, CO MRP-GSIL-SOP-03 Removal of carbonate carbon from sedimentary rocks prior to determination of d13C of total organic carbon

G3 Denver, CO MRP-GSIL-SOP-04 Removal of carbonate carbon from soils prior to determination of d13C of total organic carbon

G3 Denver, CO MRP-GSIL-SOP-05 Removal of lipids and oils from biological tissues prior to determination of d13C of total organic carbon

G3 Denver, CO MRP-GSIL-SOP-06 Extraction of oxygen from minerals and rocks using bromine pentafluoride

G3 Denver, CO MRP-GSIL-SOP-07 Isotopic analysis of CO2 gas or H2 gas by dual viscous inlet mass spectrometry

G3 Denver, CO MRP-HRES-SOP-01 Column Separation Of Lead And Strontium For Isotope Ratio

Measurements By MC-ICP-MS

G3 Denver, CO MRP-HRES-SOP-02 Strontium And Lead Isotope Ratio Measurements By MC-ICP

G3 Denver, CO MRP-HRES-SOP-04 Hg Concentrations by Direct Mercury Analysis

G3 Denver, CO MRP-SPECLAB-SOP-01 Conducting Spectral Measurements With The Nicolet

Spectrometer

Location Standard Operating Procedures

G3 Denver, CO MRP-SPECLAB-SOP-02 Conducting Spectral Measurements With The Asd

Spectrometers

G3 Denver, CO MRP-SPECLAB-SOP-08 Measuring Reference Standards For Evaluating Spectrometer

Wavelength Performance

G3 Denver, CO MRP-USGSNGL-SOP-03 Measurement Of Tritium In Water By The Helium In-Growth

Method

G3 Denver, CO MRP-USGSNGL-SOP-04 Extraction Of Dissolved Gas From Water Samples

G3 Denver, CO MRP-USGSNGL-SOP-05 Analysis Of Bulk Gas Compositions And Noble Gas Isotopes

From Gas Phase Samples

GEM Reston, VA MRP-CML-SOP-01 Critical Minerals Lab Sample Login–Logout Procedure

GEM Reston, VA MRP-CML-SOP-05 Dual Grinder/Polisher

GEM Reston, VA MRP-CML-SOP-07 Sieve Shaker

GEM Reston, VA MRP-CML-SOP-08 Critical Minerals Lab – General Housekeeping

GEM Reston, VA MRP-GEMSIL-SOP-01 Sample Login

GEM Reston, VA MRP-GEMSIL-SOP-05 Determination of the d2H and d18O in water

GEM Reston, VA MRP-GEL1-SOP-01 Sample Login and Storage

GEM Reston, VA MRP-GEL1-SOP-02 Synthetic precipitation leaching procedure (SPLP)

GEM Reston, VA MRP-GEL1-SOP-03 Cleaning labware for general use

GEM Reston, VA MRP-GEL1-SOP-04 Acid washing labware for trace element sampling and analysis

GEM Reston, VA MRP-GEL1-SOP-05 Filtration and preservation of liquid samples for chemical analysis

GEM Reston, VA MRP-GEL1-SOP-06 Humidity Cell Test Procedure

GEM Reston, VA MRP-GEL1-SOP-07 Protocols for drying, splitting, and sieving solid samples

GEM Reston, VA MRP-GEL1-SOP-08 Precipitation of Sulfate for Isotopic Analysis

GEM Reston, VA MRP-GEL1-SOP-09 Determining alkalinity by titration

GEM Reston, VA MRP-GEL1-SOP-10 Measurement Of Dissolved Oxygen Using Chemetrics Dissolved

Oxygen Kit

GEM Reston, VA MRP-GEL1-SOP-12 Measurement Of pH, Specific Conductivity, And ORP Using An

Orion Versa Star Benchtop Meter

GEM Reston, VA MRP-GEL2-SOP-04 A Support SOP for Cleaning Labware

GEM Reston, VA MRP-PREP-SOP-01 Mechanical Processing of Geologic Samples

GEM Reston, VA MRP-PREP-SOP-02 Heavy Liquid Mineral Separation

GEM Reston, VA MRP-PREP-SOP-03 Login-Logout

GEM Reston, VA MRP-RIL-SOP-01 TIMS/ICPMS Sample Receiving And Login Procedure

GEM Reston, VA MRP-RIL-SOP-02 TIMS/ICPMS Sample Storage And Handling

GEM Reston, VA MRP-RIL-SOP-03 TIMS/ICPMS Sample Archiving After Analysis

GEM Reston, VA MRP-RIL-SOP-04 Storage Of Standards And Chemical Solids

GEM Reston, VA MRP-RIL-SOP-06 Housekeeping And Maintenance Of Instrument And General

Chemistry Labs

GEM Reston, VA MRP-RIL-SOP-07 Housekeeping And Maintenance Of The Clean Lab (3B-440A-B-C)

GEM Reston, VA MRP-RIL-SOP-08 Acid Cleaning Of Labware

Location Standard Operating Procedures

GEM Reston, VA MRP-RIL-SOP-09 Acids And Water Distillation

GEM Reston, VA MRP-RIL-SOP-10 TIMS/ICPMS Weighing And Sample Dissolution

GEM Reston, VA MRP-RIL-SOP-11 TIMS Sr Chemistry Protocol MRP-RIL-SOP-12 TIMS Galena Pb

Chemistry

GEM Reston, VA MRP-RIL-SOP-13 TIMS PB CHEMISTRY PROTOCOL

GEM Reston, VA MRP-RIL-SOP-14 TIMS GALENA PB CHEMISTRY

GEM Reston, VA MRP-RIL-SOP-15 Thermal Ionization Mass Spectrometry

GEM Reston, VA MRP-RIL-SOP-16 Inductively Coupled Plasma Mass Spectrometry

GEM Reston, VA MRP-SPEC-SOP-01 Sample Login

GEM Reston, VA MRP-XRL-SOP-01 X-Ray Fluorescence Using The Reston Thermo Niton Xl3 Portable

Xrf Analyzer

GEM Reston, VA MRP-XRL-SOP-02 Operating the Jordan Valley EX-6600 Energy Dispersive X-Ray

Fluorescence Spectrometer

GEM Reston, VA MRP-XRL-SOP-03 Operation of the PANalytical X-Ray Difractometer I

GMEG Menlo Park, CA MRP-EGG-SOP-01 Sample Login Storage and Subsampling

GMEG Menlo Park, CA MRP-EGG-SOP-02 Coy Anaerobic Chamber

GMEG Menlo Park, CA MRP-EGG-SOP-03 Cleaning and Storage of Labware

GMEG Menlo Park, CA MRP-EGG-SOP-04 Eppendorf 5810R Centrifuge

GMEG Menlo Park, CA MRP-EGG-SOP-05 Measuring pH

GMEG Menlo Park, CA MRP-EGG-SOP-06 Measuring Conductivity

GMEG Menlo Park, CA MRP-EGG-SOP-09 Thermo Niton XL3t GOLDD+ Portable XRF Spectrometer

GMEG Menlo Park, CA MRP-EGG-SOP-11 Varian Cary 50 UV-VIS Spectrophotometer

GMEG Spokane, WA MRP-SPRX-SOP-01 Sample login and logout

GMEG Spokane, WA MRP-SPRX-SOP-02 Sieve cleaning

GMEG Spokane, WA MRP-SPRX-SOP-03 Wet sieving and hydroseparation for mineral separates

GMEG Spokane, WA MRP-SPRX-SOP-04 X-Ray Fluorescence Using The Delta Professional (DPO-2000)

Portable XRF Analyzer

GMEG Stanford, CA SHRIMP Laboratory User's Guide

2.4 On-Site Laboratory Audit Requirements

The Contractor shall conduct on-site audits for each MRP laboratory. The following section describes the on-site laboratory audit requirements for this task order.

2.4.1 Audit Kick-off Meetings

Contractor shall conduct an audit kick-off meeting at each location immediately prior to beginning the on-site audit.

Laboratory personnel, supervisors, managers in each Science Center, and representatives from the EMMA QMS team will be invited to attend each kick-off meeting. The kick-off meeting shall include introductions of Contractors and federal staff, and a Contractor presentation on the scope of the audit, the audit process, the specific schedule for the week, and any additional information to ensure the audits are safe, effective and efficient. The Contractor must provide to the USGS a copy of each presentation given at each location, as well as a list of attendees, within five (5) days of completing the presentation.

2.4.2 Daily Briefings

Each day during the on-site audit, prior to beginning the daily audit, the Contractor shall conduct a short meeting to provide both the Contractor personnel conducting the audits and USGS laboratory managers, scientists, laboratory personnel, and EMMA QMS staff an opportunity to interact and communicate. Contractor shall discuss the daily schedule, logistics (access, safety, and coordination protocols), and the previous day’s audit results. For example, it may be necessary to discuss and revise the planned activities for the day to minimize impact on lab productivity, provide updates on progress, discuss access issues, or other topics. At the end of each day, the Contractor shall conduct an exit meeting to present a summary of the day’s findings. The Contractor’s on-site lead and the USGS technical liaison may, if agreeable to both Contractor and USGS, cancel exit meetings. The Contractor must provide the meeting minutes and a list of attendees from both the beginning and end-of-day (if applicable) meetings to the

USGS within five (5) days of each on-site closing briefing.

2.4.3 Closing Briefings

Following completion of each on-site audit, the Contractor shall conduct a post-audit closing briefing that includes a presentation by the Contractor. The Contractor shall be prepared to present a review of observed laboratory activities and identify preliminary findings, observations and exceptional or notable practices. The meeting shall serve as a discussion forum to aid the laboratory scientists, laboratory personnel, and managers in understanding the findings, to prioritize the deficiencies, provide suggestions (if any) as to other ways of operating, reinforce best practices.

Finally, the presentation may discuss any issues that relate to data quality and reliability of collected data, ongoing laboratory analysis, and future laboratory activities. The attendees may include auditees, laboratory managers, scientists, the EMMA QMS Team, and EMMA Mission Area management to discuss the preliminary findings and conclusion of the audits at respective locations. The Contractor must provide to the USGS a copy of each presentation given at each location, as well as a list of attendees, within five (5) days of completing the presentation.

2.4.4 Performing a Comprehensive Quality Process Audit

The Contractor shall perform a comprehensive audit of each identified laboratory SOP to ensure adherence to the

EMMA QAM and QARs, including observing each process being performed (as time allows) to ensure the SOP is followed. In addition to observing the SOP process, the Contractor shall audit the following in each laboratory for traceability and transparency following samples from login to the laboratory, through laboratory processes, to final data packages, as well as for adherence to the EMMA QAM and QARs:

a. Sample receipt, tracking, storage

b. Sample preparation

c. Laboratory equipment calibration and documentation

d. Accuracy and precision of readings and measurements

e. Timing and utilization of QA/QC processes to identify issues and concerns

f. Use and evaluation of sample blind results

g. Laboratory and Scientific notebook utilization

h. Data handling and recording

i. Calculations and transformation of data

j. Entry of data into electronic systems as appropriate

k. Data packages and secondary data review

l. Response to previous internal audits

m. Evaluation of laboratory corrective actions

n. Training and demonstrations of capability

In the event of extraordinary circumstances, (for example serious illness, out of service equipment, lack of access to shared laboratory facilities, processes are unavailable or extending well beyond the audit period), the USGS

Technical Liaison may request that the Contractor omit certain activities from the audit scope. The contractor shall note in the final audit report any activities that were excluded in this way during the audit. All on-site audits shall be completed within 16 weeks of the initial on-site audit.

During on-site audit activities, in the event the Contractor identifies a condition which may pose an imminent safety hazard to workers or to the public, or a condition identified to exist which the Contractor judges to be a significant data quality issue, the Contractor shall immediately notify either the technical liaison, QMS manager, or QA specialist verbally. The Contractor shall then provide the USGS technical liaison a written document within 48 hours describing the condition.

2.4.5 External Audit Report

After the closing briefing at the last audit location, the Contractor shall prepare a comprehensive report documenting in detail all aspects of the MRP laboratory audits including:

a. Description of which SOPs were reviewed including titles and unique identifiers for each of the documents and the processes observed

b. List of evidence observed during the on-site audits including specific unique identifiers for evidence where available (i.e., logbook IDs, scientific notebook IDs, names or identifiers for data packets, Job

Numbers)

c. List of auditees, dates of audits, and names of the laboratories visited

d. The following categories and definitions will be used during the on-site audit and in the report to discuss the results of the evaluation of the collected audit evidence against the audit criteria (EMMA

QAM and QARs):

i. Finding:

Level 1: A statement of fact describing a deviation from the QAM, QARs or laboratory SOPs that will have a significant impact on the quality of the data produced.

Level 2: A description of a series of similar deviations from the QAM, QARs or laboratory SOPs that individually do not have a significant impact on the quality of the data but together represent a systemic issue that will have a significant impact on the quality of data produced.

ii. Observation: A statement of fact describing a deviation from the QAM, QARs, or laboratory

SOPs that will not have a significant impact on quality of data produced, which needs to be addressed and is potentially an opportunity for improvement rather than just for conformance to the audit criteria.

iii. Best Practice: A procedure, process, or activity that goes above and beyond the audit criteria.

e. For each Finding and Observation, the Contractor must provide a Citation (i.e., QAM and Section, or

QAR and Section) of the requirement that was not met in the QAM, QARs or SOP

f. When the audit report includes issues identified as findings or observations obtained from interviews with auditees, auditors must have documented corroborated of issues through additional interviews with multiple individuals to ensure the identified issues are pervasive rather than a single viewpoint.

The process for final delivery of the external audit report is provided in Table 3. Specifically, the Contractor shall provide an electronic (Microsoft Word) preliminary draft report to the USGS Technical Liaison for review and comment within twenty (20) business days of the completion of the post audit meeting for the last on-site audit.

A consolidated set of USGS comments will be provided back to the Contractor within five (5) business days of receipt of the preliminary draft report. The Contractor shall prepare and provide back to the USGS within five (5), a written response to address each of the review comments received from the USGS.

The Contractor shall provide the draft final report within five (5) business days after sending the written responses to the USGS comment to the USGS Technical Liaison for final concurrence prior to finalizing the report.

The final report shall be provided by the Contractor within five (5) business days of final concurrence on the draft final audit report.

Table 3. Process for final delivery of the external audit report.

Activity Timeline

Preliminary Draft Report

Within twenty (20) business days of the completion of the post audit exit briefing for the last on-site audit.

USGS Response to Preliminary Draft Report Within five (5) business days of receipt of the preliminary draft report.

Written response to USGS comments Within five (5) business days of receipt of the

USGS comments to the preliminary draft report

Draft Final Audit Report Within five (5) business days of receipt of the final concurrence prior to finalization of the report

Deliver Final Audit Report Within five (5) business days of final concurrence on draft final audit report

The Contractor shall provide the USGS Technical Liaison with an electronic version of the final report. The final report shall be Microsoft Word/Excel compatible.

Quality Assurance Surveillance Plan (QASP)

1. INTRODUCTION

This quality assurance surveillance plan (QASP) provides a systematic method for evaluating contractor performance for the stated contract.

Purpose

This QASP explains what will be monitored, how monitoring will take place, who will conduct the monitoring, and how the monitoring will be documented. The QASP provides a means for evaluating whether the contractor is meeting the performance standards/quality levels identified in the Performance Work Statement (PWS) and to ensure that the government pays only for the level of services received.

This QASP defines the roles and responsibilities of all members of the project team, identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor’s performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results.

Performance Management Approach

Performance management focuses on assessing whether outcomes are being achieved and to what extent. A performance-based approach enables the contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. By focusing on results, the contractor has the flexibility to perform the tasks as they see fit while continuously improving and innovating over the course of the contract as long as the expected results are achieved and/or the desired performance levels are met.

Performance Management Strategy

The contractor is responsible for the quality of all work performed. The contractor measures that quality through the contractor’s own quality control program (QCP). Quality is measured through work output, not workers, and therefore includes all work performed under this contract regardless of whether the work is performed by contractor employees or by subcontractors. The contractor’s QCP sets forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS. The contractor’s QCP sets forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS. This QASP enables the government to take advantage of the contractor’s QCP.

Government representative(s) monitor performance and review performance reports furnished by the contractor to determine how the contractor is performing against communicated performance objectives. The government makes a determination regarding incentives based on performance measurement metric data and notifies the contractor of those decisions. The contractor is responsible for making required changes in processes and practices to ensure performance is managed effectively.

2. ROLES AND RESPONSIBILITIES

The Contracting Officer

The contracting officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control, as well as for resolving any differences between the observations documented by the contracting officer’s representative (COR) or Technical Liaison (TL) and the contractor. The CO designates one full-time COR or TL as the government authority for performance management.

The Contracting Officer’s Representative (COR) or Technical Liaison (TL)

The COR or TL is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR or TL limitations are contained in the written appointment letter. The COR or TL is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The COR or TL is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action. The COR or TL has the responsibility for completing Quality Assurance (QA) documentation used for the inspection and evaluation of the contractor’s work performance (Table 5).

3. IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

The required performance standards and/or quality levels are included in Table 4, Performance Requirements

Summary Table. If the contractor meets the required service or performance level, it will be paid the monthly amount agreed on in the contract. If the contractor exceeds the service or performance level, it is eligible to receive an incentive or award fee as stated in the contract. Failure to meet the required service or performance level will result in a deduction from the monthly amount.

4. METHODOLOGIES TO MONITOR PERFORMANCE

Surveillance Techniques

To minimize the performance management burden, simplified surveillance methods are used by the government to evaluate contractor performance when appropriate. The primary methods of surveillance are:

➢ 100% Inspection – Each month, as described in the schedule, the COR shall review the generated documentation and document the results in an email to the contractor.

➢ Progress meetings – Each month (at the discretion of the COR) the COR participates in progress meetings with the contractor to discuss the status of milestones and deliverables on the schedule. The meetings ensure the contractor is on target to meet milestones and deliverables as described in the schedule.

➢ Email and phone communications – The COR has ad hoc communications with the contractor when issues arise in-between monthly progress meetings.

Acceptable Quality Levels

The acceptable quality levels (AQLs) included in Table 4, Performance Requirements Summary Table, for contractor performance are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities such as those involving draft and final external audit reports, the desired performance level is established at 100 percent.

5. QUALITY ASSURANCE DOCUMENTATION

The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the performance monitoring techniques shown in Table

4.

Monitoring Documentation

The government’s QA surveillance, accomplished by the COR or TL is reported via email, phone and WebEx communications. Meeting minutes are kept from monthly meetings to document the government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

The COR or TL retains a copy of all emails and meeting minutes as QA documentation.

6. ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

Determining Performance

Government shall use the monitoring methods cited to determine whether the performance standards/service levels/AQLs have been met. If the Contractor has not met the minimum requirements, the Contractor may be asked to develop a corrective action plan to show how, with an associated timeline and milestones, the Contractor intends to bring performance up to the required levels. If the Contractor has been asked to develop a corrective action plan, the Contractor must submit the plan within ten (10) days of being notified of the substandard performance.

Reviews and Resolution

The CO, COR or TL may require the contractor’s project manager, or a designated alternate, to meet with government personnel as deemed necessary to discuss performance evaluation (Table 5). The COR or TL defines the frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor.

However, if the need arises or due to substandard performance by the contractor, the Contractor meets with the COR or TL as often as required or per the Government’s request. The agenda of the reviews may include:

➢ Issues and concerns of both parties.

➢ Corrective action plan analysis for issues arising from performance monitoring processes, as appropriate.

➢ Recommendations for improved efficiency and/or effectiveness

The CO, COR or TL coordinates and communicates with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

The CO, COR or TL and contractor jointly formulate tactical and long-term courses of action, including decisions regarding changes to performance standards in Table 4. Changes are documented in emails or meeting minutes with the contractor. Changes to service levels, procedures, and metrics are incorporated as a contract modification. Only the CO, bilaterally with the contractor, can make changes to the work statement performance standards.

Table 4. Performance Requirements Summary Table

Required Services

(Tasks)

Performance

Standards

Acceptable Quality

Levels (AQLs)

1. Submit a work schedule for onsite audits at each audit location that includes: labs to be audited, auditee names, daily schedules, and auditor names.

Schedule is 100% accurate.

99%

2. Submit a report of scheduled, completed, and outstanding tasks monthly.

Report is 100% accurate in depicting the current status of tasks.

95%

3. Post-audit exit briefing (closing meeting)

PowerPoint presentation for each location that includes a review of observed laboratory activities and identifies preliminary findings, observations and exceptional or notable practices

Presentation is

100% free of typographical errors, and100% factually accurate.

Draft presentation is timely.

< 2 errors per slide

Within 3 business days of the scheduled deadline.

4. Written responses to the USGS comments on the preliminary draft report.

Draft report is

100% free of typographical timely.

< 3 errors per page

5. Draft Final Audit Report Draft report is typographical

6. Final Audit Report Draft report is

100% free of typographical

7. Contractor is responsive to Technical

Liasion’s emails and phone calls.

Respond within 2 business days of receiving email or phone call.

Respond within 4 business days of receiving email or phone call.

Table 5: QASP Performance Evaluation

Required Services

(Tasks)

Performance

Standards

Acceptable Quality

Levels

Methods of

Surveillance

PERFORMANCE

RESULTS

(POSITIVE OR

NEGATIVE)

1. Submit a work schedule for onsite audits at each audit location that includes: labs to be audited, auditee names, daily schedules, and auditor names.

Schedule is

100% accurate.

99%

Inspections, status meetings, and progress emails.

2. Submit a report of scheduled, completed, and outstanding tasks monthly.

accurate in depicting the current status of tasks.

95%

3. Post-audit exit briefing

(closing meeting)

PowerPoint presentation for each location that includes a review of observed laboratory activities and identifies preliminary findings, observations and exceptional or notable practices

Presentation is

100% free of typographical errors, and100% factually

Presentation is

< 2 errors per slide

Within 3 business days of the scheduled deadline.

4. Written responses to the

USGS comments on the preliminary draft report.

Responses are typographical errors, and100%

Responses are deadline.

Inspections, status meetings,

5. Draft Final Audit Report Draft Final

Audit Report is

100% free of typographical errors, and100%

Draft Final

Audit Report is deadline.

Inspections, status meetings,

6. Final Audit Report Final Audit free of typographical errors, and100%

Final Audit

Report is deadline.

Inspections, status meetings,

7. Contractor is responsive to Technical Liaison’s emails and phone calls.

Respond within

2 business days of receiving email or phone call.

Respond within 4 business days of receiving email or phone call.

Monitoring email and phone communications.

Technical

Liaison will report this to

Contracting

Officer.