14.1_Tech_Exhibit_1_-_QASP_(1).docx

TECHNICAL EXHIBIT 1 - QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

MIDDLE RIO GRANDE ENDANGERED SPECIES COLLABORATIVE PROGRAM (PROGRAM)

Program and Science Support Services i

TABLE OF CONTENTS

1.1

NTRODUCTION

Purpose

1.3 R Performance Management Strategy

OLES AND RESPONSIBILITIES

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

ATTACHMENT 2: PERFORMANCE AREAS AND EVALUATION CRITERIA

ATTACHMENT 3: QUALITY ASSURANCE SURVEILLANCE PLAN MATRIX

ATTACHMENT 4: QUALITY ASSURANCE MONITORING FORM

ATTACHMENT 5: CUSTOMER/STAKEHOLDER COMPLAINT FORM

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

1 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirements listed in the Performance Work Statement (PWS) entitled Program and Science Support Services. This plan sets forth the procedures and guidelines the Bureau of Reclamation (Reclamation) will use in ensuring the required performance standards or services levels are achieved by the contractor.

1.1 Purpose

1.1.1 The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards/quality levels identified in the PWS and the contractor’s quality control plan (QCP).

1.1.2 This QASP defines the roles and responsibilities of all members of the integrated project team (IPT), identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor’s performance, describes quality assurance documentation requirements, and describes the analysis of quality assurance monitoring results.

1.2 Performance Management Approach

1.2.1 The PWS structures the acquisition around “what” services are required, as opposed to “how” the contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by Reclamation to monitor and manage the contractor’s performance to ensure the expected outcomes or performance objectives communicated in the PWS are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the contractor.

1.2.2 Performance management represents a significant shift from the more traditional quality assurance (QA) concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus provides the contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met.

1.3 Performance Management Strategy

1.3.1 The contractor is responsible for the quality of all work performed. The contractor measures that quality through the contractor’s own quality control program (QCP). QC is work output, not workers, and therefore includes all work performed under this contract regardless of whether the work is performed by contractor employees or by subcontractors. The contractor’s QCP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS. The contractor will develop and implement a performance management system with processes to assess and report its performance to the designated government representative. The contractor’s QCP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS. This QASP enables the government to take advantage of the contractor’s QCP.

1.3.2 The government representative(s) will monitor performance and review performance reports furnished by the contractor to determine how the contractor is performing against communicated performance objectives. The government will make a determination regarding proposed and final performance ratings based on performance measurement metric data and will notify the contractor of those decisions. The contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

2 ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer

The contracting officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the contracting officer's representative (COR) for performance management and the contractor.

The CO will designate one full-time COR as the government authority for performance management. The number of additional representatives serving as performance monitors depends on the complexity of the services measured, as well as the contractor’s performance, and must be identified and designated by the CO.

The responsibilities of the CO include:

1. Establish the Performance Evaluation Board (PEB).

2. Consider the PEB findings for each evaluation period and discuss it with the PEB chair and, if appropriate, with others such as the contractor.

3. Any variance between the PEB performance rating recommendation and CO performance rating determination must be justified and documented in the official contract file.

4. Ensure that performance evaluations in the Contractor Performance Assessment Rating System are completed on an annual basis based on the biannual summary performance ratings.

5. Approves changes proposed to the Quality Assurance Surveillance Plan (QASP), as appropriate.

6. Appoint the voting members of the PEB by memorandum.

7. The CO is the approval authority for any substantive changes to this plan.

8. Receive and analyze the Performance Monitor Reports submitted by the Performance Monitors via the assigned COR.

9. Advise the PEB on performance rating standards, policies, and procedures and ensure the consistent application of Agency policy in these matters.

2.2 Performance Evaluation Board (PEB)

The primary responsibilities of the PEB are to:

1. Conduct ongoing evaluations of contractor performance based upon Performance.

2. Monitor Reports and such additional performance information as may be obtained from the contractor and other sources. The PEB will evaluate the contractor's performance according to the standards and criteria stated in this Quality Assurance Surveillance Plan.

3. Submit a performance rating letter to the CO covering the PEB's findings and recommendations for each quarterly evaluation period. If the Contractor’s performance ratings for the first year of the Contract is rated at least “Very Good,” the PEB may elect to perform evaluations less frequently, i.e., annual or less.

4. Recommend, for approval by the CO, proposed changes in the Quality Assurance Surveillance Plan.

2.3 PEB Chairperson

The PEB Chairperson is the assigned Contracting Officer’s Representative. The primary responsibilities of the PEB Chairperson are to:

1. Appoint non-voting members, if appropriate, to assist the PEB in performing its functions, e.g., a recording secretary.

2. Appoint performance monitors for the contract effort and assure that they are provided appropriate instructions and guidance to carry out their assigned duties.

3. Request and obtain performance information from other units or personnel involved in observing contractor performance, as appropriate.

4. Call on personnel from various organizational units to consult, as needed, with the

PEB.

5. Assume responsibility for the actual preparation and approval of the performance rating recommendation letter and other documentation such as PEB minutes.

6. Ensure the timeliness of performance evaluations.

7. Recommend appropriate changes in this plan for consideration.

8. Meet with the contractor during the evaluation period to provide preliminary performance feedback.

2.4 The Contracting Officer’s Representative

The contracting officer’s representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR limitations are contained in the written appointment letter. The COR is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for completing performance monitoring forms used to document the inspection and evaluation of the contractor’s work performance. Government surveillance may occur under the inspection of services clause for any service relating to the contract.

Contracting Officer’s Representative (COR)

The COR will be located at Reclamation’s Area Office in Albuquerque, New Mexico.

The primary responsibilities of the COR are to:

1. Monitor, evaluate, and assess Contractor performance.

2. Prepare the Contract Performance Summary Report for the Contracting Officer (CO).

3. As a member of the PEB, attend all PEB meetings, record the findings of the PEB, and prepare the performance rating recommendation letter for the CO’s review and signature in coordination with the CO.

4. Recommend appropriate changes in this plan for consideration.

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

The required performance standards and quality levels are included in Attachment 1, “Performance Requirements Summary”; Attachment 2, “Performance Areas and Evaluation Criteria”; and Attachment 3, “Quality Assurance Surveillance Plan Matrix.” Failure to perform the required services at a minimum level of “Satisfactory” will likely result in adverse past performance reporting by Reclamation to current Government-Wide past performance reporting systems such as CPARS and PPIRS.

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the government to evaluate contractor performance when appropriate. This is summarized in Attachment 3, “Quality Assurance Surveillance Plan Matrix.” The primary methods of surveillance are:

a) Random monitoring, which shall be performed by the COR and designated performance monitors.

b) 100% Inspection – Each month, the COR shall review the monthly status report submitted by the Contractor, other submitted deliverables, feedback submitted by stakeholders on draft deliverables, feedback on the effectiveness of public involvement activities, and enter summary results into the Quality Assurance Monitoring Form, Attachment 4.

c) Periodic Inspection – The COR typically performs periodic inspections on a monthly basis.

4.2 Stakeholder Feedback

The contractor is expected to establish and maintain professional communication between its employees and the Federal and non-Federal members of the Program’s Executive Committee (EC), Adaptive Management Committee, and other implementation teams. The EC is comprised of a variety of stakeholders with disparate interests in the resource. The Executive Committee strives for balance and consensus on how to improve the status of protected species while protecting existing and future water uses. The Adaptive Management Committee and implementation teams are comprised of EC entity staff or contractors hired by EC entities and will perform or contract out scientific work for the Program.

These EC members and staff are the primary stakeholders for this project. The primary objective of this feedback is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through stakeholder complaints.

Performance management drives the contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COR as opposed to the contractor.

Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept those customer complaints and investigate using the Stakeholder Complaint Investigation, included as Attachment 5.

Stakeholder feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

4.3 Desired and Minimum Quality Levels

The quality levels are defined in Attachment 1 Performance Requirement Summary Quality Levels and further described in Attachment 2 Performance Areas and Evaluation Criteria. With the exception of non-critical activities such as monthly status reports and meeting minutes, the desired quality level for all tasks is the “Very Good” level. The minimum quality level for any tasks is “Satisfactory.”

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and assessed using the performance monitoring techniques described in this plan.

5.2 Monitoring Forms

The government’s QA surveillance, accomplished by the COR and other performance monitors, will be reported using the monitoring forms in Attachments 4, Quality Assurance Monitoring Form and Attachment 5, Stakeholder Complaint Form. The forms, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

5.2.1 The COR will retain a copy of all completed QA surveillance forms.

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.1.1 Government shall use the monitoring methods cited to determine whether the performance standards and desired and/or minimum quality levels have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. The government will evaluate the results which then may be translated into an adjustment to the most recent performance evaluation ratings, should performance not meet, or conversely exceed the minimum quality level.

6.2 Reporting

6.2.1 The performance evaluation periods will cover one year and will begin on the date of contract award. At the end of each performance evaluation period, the COR will prepare a written report for the PM summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes the contractor’s submitted monthly reports and the completed quality assurance monitoring forms (Attachments 4 and 5), will become part of the QA documentation. It will enable the government to demonstrate whether the contractor is meeting the stated objectives and performance standards, including cost, technical, and scheduling objectives.

6.3 Reviews and Resolution

6.3.1 The COR may require the contractor’s project manager, or a designated alternate, to meet with the CO, COR, or PM and other government IPT personnel as deemed necessary to discuss performance evaluation. The COR and PM will define a frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor; however, if the need arises, the contractor will meet with the COR, CO, or PM as often as required or per the contractor’s request. The agenda of the reviews may include:

a) Performance assessment data and trend analysis

b) Issues and concerns of both parties

c) Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

d) Recommendations for improved efficiency and/or effectiveness e) Issues arising from the performance monitoring processes

6.3.2 The COR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

The COR and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification as determined by the CO.

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

Adjectival

performance. All deliverables are submitted on time and in accordance with the approved project schedule, or early, in spite of schedule delays not caused by the Contractor.

Very Good Very effective performance, fully responsive to contract requirements; contract requirements accomplished in a timely, efficient, and economical manner for the most part; only minor issues. All deliverables are submitted on time and in accordance with the approved project schedule, in spite of schedule delays not caused by the Contractor. Some deliverables are submitted ahead of schedule when there were no schedule delays caused by the Government.

Good Effective performance; fully responsive to contract requirements; reportable problems or issues, but with little identifiable effect on overall performance. Most deliverables are submitted on time and in accordance with the approved project schedule in spite of schedule delays not caused by the Contractor. All deliverables are submitted on time when there were no schedule delays caused by the Government.

Satisfactory Meets minimum acceptable standards; adequate results; reportable problems with identifiable, but not substantial, effects on overall performance. All deliverables are submitted on time, with additional time allowed to make up for schedule delays caused by the Government.

Poor/Unsatisfactory Does not meet minimum acceptable standards in one or more areas; remedial action required in one or more areas; deficiencies in one or more areas which adversely affect overall performance. Some deliverables are not submitted in accordance with the approved project schedule.

1 In order to earn a total overall rating of "Excellent," the contractor must be at or under cost (i.e., not requiring modifications to increase the contract price), on or ahead of schedule, and be rated "Excellent" for Technical Performance.

ATTACHMENT 2: PERFORMANCE AREAS AND EVALUATION CRITERIA

The performance factors to be evaluated are identified below. The evaluation criteria for each factor are specified in the indicated section of this attachment.

Factor A. Timeliness B. Technical Performance C. Cost Control Performance D. Program and Business Management Performance

Basis for Measuring Performance: Using the performance standards associated with each task, and the subfactors described below and a standard of reasonable performance for them, the COR and the Performance Monitors will prepare a Quality Assurance Monitoring Report (Attachment 4) that addresses each task worked on during the performance evaluation period. On the basis of those evaluations, each Quality Assurance Monitoring Report will be assigned a rating of "Excellent," "Very Good," "Good," "Satisfactory" and/or "Poor/Unsatisfactory" for each evaluation factor measured. For each task, the Quality Assurance Monitoring Report will characterize it as having major or minor impacts on overall schedule and performance in related areas and will describe any extraordinary circumstances relating to the task performance. These impact statements will be used by the PEB to weigh the inclusion of specific issues in the performance rating recommendation letter.

A. Timeliness

Description of Factor: Submittal of deliverables by the original approved baseline project schedule

Subfactors Considered for Evaluation:

Timeliness Requirements – The Contractor shall be evaluated on effective and efficient performance based services to meet the timeliness requirements and schedules as described in the Statement of Work. This includes adherence to the approved project schedule; the timely completion of deliverables and tasks; anticipating and resolving problems; recovery from delays, whether caused by the Contractor or the Government; reaction time and appropriateness of response to change requests. The Government will make evaluations only on services that have been requested/ordered.

B. Technical Performance

Description of Factor: For each-monthly evaluation period, technical performance is broadly assessed as the work performed in meeting the technical requirements of assigned tasks, including a variety of subfactors related to how the work was accomplished, as indicated below:

Subfactors Considered for Evaluation:

1. Technical Requirements – The Contractor shall be evaluated on complete, accurate, effective and efficient performance based services to meet the technical requirements described in the Statement of Work and this plan. This includes a subjective assessment of the quality of performance based services provided, i.e., thoroughness, accuracy of contractor performance in providing services, anticipating and resolving problems; recovery from delays; reaction time and appropriateness of response to change requests. Also to be considered is the quality of monthly progress reports and other required deliverables as outlined in the PWS. The Government will make evaluations only on services that have been requested/ordered.

Evaluation of technical requirements shall also consider the following:

a. Assessment of ability to meet the functional and performance requirements of each task;

b. Assessment of the Contractor’s ability to identify risks; analyze their impact and prioritize them; develop and carry out plans for risk mitigation, acceptance, or other action; track risks and the implementation of mitigation plans; support informed, timely, and effective decisions to control risks and mitigation plans; and assure that risk information is communicated among all levels of the program.

c. Compliance with the technical criteria and guidance contained in the Contract, particularly the Statement of Work and Attachments in Section J.

3.

Thoroughness and accuracy- The Contractor shall be evaluated on its appropriate analysis and evaluation of alternative methods, processes, or procedures to accomplish overall requirements within schedule and budget.

4.

Liaison effectiveness

5.

Independence and initiative

C.

Cost Control Performance

Description of Factor: Cost Control includes the measure of the contractor's success in completing each task within the original quantities negotiated for all active Contract Line Item Numbers (CLINs). The Cost Control Performance Rating, termed Cost, for each period shall be evaluated and determined as described below.

Subfactors Considered for Evaluation:

Cost – Accuracy and ability to maintain cost performance for each individual task shall be evaluated. The cost performance shall be evaluated based on completing the task without requesting contract modifications that increase the cost of performing the task. The analysis of cost control performance will give consideration to changed support requirements, changed statutory requirements, and/or changes beyond the Contractor’s control which impact contract costs.

D. Program and Business Management Performance

Description of Factor: Program and Business Management is the manner in which the Contractor plans, organizes, and controls work under the contract and implements contract provisions. It includes personnel, inter-organizational interfaces, team communication, work flow, and financial management.

Subfactors Considered for Evaluation:

1. Compliance with contract provisions - The Contractor shall be evaluated on the overall administration of the contract. This shall include accuracy and timeliness of all reporting requirements (e.g., change order proposal submissions, subcontract consent documentation, financial management reporting requirements, etc.) and overall compliance of all terms and conditions of the contract. The Contractor's responsiveness to requests for change proposals/task plans will be evaluated. This evaluation will include the submission of timely, complete change proposals and cooperation in negotiating the change.

2. Subcontracting - The Contractor will be evaluated on their overall effectiveness of managing subcontracts. This will include the business relationship between the prime and subcontractor, the level of cooperation between the two parties and the Contractor's ability to ensure quality performance based services from subcontractors.

3. Program planning and organizational management - The Contractor shall be evaluated on the use of personnel; recognition and correction of critical problem areas; cooperation and working relationships with other organizations' personnel; employee relations; management actions related to performance and productivity; and response to unexpected situations.

4. Business Management - The Contractor shall be evaluated on their local and corporate business management. This area shall include an evaluation of the overall ability and effectiveness to respond to operational and management problems of the Contractor. The Contractor shall be evaluated on the overall effectiveness of their project and task management.

ATTACHMENT 3: QUALITY ASSURANCE SURVEILLANCE PLAN MATRIX

QUALITY ASSURANCE SURVEILLANCE PLAN MATRIX

PROGRAM AND SCIENCE SUPPORT SERVICES

CONTRACT NUMBER: TBD

CONTRACTOR: TBD

CONTRACTING OFFICER: TBD

CONTRACT SPECIALIST: TBD

INSPECTOR(S): Members of the Performance Evaluation Board as Identified in the QASP

TASK

DESCRIPTION

WHAT

WHO

WHEN

WHERE

METHOD

Evaluation Board (PEB)

to determine.

All required deliverables will be visually reviewed by the PEB. Timeliness, Technical Performance, Cost Control Performance, and Program and Business Mgmt Performance in performing this task will be evaluated in accordance with the QASP.

Evaluation Board (PEB)

to determine.

All required deliverables will be visually reviewed by the PEB. Timeliness, Technical Performance, Cost Control Performance, and Program and Business Mgmt Performance in performing this task will be evaluated in accordance with the QASP.

Evaluation Board (PEB)

to determine.

All required deliverables will be visually reviewed by the PEB. Timeliness, Technical Performance, Cost Control Performance, and Program and Business Mgmt Performance in performing this task will be evaluated in accordance with the QASP.

ATTACHMENT 4: QUALITY ASSURANCE MONITORING FORM

TASK and PERFORMANCE STANDARD(S):

PERFORMANCE EVALUATION PERIOD:

SURVEILLANCE METHOD (Check All That Apply):

Random Sampling 100% Inspection Periodic Inspection Customer Complaint

ANALYSIS OF RESULTS:

Service Provider’s Performance (Check): Meets Standards

Does Not Meet Standards

Narrative of Performance During Evaluation Period:

EVALUATION FACTORS (Check only those that were rated on this form)

A. Timeliness:

Excellent Very Good Good Satisfactory Poor/Unsatisfactory

B. Technical Performance:

Excellent Very Good Good Satisfactory Poor/Unsatisfactory

C. Cost Control Performance:

Excellent Very Good Good Satisfactory Poor/Unsatisfactory

D. Program and Business Management Performance:

Excellent Very Good Good Satisfactory Poor/Unsatisfactory

PREPARED BY:

DATE:

ATTACHMENT 5: STAKEHOLDER COMPLAINT FORM

STAKEHOLDER COMPLAINT INVESTIGATION

TASK AND PERFORMANCE STANDARD:

EVALUATION PERIOD:

DATE/TIME COMPLAINT RECEIVED: SOURCE OF COMPLAINT:

NATURE OF COMPLAINT:

AM / PM (NAME) (ORGANIZATION) (PHONE NUMBER) (EMAIL ADDRESS)

RESULTS OF COMPLAINT INVESTIGATION:

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT:

AM / PM

CORRECTIVE ACTION TAKEN BY CONTRACTOR:

RECEIVED AND VALIDATED BY:

PREPARED BY:

DATE: