Attachment 3 - Quality Assurance Surveillance Plan (QASP).pdf

DEPARTMENT OF THE TREASURY

BUREAU OF ENGRAVING AND PRINTING

FORT WORTH, TEXAS 76131

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

Police Training Services Firing Range

TABLE OF CONTENTS

1 INTRODUCTION

1.1 Purpose

1.2 Performance Management Approach

1.3 Performance Management Strategy

2 ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer

2.2 The Contracting Officer’s Technical Representative

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

4.2 Customer Feedback

4.3 Acceptable Quality Levels

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

5.2 Monitoring Forms

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.2 Reporting

6.3 Reviews and Resolution

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

ATTACHMENT 2: QUALITY ASSURANCE MONITORING FORM

ATTACHMENT 3: CUSTOMER COMPLAINT FORM…………………………………………………7

BEP_MSR_Rev#1_1_23_18

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

1 INTRODUCTION

This quality assurance surveillance plan (QASP) is pursuant to the requirements listed in the Statement of Work (SOW) entitled. This plan sets forth the procedures and guidelines the Police Services Branch, Bureau of Engraving and Printing will use in ensuring the required performance standards or services levels are achieved by the contractor.

1.1 Purpose

1.1.1 The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards/quality levels identified in the SOW, contractor’s quality control plan (QCP), and to ensure the government pays only for the level of services received.

1.1.2 This QASP defines the roles and responsibilities of all parties involved which identifies the following:

performance objectives, the methodologies used to monitor and evaluate the contractor’s performance, quality assurance requirements to be documented, and describes the analysis of quality assurance monitoring results.

1.2 Performance Management Approach

1.2.1 The SOW structures the acquisition around “what” service or quality level is required, as opposed to “how” the contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by the Bureau of Engraving and Printing to monitor and manage the contractor’s performance to ensure the expected outcomes or performance objectives communicated in the SOW are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether the contractor is expected to achieving outcomes for the project.

1.2.2 Performance management represents a significant shift from the more traditional quality assurance (QA) concepts in several ways. Performance management focuses on assessing whether outcomes are achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus provides the contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are achieved and/or the desired performance levels are met.

1.3 Performance Management Strategy

1.3.1 The contractor is responsible for the quality of all services provided. The contractor measures that quality through the contractor’s own quality control (QC) program. The contractor’s QCP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the SOW. The contractor will develop and implement a performance management system with processes to assess and report its performance to the designated government representative. The contractor’s QCP will set forth the staffing and procedures for examining the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the SOW. This QASP enables the government to take advantage of the contractor’s QC program.

1.3.2 The government representative(s) will monitor services provided and review customer feedback reports (as related to BEP personnel) furnished by the contractor to determine how the contractor is performing against communicated performance objectives. The contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

2 ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer

The contracting officer (CO) is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the Police Services Branch, Bureau of Engraving and Printing, COR and the contractor. The CO will designate one full-time COR as the government authority for performance management. The number of additional representatives serving as technical inspectors depends on the complexity of the services measured, as well as the contractor’s performance, and must be identified and designated by the CO.

2.2 The Contracting Officer’s Technical Representative

The contracting officer’s representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. COR, limitations are contained in the written appointment letter. The COR is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deem may affect contract price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for completing QA monitoring forms used to document the inspection and evaluation of the contractor’s work performance. Government surveillance may occur under the inspection of services clause for any service relating to the contract.

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

The required performance standards and/or quality levels are included in the SOW. If the contractor meets the required service or performance level, the monthly amount agreed on in the contract will be paid. Failure to meet the required service or performance level will result in a deduction from the monthly amount or cancellation of contract.

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

In an effort to minimize the performance management burden, the government to evaluate contractor performance when appropriate shall use simplified surveillance methods. The primary methods of surveillance are:

100% Inspection – When services are utilized, the COR, shall review the generated documentation (Level 1 – Course Evaluation Form) and enter summary results into the Surveillance Activity Checklist.

Periodic Inspection – Based on the annual Police Training schedule the COR and designated supervisory personnel will perform a periodic inspection when training is conducted at contractor facility.

4.2 Customer Feedback

The contractor is expected to establish and maintain professional communication between its customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided, measured through customer complaints.

Performance management drives the contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems, but the customer always has the option to communicate complaints to the COR, as opposed to the contractor.

Customer complaints, to be considered valid, must be set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept those customer complaints and investigate using the Quality Assurance Monitoring Form – Customer Complaint Form, identified in Attachment 3.

Customer feedback may also be obtained either from the results of formal customer satisfaction surveys (Level 1 – Course Evaluation Form) or from random customer complaints.

4.3 Acceptable Quality Levels

The acceptable quality levels (AQLs) included in Attachment 1, Performance Requirements Summary Table, for contractor performance is structured to allow the contractor to manage how the services are provided. For certain critical activities such as those involving accessibility to facilities, the desired performance level is established at 100 percent.

Other levels of performance are keyed to the relative importance of the task to the overall mission performance at the Bureau of Engraving and Printing, Fort Worth, Texas.

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards expressed in the SOW and is assessed using the performance monitoring techniques shown in Attachment 1.

5.2 Monitoring Forms

The government’s QA surveillance, accomplished by the COR, will be reported using the monitoring forms in Attachments 2 and 3 (when necessary). The forms, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required results – expected quality levels are being achieved.

5.2.1 The COR will retain a copy of all completed QA surveillance forms.

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.1.1 Government shall use the monitoring methods cited to determine whether the service levels/AQLs have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. Failure to meet AQL may result in cancellation of the contract services.

6.2 Reporting

6.2.1 At the end of each month (services/facilities are utilized), the COR will prepare a written report for the Bureau of Engraving and Printing summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes the contractor’s submitted monthly report and the completed quality assurance monitoring forms (Attachment 2), will become part of the QA documentation. It will enable the government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards. (*All documentation only required when services/facilities utilized or upon request of the BEP).

6.3 Reviews and Resolution

6.3.1 The Bureau of Engraving and Printing (COR) may require the contractor’s project manager, or a designated alternate, to meet with the COR and other government personnel as deemed necessary to discuss performance evaluation items. The COR will define a frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor; however, if the need arises, the contractor will meet with the COR as often as required or per the contractor’s request. The agenda of the reviews may include:

Monthly performance assessment data and trend analysis

Issues and concerns of both parties

Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

Recommendations for improved efficiency and/or effectiveness

6.3.2 The COR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

6.3.3 The COR and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to service levels should be clearly documented. Changes to service levels, and procedures will be incorporated as a contract modification at the convenience of the Bureau of Engraving and Printing and Contractor.

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

To ensure the Government has an effective and systematic method of surveillance for the services in the Performance Work Statement. The Quality Assurance Surveillance Plan (QASP) will be used primarily as a tool to verify that the contractor is performing all services as required by the Performance Work Statement in a timely, accurate and complete fashion. Performance will be measured in accordance with the Government QASP table below.

Primary Performance Objective

(The Service required—usually a shall statement) Standard

Performance Threshold (This is the maximum error rate. It could possibly be “Zero deviation from standard”)

Method of Surveillance

The contractor shall provide full access to the facility.

This standard is in accordance with all applicable guidelines identified within this agreement.

Zero Tolerance

Allowed.

100% inspection made by COR or other designated

BEP personnel.

Immediately notify the Government of any contractual deficiencies identified

Report to Government Personnel 99% of the time as required Zero Tolerance

Allowed COR will monitor for compliance The Contractor met the performance requirements of the Performance Work Statement

99% of the services performed per requirements of the

SOW

Zero Tolerance Allowed

COR will assess compliance through review of SOW

The Contractor provided required services in a timely manner

99% of the services performed in a timely manner Zero Tolerance

Allowed

COR will review various Range Usage Data/Reports

(Level 1 Evaluations) The Contractor maintained close liaison and good communications with the Government

Any problems or questions encountered by either party must be addressed immediately

Zero Tolerance Allowed

Gov’t will assess compliance through coordination between COR & Contractor

Quality of Services

Quality of services meet or exceed task

99% of the time Zero Tolerance

Allowed Regular contact between

COR and Contractor

ATTACHMENT 2:

COR MONTHLY STATUS REPORT

REPORT FOR MONTH ENDING

COR Name:

Contract Number:

Delivery/Task Order:

Contract Period of Performance From To

CONTRACTORS QUALITY OF WORK

Exceptional

Very Good

Satisfactory

Marginal

Unsatisfactory

1. Is the Contractor up –to-date in submitting or performing the following:

a. Monthly Invoices or Cost Vouchers Yes Comments

No

N/A

b. Sub-Contractor Information

Yes Comments No N/A

c. Required Deliverables Yes Comments No N/A

d. Status Reports Yes Comments No N/A

2. Has the Contractor conducted duties in a timely manner?

Yes Comments No

N/A

3. Were all the items in the QASP inspected?

No N/A

4. Is the contract progressing on schedule?

Yes Comments No N/A

5. Are there any significant issues to report?

Yes Comments No

N/A

6. Are there any discrepancies to report

Yes Comments No N/A

7. Are there any customer complaints this month and if so how many?

Yes Comments No N/A

8. Are there any issues for the Contracting Officer to be aware of?

Yes Comments

No N/A

9. Date the final goods or services were accepted? (MM/DD/YY)

N/A Comments

10. Monthly COR Certification regarding personal services. If “YES”, describe what occurred, the date of occurrence, witnesses and other pertinent facts in the corresponding comment field below.

a. A government employee asking the contractor to hire or remove a particular employee:

Yes Comments No N/A

b. A government employee asking the contractor to reassign or discipline an employee:

Yes Comments No N/A

c. A government employee asking the contractor to grant or deny leave to a contractor employee:

Yes Comments No N/A

d. A government employee changing a contractor’s personnel duty hours:

e. A government employee asking the contractor to perform work outside the scope of the current contract without an official modification or direction by the Contracting Officer:

Yes Comments No

N/A

f. Contractor personnel appear to be under relatively continuous supervision and control by a Government employee:

Yes Comments

No N/A

g. Contractor personnel performing inherently Governmental work functions (i.e. serving in a decision-making capacity, making budget decisions, supervising, or rating Government employees, etc.) FAR 7.503:

Yes Comments

COR Signature:

I certify that this report is in accordance with the terms and conditions of this contract in conjunction with the QASP and monthly surveillance schedule.

MONTHLY STATUS REPORT EVALUATION RATING CRITERIA

RATING DEFINITION NOTE

Exceptional Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should have been NO significant weaknesses identified.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should have been no significant weaknesses identified.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the contract/order.

There should have been NO significant weaknesses identified.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsat Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

ATTACHMENT 3:

CUSTOMER COMPLAINT FORM

CUSTOMER COMPLAINT RECORD

DATE OF COMPLAINT

TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

INDIVIDUAL

NATURE OF COMPLAINT___________________________________________________________

CONTRACT REFERENCE

COMPLAINT VALIDATED BY

DATE CONTRACTOR INFORMED

TIME CONTRACTOR INFORMED

ACTION TAKEN BY CONTRACTOR_____________________________________________________

RECEIVED AND VALIDATED FINAL RESULTS

NOTES