Attachment 5 - Quality Assurance Surveillance Plan.docx

QUALITY ASSURANCE SURVEILLANCE PLAN

RFP 36C79123R0001

Department of Veteran Affair Quality Assurance Surveillance Plan For CRT Repair Services Contract

Contract Number: ___________________________

Contractor: ________________________________

1.0

Initial Version

TABLE OF CONTENTS

1. INTRODUCTION

This IDIQ level quality assurance surveillance plan (QASP) sets forth procedures and guidelines the Department of Veteran Affairs, Commodities Services and Acquisition Services (CSAS) and Prosthetics and Sensory Aids Service (PSAS) will use in ensuring the required performance standards and supplies are achieved by the contractor.

Authority to issue this QASP is provided under Federal Acquisition Regulation (FAR) Clause 52.212-4(a) Alt I Inspection/Acceptance, and FAR 52.246-4 Inspection of Services which provided for inspections and acceptance of the articles, services, and documentation called for in the contract to be accomplished by the contracting officer or a duly appointed representative.

1.1 PURPOSE

The purpose of the QASP is to describe the systematic methods used to monitor the IDIQ performance and the identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards identified in the Performance Work Statement.

The QASP defines the roles and responsibilities of all the members, identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor’s performance, describes the quality assurance documentation requirements and describes the analysis of quality assurance monitoring.

This QASP is based on the premise the Government desires to maintain a quality standard in customer service operations and repairing defective devices and that a service contract to provide the service is the best means of achieving that objective.

1.2 PERFORMANCE MANAGEMENT APPROACH

The QASP will define the performance management approach taken by CSAS to monitor contractor’s performance to ensure performance standards identified in Section D (Attachment) are achieved. Performance Management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether acceptable performance under the IDIQ is being achieved by the contractor.

1.3 PERFORMANCE MANAGEMENT STRATEGIES

The government representative(s) will monitor performance and review performance reports furnished by the contractor to determine how the contractor is performing against the performance standards shown in the performance work statement.

The contractor is responsible for the quality of all work performed. The contractor measures that quality through the contractor’s own Quality Control (QC) program which can be commercial standard practices. QC is work output, not workers, and therefore includes all work performed under this IDIQ regardless of whether the work is performed by contractor employees, teaming partners, or subcontractors.

The first major step to ensuring a “self-correcting” contract is to ensure that the quality control program approved at the beginning of the contract provides the measures needed to lead the contractor to success.

The Contractor shall provide a general Quality Control Plan (QCP) with the technical proposal. The contractor is responsible for ensuring services are performed in accordance with PWS requirements. The contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The QCP shall outline how performance issues will be remedied or addressed (i.e. performance improvement plans, suspensions, reprimands and/or terminations).

Once the QCP is approved, careful application of the process and standards presented in the remainder of this document will ensure a robust quality assurance program. This QASP is a “living document” and the Government may review and revise it on a regular basis. The Government will coordinate changes with the contractor through contract modification. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing the surveillance activities. The contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

2. ROLES AND RESPONSIBILITIES

2.1 Contracting Officer

The contracting officer (CO) is responsible for monitoring IDIQ compliance and administration and for resolving any differences between observations documented by the program office contracting officer’s representative (COR) and the contractor. The CO will designate a COR as the government authority for performance management.

Assigned CO: Susan Dela Cruz.

2.2 Contracting Officer Representative

The contracting officer’s representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering the IDIQ. COR limitations are contained in the written appointment letter. The COR is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deems may affect price, terms, or conditions shall be referred to the CO for action. The COR will have the responsibility for completing QA monitoring forms used to document the inspection and evaluation of the contractor’s work performance.

Assigned COR: TBD

2.3 Other Key Government Personnel

Contract Specialist(s), Contracting Officer Representative (COR), and VISN Level COR or VISN Level Program Manager (PM) will provide contractual guidance and requirements when needed or appropriate. Any unforeseen or potential conflicts or problems can be addressed with the individuals listed but for any contractual changes, the Contracting Officer is the ONLY authority to make any change to the contract. If applicable, this QASP will be revised to include relevant POC information Assigned PM: TBD

2.4 Contractor Representative(s)

The Contractor has the responsibility for complying fully with the terms and conditions of the contract, participating as a member of the multi-functional team in the post-award management phase, maintaining and implementing a Quality Control Plan that complements the Performance Plan insuring that non-conforming contract services are identified and corrected. The Contractor will perform for the Government services that conform to the contract requirements and recommend changes to the contract that will provide more effective operations or eliminate unnecessary costs. The Contractor will be responsible for the execution of both the contractual and technical aspects of the contract performance.

The following employee(s) of the contractor shall serve as the contractor’s program manager(s) for this contract.

Primary: _________________________________ Alternate: ________________________________

3. IDENTIFICATION OF REQUIRED PEFORMANCE STANDARDS/QUALITY LEVELS

The required IDIQ performance standards and/or quality levels are included in Appendix 1- Performance Requirements Summary.

3.1 General Areas to be Monitored during Contract Performance The following Performance Based Service Contract (PBSC) items are identified within the PWS to be applicable on a wide basis and are to be monitored under this QASP.

See the Performance Requirements Summary Attachment and Section (Appendix 1)

Quality of Services Provided

· Compliance with Contract Requirements

· Technical Excellence Timeliness of Performance / Adherence to Schedule

· Responsiveness to Customer requests

· Completion of Repair Services

· Reliability Cost Control

· Demonstration of Daily Operational Efficiencies

· Incorporation of Innovative Practices Resulting in Cost Savings

· Effectiveness in Forecasting, Managing & Controlling Contract Cost/Price

· Ability to Keep Within the Total Estimated Cost or Fixed Prices

· Accuracy, Completeness & Timeliness of Invoicing Business Relations

· Customer-focus

· Management Effectiveness

· Flexibility

· Timeliness of Notification and Mitigation of Problems

· Effectiveness of Contractor’s Recommended Contractor solutions

· Effectiveness of Execution of Problem Resolution Contractor solution Regulatory Compliance

· Contract security

· Contract clauses

· Contract reporting requirements

· Quality assurance surveillance plan

· Industry-related requirements, safety/environmental requirements, and standard and unique contract-specific requirements

· FDA

· Contractor Technician Certification

Utilization of Small Business

· Maximum practicable opportunity

· Compliance with terms and conditions

3.2 Performance Standards

Performance Standards outlined in this QASP shall be used to determine the level of contractor performance in the elements defined. The Performance Work Statement identifies Government objectives and performance standards for each task contracted. The Government will use these standards to assess Contractor performance against these objectives to ensure performance is, at a minimum, satisfactory. The VA surveillance will measure contractor performance in meeting the standards using the QASP ratings.

The COR will use the performance standards to determine contractor performance rating and will compare contractor performance to the performance threshold of Table A: Performance Standard Measures. The standards, measures and ratings will be used in the assessment process for the Contractor Performance Assessment Report (CPARS).

3.3 Contract Monitoring

It is the intention of both parties to conduct joint reviews prior to the expiration/or exercise of an option year of the contract to determine and evaluate that services are being performed in accordance to contract terms, to include invoicing and payment are being handled in accordance to contract terms. The parties will jointly determine if this agreement is satisfactory to both parties in the terms of services provided and consideration received. This review may include but not limited to:

· Analyze all billings

· Analyze all payments

· Analyze cost administrative issues

· Veteran beneficiary satisfaction

· Quality of Service

· Other documentation related to services received.

Table A – Performance Standard Measures

1. Customer Service

2. Veteran Satisfaction

3. Security Breaches/Privacy/HIPAA

4. Methods of Quality Assurance (QA) Surveillance Various methods exist to monitor performance. Reviews and reports will be conducted in compliance with VA Privacy and Information Security Standards. The PM or COR shall use surveillance methods listed below in administration of the QASP.

The COR and/or designated VA Contractor personnel will perform scheduled and unplanned inspections in accordance with the frequency and standards contained in this QASP to ensure Contractor compliance with the appropriate paragraphs of the PWS and will record the results of inspection, noting the date and time of inspection. If inspection indicates minor problems, the COR will notify the Contractor of the discrepancy. If inspection indicates serious problems in performance, the COR will notify the Contracting Officer, who in turn will notify the Contractor of the deficiencies for correction. The Contractor will be given a Contractor reasonable time after notification to correct the unacceptable performance if such correction is possible. The length of time allowed to correct the problem will depend upon the requirement and the deficiency, and the Contractor will be notified of the time allowed for correction when the deficiency is reported to the Contractor. Should the COR and contractor reach an impasse on any matter dealing with inspection and re-performance, the Contracting Officer will be consulted for Contractor resolution.

In addition to specific instructions that may be mentioned, the appropriate and standardized form that is to be used for documentation of QA surveillance is the Surveillance Activity Checklist (Appendix 5).

4.1 Customer Feedback – Customer feedback may be obtained either from the results of formal customer satisfaction surveys or from random customer complaints. Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed and must be forwarded to the PM.

The PM shall maintain a summary log of all formally received customer complaints as well as a copy of each complaint in a documentation file. The PM shall also keep the tabulated results of all customer satisfaction surveys on file and shall enter the summary results into the Surveillance Activity Checklist (Appendix 5).

4.2 Verification of Documentation Provided by Contractor - This level of inspection shall be accomplished by monitoring and documentation. Each month, the PM or if there is a Contractor designated Technical Monitor, shall review the generated documentation and enter summary results into the Surveillance Activity Checklist (Appendix 5).

4.3 Periodic Inspection - Periodic inspections shall be conducted if and when specified in individual task orders. For the potential tasks that have been identified, Contractor’s appropriate Technical Monitor included in this QASP typically performs the periodic inspection on a monthly basis.

4.4 Random Monitoring - Random monitoring shall be conducted if and when specified in individual task orders. For the potential tasks that have been identified Contractor’s appropriate Technical Monitor included in this QASP shall randomly be performed the inspection.

4.5 Acceptable Quality Levels - The acceptable quality levels (AQLs) included in Performance Requirements Summary (Appendix 1) Table, for contractor performance are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls.

4.6 Process for Missed Targets

Process for justification/explanation of missed targets:

· Contracting Officer will notify the Contractor of failure to meet target/metric/performance standard

· Contractor must submit a written appeal to the CO within 10 days of receipt of notification of failure to meet the standard. The appeal must contain the following:

· An explanation for the failure to meet the standard

· A performance improvement plan (PIP)

· A date for completion of the performance improvement plan, and for regaining satisfactory target goals

· If standards are not met, the Contracting Officer may take appropriate steps to remedy the Contractor’s failure to meet the terms and conditions of the contract.

5. Quality Assurance Documentation

5.1 The Performance Management Feedback Loop The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the performance requirements summary shown in Appendix 1.

5.2 Monitoring Forms

The government’s QA surveillance, accomplished by the COR, will be reported using the monitoring forms in Quality Assurance Monitoring Form (Appendix 2) and Customer Complaint Investigation (Appendix 3). The forms, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required minimum quality service are being achieved.

5.2.1 The COR will retain a copy of all completed QA surveillance forms in COR File as well as upload to eCORS.

6. Analysis of Quality Assurance

6.1 Determining Performance

Government shall use the monitoring methods cited to determine whether the performance standards/service levels/AQLs have been met. If the contractor has not met the minimum requirements, contractor may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels shown in Appendix 1.

6.1.1 Performance Documentation

The COR will, in addition to providing documentation to the Contracting Officer maintain a complete Quality Assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the Government’s performance of the quality assurance function, including the originals of all Surveillance Activity Checklists (Appendix 5). All such records will be retained for the life of this contract. The COR shall forward these records to the Contracting Officer at termination or completion of the contract.

· Contractor performance on this contract will be documented in the manner described in 6.4.3:

· The Government will document performance that meets the minimum expectation or exceeds the expectations. A Performance Report will be used for this documentation effort and is to be used whenever the COR performs monitoring/surveillance (i.e. inspections, visits, etc.) on the contract. A template for this Report can be found at Appendix 2. All Performance Report documentation will be placed in the COR file. It is important to note that these Performance Reports can become part of supporting documentation for appropriate actions taken under this contract.

· The Government will document performance that did not meet the minimum expectation that can be a serious problem resulted from the performance or recovery is not likely to be considered during the course of performing the contract. When unacceptable performance is observed, in addition to completing a Performance Report, the COR will complete a, Contract Discrepancy Report (CDR). An example of this form can be found at Appendix 3. This form serves as the official written notification to the contractor of an observed contract discrepancy. (Note: Only under unique circumstances will verbal communication be used to initially inform the contractor of a potential contract discrepancy. However, in those cases, the COR will follow-up with written notification of the finding as contractor on as possible, coordinate it through the CO, and provide to the contractor.) As part of the CDR process:

0. The contractor shall acknowledge receipt of the CDR in writing.

0. The contractor will prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid recurrence.

0. The CDR will clearly indicate a due date by which the contractor has to present this corrective action plan to the COR.

The Government will review the contractor’s corrective action plan to determine acceptability.

· Summary of Methodology & Purpose for Performance Documentation. The reports & forms referenced in this QASP are designed to collectively serve as a traceable documented history of contractor performance. These will align Contractor support recordkeeping requirements mandated by the VA Contracting Officer’s Representative Training (CORT) tool, as well as annual performance assessments mandated by CPARS. Specifically:

· Quality Assurance Monitoring Form (Appendix 2) -- Used by the COR whenever performing contract monitoring/surveillance on the contract. Both negative and positive aspects of the contractor’s performance are documented. The information recorded in Performance Reports can serve as a significant contractor source of documented contractor performance. This information can subsequently be used in developing the COR Monthly Report to the Contracting Officer.

6.2 Reporting

At the end of each month, the COR will prepare a written report for the CO summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes the contractor’s submitted monthly report and the completed quality assurance monitoring forms (Appendix 2), will become part of the QA documentation. It will enable the government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives.

6.3 Reviews and Resolution

The CO may require the contractor’s project manager, or a designated alternate, to meet with the COR, CO and other government IPT personnel as deemed necessary to discuss performance evaluation. The CO will define a frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor; however, if the need arises, the contractor will meet with the COR and CO as often as required or per the contractor’s request. The agenda of the reviews may include:

· Monthly performance assessment data and trend analysis.

· Issues and concerns of both parties.

· Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis.

· Recommendations for improved efficiency and/or effectiveness.

6.3.1 The QAR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

6.3.2 The COR and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification at the convenience of the CO/PM.

6.4 CPARS Annual Assessment

The Contractor will be annually assessed and rated for their performance and operation of the utility system listed per the contract. Metrics and methods are designed to determine rating for a given standard and acceptable quality level. The Contractor Performance Assessment Reporting System (CPARS) will be utilized to perform this assessment:

6.4.1 CPARS training can be accessed at: https://www.cpars.gov/ (Use “Assessing Official Representative” role) (Approx. 2 hrs. to complete)

6.4.2 The COR will establish an account in CPARS in order to officially contribute to the Government’s annual assessment of the Contractor’s performance. The COR’s input will carry significant weight in the development of the Government’s overall annual rating of the Contractor’s performance under this contract.

6.4.3 The COR will assign a rating to the Contractor’s performance under the contract using the following table:

· Few minor problems resulted from the performance; any corrective actions taken were highly effective.

· Contractor me minor problems resulted from the performance; any corrective actions taken were effective.

Contractor some minor problems resulted from the performance; corrective actions taken were satisfactory.

· Serious problems resulted from the performance, but recovery was still possible; corrective actions taken were either marginally effective or not fully implemented.

· Serious problems resulted from the performance, and recovery was not considered likely; corrective actions taken were ineffective.

6.4.4 The COR’s assessment and recommended rating will be reviewed by the Contract Specialist /Contracting Officer to ensure sufficient justification. Upon completion of the Government’s annual assessment and rating of the Contractor’s performance, the assessment and rating will be officially recorded in the Past Performance Retrieval System (PPIRS). Information in PPIRS will serve as a valuable Government Contractor source of contractor’s past performance information for future Contractor solicitations upon which the Contractor may offer.

a. PPIRS training can be accessed at:

http://www.ppirs.gov/webtrain/webtrain.htm

b. Contractor shall assign point of contact for CPARs reviews

5 QASP ANNUAL ASSESSMENT

To provide for changing quality assurance and quality performance conditions, due to changes in regulatory requirements and/or VA and contractor capabilities, the components of QASP measurement and reporting will be reviewed at least annually or more frequently if required by changes in regulatory requirements.

6 SUMMARY

The QASP serves as an important guiding document, providing the framework for government monitoring, surveillance, and coordination of the Contractor’s activities and responsibilities under this CRT Repair Services contract. The typical long-life of this contract, its many moving parts, and the fact that the Contractor manufactures, requires a close, long-term partnership between the government and the Contractor. The deliberate and methodical approach to the development and maintenance of this QASP will help ensure this partnership continues to be nurtured, resulting in long-term quality utility service to Department of Veteran Affairs.

7 LIST OF APPENDICES

Appendix 1 – Performance Requirement Summary Appendix 2 – Quality Assurance Monitoring Form Appendix 3 – Customer Complaint Investigation Appendix 4 - Submittals Table Checklist Appendix 5 - Surveillance Activity Checklist Appendix 6 – COR/PM Quarterly Report Instructions Appendix 7 - QASP Periodic Maintenance, Review, and Acceptance

Appendix 1: Performance Requirement Summary

The inquiry response database may not contain more than 5% of incorrect, inaccessible or inappropriate lines of information.

-100% Scheduled down times are announced 5 days ahead of event.

· Random inspection (site visits, telephone calls, remote access)

· Review of Contractor Operations Reports.

· Review of Telephone System Reports.

· Review database for accuracy, appropriateness and accessibility information.

· Reviewed automated systems data.

· Randomly verify

· Random assessments

· Review complaints

· Review quality control activities

· No OSHA reportable violations resulting from contractor action or inaction.

APPENDIX 2: Quality Assurance Monitoring Form

ROES ORDER NUMBER: _______________________________________________

CLINICIAN/CLINIC: ____________________________________________________

SURVEY PERIOD: _____________________________________________________

SURVEILLANCE METHOD (Check):

· Random Sampling

· 100% Inspection

· Periodic Inspection

· Customer Complaint LEVEL OF SURVEILLANCE (Check):

· Monthly

· Quarterly

· As Needed

ANALYSIS OF RESULTS:

Contractor’s Performance (Check):

· Meets Standards

· Does Not Meet Standards Narrative of Performance During Survey Period: ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Prepared By: ______________________________________________________________________ (Name, Title, Date)

APPENDIX 3: Quality Assurance Monitoring Form – Customer Complaint Investigation

CONTRACTOR: _____________________________________________________

BPA/TASK ORDER NUMBER: __________________________________________

DATE/TIME COMPLAINT RECEIVED: ____________________ AM / PM

SOURCE OF COMPLAINT: _____________________________ (NAME)

_______________________________ (ORGANIZATION)

_______________________________ (PHONE NUMBER)

_______________________________ (EMAIL ADDRESS)

NATURE OF COMPLAINT: ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________

RESULTS OF COMPLAINT INVESTIGATION: ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________

DATE/TIME SERVICE PROVIDER INFORMED OF COMPLAINT: _________ AM / PM

CORRECTIVE ACTION TAKEN BY SERVICE PROVIDER: ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________

RECEIVED AND VALIDATED BY: __________________________________________________

PREPARED BY: ___________________________________

DATE: _____________________

APPENDIX 4: Submittals Table Checklist

Contract Number: ___________________________________________________ POC/Phone/Email: __________________________________________________

Example references below are Contractor of the “common” submittals that are seen (but not all inclusive).

NOTE: Actual deliverable/submittal are very specific per contract. Submission may be required to match contract requirements.

Quality Control Plan

Safety & Accident Prevention Plan

Infection Control Plan

Customer Relationship Management Plan

APPENDIX 5: Surveillance Activity Checklist

Contract Number: ___________________________________________________ POC/Phone/Email: __________________________________________________

Standard Method of Measurement

Performance Metrics Method of Surveillance Date Accomplished Compliance (Exceeded, Met or Partially Met)

APPENDIX 6: PM/COR Monthly Report to Contracting Officer

The PM or COR shall complete either a Monthly Report depending on the phase of the contract to the Contracting Officer to document both negative and positive aspects of the Contractor’s performance for the contract. The report shall be submitted to the Contracting Officer in accordance with a mutually established submittal schedule.

The report shall minimally include the following information:

1. Contract Number

2. Contract Period of Performance

3. Contractor Name:

4. Method of Surveillance (ex: Direct Observation, Assessment of Deliverables, Review of Invoice/Payments):

5. Compliance: This document is in accordance with the Inspection of Services Clauses under FAR 52.246-4, 52.246-6, and the Quality Assurance Surveillance Plan Requirements.

6. Overall Assessment: Complete items 6a to 6f below by providing a detailed explanation in Item 7. Attach additional documentation for support evidence.

a. Quality of Work (Exceptional, Very Good, Satisfactory, Marginal, Unsatisfactory)

b. Deliverable provided by the contractor this moth (list here):

c. Was contract performance timely?

d. Are there any significant issues identified and should be brought to Contracting Officer’s attention?

e. Are actions being taken by the Contractor to correct discrepancies?

f. Customer Complaints?

i. # of Complaints

ii. # of Valid Customer Complaints

7. Additional Comments

8. Sign and Date the report.

APPENDIX 7: QASP Periodic Maintenance, Review and Acceptance

The functional team members’ signatures below indicate their review and acceptance of the Quality Assurance & Surveillance Plan, dated _________ , for (insert type utility system) services provided at (insert installation name) under contract (insert contract number). These signatures indicate coordination on this Government QASP at the time of (insert “development” or “revision”) and will remain in effect until the Plan is modified.

(insert title and organization)