Draft Quality Assurance) Surveillance Plan Attachment 2.pdf

FORM # USPTO-OP-014.v2 1

Electronic Business Center (EBC)

Quality Assurance Surveillance Plan (QASP)

ACQ-26-0475

FORM # USPTO-OP-014.v2 2

DOCUMENT REVISION HISTORY

REVISION

NUMBER DATE REVISION DESCRIPTION

FORM # USPTO-OP-014.v2 3

1. INTRODUCTION

(a) The United States Patent and Trademark Office (USPTO) is the federal agency for granting U.S. patents and registering trademarks. The strength and vitality of the U.S. economy depends directly on effective mechanisms that protect new ideas and investments in innovation and creativity. The continued demand for patents and trademarks underscores the ingenuity of American inventors and entrepreneurs.

(b) Within the Patents business unit, the Office of Patents Stakeholder Experience’s (OPSE) mission is to assist external stakeholders as they journey through the U.S. patent system and patent application processes. OPSE works with Patents external stakeholders to ensure they find it clear, consistent, and intuitive to do business with Patents, and for Patents personnel to be empowered and equipped with the knowledge and skills necessary to meet stakeholders' needs.

OPSE’s work emphasizes reducing barriers and burdens so that all innovators have the knowledge, understanding and resources needed to navigate the Patents system and processes with ease.

(c) The Patent Electronic Business Center (EBC) provides comprehensive technical support and electronic filing assistance to external stakeholders utilizing the USPTO's electronic patent systems, platforms, and tools. This support covers submitting and reviewing electronic patent applications via Patent Center, along with managing applications that have already been filed.

(d) The EBC proudly services the intellectual property (IP) community with extended hours of operations during which it provides technical assistance on a wide range of support topics related to these systems and tools.

(e) The USPTO’s OPSE requires support for the EBC processing activities. The EBC provides technical support services to external Patent stakeholders who file or access patent information electronically or use other EBC website features and external Trademark stakeholders using the Trademark filing system for identity (ID) verification.

(f) Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the contracting officer or a duly authorized representative.

(g) This PWS covers a 12- month base period, four (4), 12-month option periods, and a 6 month extension period for a total estimated contract value of $25,614,906.24.

Period of Performance Start Date End Date Base Period June 15, 2026 June 14, 2027 Option Period 1 June15, 2027 June 14, 2028 Option Period 2 June 15, 2028 June 14, 2029 Option Period 3 June 15, 2029 June 14, 2030 Option Period 4 June 15, 2030 June 14, 2031

FORM # USPTO-OP-014.v2 4

Option Period 5 (52.217-8; 6-month extension)

June 15, 2031 December 14, 2031

(h) This is a USPTO authored Performance Work Statement (PWS). OPSE and the CORs developed the PWS and QASP to ensure effective oversight of this requirement. Peformance will be evaluated using a combination of methods, including variations of random sampling, internal and external stakeholder’s feedback, monthly and adhoc reporting, deliverable inspection, etc.

These monitoring techniques, applied in accordance with the defined performance metrics, will enable the Government to verify that the Contractor’s performance consistently meets the requirements and support the agency’s mission needs.

2. CONTRACT TYPE AND REQUIREMENTS

The USPTO anticipates awarding a single labor hour contract.

3. SURVEILLANCE PLAN OBJECTIVE

Performance-based and high-risk

The purpose of this Quality Assurance Surveillance Plan (QASP) is to describe the systematic methods that the USPTO will use to monitor contract performance, and to identify the required documentation and the resources to be employed. This QASP provides a means for determining whether the contractor is meeting the performance standards/quality levels identified in the Performance Work Statement (PWS), and to ensure that the government pays only for the level of services received.

This Plan states what, how, when, and by whom the Government will evaluate contractor performance to determine whether the contractor has met the specified performance standards.

This QASP is not intended to duplicate the contractor’s Quality Control Plan (QCP). The contractor, not the Government, is responsible for management and quality assurance actions to successfully meet the terms of the contract.

Considering the contract type selected for this effort, the team included surveillance methods that will ensure effective cost/price control in accordance with FAR 16.202-2 for firm-fixed-price conracts and 16.601(c)(1) for time-and- materials or labor hour type contracts.

This is a living document, and provided to the contractor to facilitate communication, cooperation, and successful completion of the contract. Flexibility is required to allow for an increase or decrease in the level of Government surveillance necessary based on contractor performance. This Plan will be updated as needed during contract administration.

FORM # USPTO-OP-014.v2 5

4. ROLES AND RESPONSIBILITIES/TEAM MEMBERS AND QUALIFICATIONS

4.1 USPTO Personnel

Contracting Officer (CO)

The Contracting Officer (CO) has overall responsibility for the administration of all USPTO contracts and orders. The USPTO CO is part of the USPTO contract management team and is the only individual authorized to take actions on behalf of the Government to approve, amend, modify, or deviate from the contract terms, conditions, requirements, specifications, details, and/or delivery schedules. Within that authority, the CO has the ability to delegate to the Contracting Officer’s Representative (COR) for the everyday administration of the contract. The CO, who is responsible for the overall administration and final closeout of the contracts, and as it relates to the implementation of the QASP, shall, perform the following duties, which are representative but not all-inclusive:

• Ensure performance of all necessary actions for effective contracting;

• Ensure compliance with the contract terms;

• Safeguard the interests of the USPTO in the contractual relationship;

• Ensure that the contractor receives impartial, fair, and equitable treatment under this contract;

• Ensure surveillance is adequately addressed in the contract and work with the team to resolve pricing, invoicing, delivery, and performance issues;

• Determine the final assessment of the contractor’s performance;

• Issue all modifications against the contracts

• Negotiate and issue contract modifications

• Resolve contractor claims and disputes

• Issue cure notices (notification that unless unacceptable performance is corrected, the

Government may terminate the contract for default, IAW FAR 49.607)

• Issue show-cause letters (following a cure notice, requesting facts bearing on the case)

• Terminate contracts

Assigned CO: Kelly Gustafson Qualifications: FAC-C Level 3 Certified, Lead Contracting Specialist

Contract Specialist (CS) The CS will assist the CO to ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and will safeguard the interests of the USPTO in the contractual relationship. The CS will assist the CO to ensure surveillance is adequately addressed in the contract and work with the team to resolve pricing, invoicing, delivery, and performance issues. With CO concurrence, the CS may attend meetings and may represent the interests of the CO.

Assigned CS: Denise Sanders Qualifications: FAC-C Level 3 Certified

FORM # USPTO-OP-014.v2 6

Contracting Officer’s Representative (COR) In accordance with the COR Appointment Memorandum, the COR will be responsible for providing day-to-day, continuous technical oversight and surveillance of the contractor’s performance. The COR will use this Plan to conduct the oversight/surveillance process. The COR will keep surveillance documentation in the COR file that accurately documents the contractor’s actual performance. The COR is responsible for reporting early identification of performance problems to the CO. The COR is required to provide an annual past performance assessment to the CO. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

In implementing this QASP, the COR responsibilities includes, but not limited to managing Quality Assurance (QA) activities to ensure compliance and completeness by:

• Overseeing the quality aspects of the project

• Serving as the technical liaison with the Contractor Quality staff

• Conducting final inspection and acceptance of all reports and deliverables

• Other duties as specified.

Assigned COR: Arthur Banks Qualifications: FAC-COR Level 3

Alternate Contracting Officer’s Representative (ACOR) An ACOR may be needed for contract administration, including surveillance and risk mitigation.

If an ACOR is appointed, the team will document how they will work together and differentiate responsibilities. The USPTO ACOR is part of the USPTO contract management team. Specific responsibilities are outlined in the ACOR Appointment and Delegation Memorandum. The ACOR responsibilities may include all of the same COR responsibilities (see above and the Appointment Memorandum) exercised in the absence of the COR.

Assigned ACOR: Shellie Eaton Qualifications: FAC-COR Level 3

Alternate Contracting Officer’s Representative (ACOR) An ACOR may be needed for contract administration, including surveillance and risk mitigation.

If an ACOR is appointed, the team will document how they will work together and differentiate responsibilities. The USPTO ACOR is part of the USPTO contract management team. Specific responsibilities are outlined in the ACOR Appointment and Delegation Memorandum. The ACOR responsibilities may include all of the same COR responsibilities (see above and the Appointment Memorandum) exercised in the absence of the COR.

Assigned ACOR: Lissette Villafan Qualifications: FAC-COR Level 3

Subject Matter Expert (SME)

FORM # USPTO-OP-014.v2 7

SMEs may be needed to advise and provide subject matter expertise to the COR/ACOR(s) for contract administration, including risk mitigation and surveillance. If SMEs are utilized, the team will document how they will work together and differentiate responsibilities. SMEs are part of the USPTO contract management team. Individuals acting as SMEs are not and will not act as CORs or ACORs. The following individuals are SMEs for EBC:

Assigned SME: Sean Hunter Qualifications: Deputy Commissioner for International Patent Coopertion

Assigned SME: Jolyn Eley Qualifications: Office of Patents Stakeholder Experience (OPSE)

Assigned SME: Richard Fernandez Qualifications: Patent Center Lead Product Owner (LPO)

Assigned SME: Matt Schwab Qualifications: Senior Legal Administrator for Trademarks Technology and Systems, Office of the Deputy Commissioner for Trademark Administration

Program/Project Manager (PM) The PM provides primary program oversight, nominates the COR, ensures the COR is trained before performing any COR duties, and supports the COR’s performance assessment activities.

The PM may provide input to the COR. They are not empowered to make any contractual commitments or any contract changes on the government’s behalf.

Assigned PM: Marla Robinson Qualifications: OPFM Acquisition Division Director

4.2 Contractor Personnel

Contractor Project/Program Manager The Contractor PM will perform the following duties, which are representative but not all-inclusive:

• Oversee the contractor’s risk mitigation and quality assurance procedures and plan;

• Provide deliverables and metrics to the COR;

• Respond to questions and discrepancy reports;

• Meet with the USPTO COR on a bi-weekly or on an as needed basis to review contract quality, pricing, and schedule.

• Monitor daily throughput of the Cotnractor’s work

• Notify the Government of problems and concerns as soon as practical

• Ensure smooth functioning of the services required under contract which includes planning, executing, and controlling project as indicated Contractor Team Leads The Contractor Team Leads will perform the following duties, which are representative but not all-inclusive:

FORM # USPTO-OP-014.v2 8

• Oversee individuals assigned to support the requirements

• Provide back-up support when PM is unavailable.

• Manage and oversee day-to-day contract operations

• Delegate and monitor work assignments

• Train agents as needed

• Provide support during escalation process

• Problem solving

• Ensure clear and timely communication among team members, PM and Government management.

• Meet with USPTO COR on a bi-weekly or on an as needed basis to review contract quality, pricing and schedule.

• Provide required monthly and adhoc reporting when PM is unavailable

• Oversee Quality Assurance

5. DELIVERABLES AND SCHEDULE

The deliverables below are listed in the PWS requirement document. The deliverables will assist the COR and OPSE in ensuring that all services are delivered as scheduled in accordance with the terms and the conditions of the contract. .

PWS

Section #

Deliverable

Delivery Dates

Delivery Format

3(e) NDA (Attachment 1) Prior to commencement of services

Via email agreed upon using Microsoft suite of tools as designated by the COR/Government POC

3(f) Contractor POC Two (2) business days after award.

Via email agreed upon using Microsoft suite of tools as designated by the COR/Government POC

6(a) Kick-off Meeting & Minutes Meeting: No later than five (5) business days after award

Minutes: Two (2) business days after Kickoff Meeting

Meeting via Teams with CO, Contract Specialist, COR and designated Government POC(s)

6(b) Periodic or ad-hoc Meeting; As Required Meeting via Teams with CO, Contract Specialist, COR and designated Government POC(s)

FORM # USPTO-OP-014.v2 9

6(c) Agenda(s); Minutes As Required Via email agreed upon using Microsoft suite of tools as designated by the COR/Government POC

6(d) Patents EBC Monthly Report Monthly – 15th of each month by 5pm

Via email agreed upon using Microsoft suite of tools as designated by the COR/Government POC

6(d) Trademarks EBC Monthly Report

Monthly – 15th of each month by 5pm

Via email agreed upon Microsoft suite as designated by the COR/Government POC

6(d) Combined EBC Report Monthly – 15th of each month by 5pm

Via email agreed upon Microsoft suite of tools as designated by the COR/Government POC

6(d) & 10

Performance Metrics with respects to any Training, Outreach, Demonstration, etc. in the Knowledge Transfer Section

10th of each month Via email agreed upon Microsoft suite of tools as designated by the COR/Government POC

6(d) Overall Monthly Program Status Report -Status of Tasks -Planned Activities -Management Issues -Other Sections as needed by COR or Contractor Program Manager

10th of each month Via email agreed upon Microsoft suite of tools as designated by the COR/Government POC

6(d) Ad-hoc Report As Required Via email agreed upon Microsoft suite of tools as designated by the COR/Government POC

6(f) USPTO-specific and specialized training

As Required Via email, submission of completion agreed upon format

8(a) Quality Control Plan Presented at Kick-off meeting

Meeting via Teams with CO, Contract Specialist, COR and designated Government POC(s)

14(d)(1) AWS (Attachment 3) Within ten (10) business days after receipt

Via email

FORM # USPTO-OP-014.v2 10

16.1(e) Key Personnel resumes (substitution)

As Required; Via email agreed upon Microsoft suite of tools as designated by the COR/Government POC

6. CONTRACTOR PERFORMANCE MONITORING

Under the EBC contract, the Contractor is required to utilize Government Furnished Equipment (GFE) and USPTO Automated Information Systems (AIS) to perform assigned tasks.

To monitor schedule and performance, The COR Team will conduct routine oversight activities, including but not limited to: randomized inspections, monthly invoice reviews, bi-weekly COR meetings with Contractor and Government Teams, monthly Government Team discussions meetings, etc. to ensure that the Contractor meets the requirements of the Contract in line with the performance metrics of the PWS.

The COR Team will also monitor price by tracking hours worked per labor category and compare against the Contractor’s billed hours for each invoice period. Supporting documentation such as timesheets or other verifiable records will be reviewed to ensure accuracy and confirm that the Contractor is compensated only for actual work performed.

The COR and CO will meet with the contractor bi-weekly to discuss cost, schedule, and performance. At each of the meetings the Government may raise specific concerns identified via the surveillance process. Additionally, the Government may raise a concern immediately as soon as it is identified. Copies of meeting slides and written meeting minutes will be filed in the COR file and provided to the Contractor, with the expectation that action items be addressed in the next meeting.

As needed and on an ongoing basis, the COR will solicit input from Business Units using the Customer Feedback Form (attached). If an incident is identified, the COR will request the user to detail their concerns. Documented concerns will be addressed with Contract Management staff and the COR team in the bi-weekly meetings. If immediate action is required, a meeting will be scheduled to address, correct, and document the issue and the resolution will be added to the bi-weekly meeting. All customer feedback forms will become part of the COR Electronic Contract File.

7. SURVEILLANCE DOCUMENTATION

Evidence of USPTO surveillance throughout the life of the contract will be filed in the COR file;

part of the official Government record. Documented surveillance will be used to substantiate invoice review, write a past performance evaluation in accordance with FAR 42 and PTAM Subpart 42.15, and may be used to resolve conflicts during contract performance. Data provided by the Contractor is reviewed and analyzed for trend analysis. Any outliers in the data is compared to data pulled from the Patent Center/USPTO Customer Contact Management System.

Investigation into the discrepancy is discussed with the Contractor and all Government involved parties.

FORM # USPTO-OP-014.v2 11

Attached are the following forms the COR and team will use to document surveillance:

1. Performance Metrics

2. COR Quality Assurance Monitoring Form

3. SME Quality Assurance Monitoring Form

4. Contractor Discrepancy Report (CDR)

5. USPTO Internal Customer Feedback

Copies of the above reports will be maintained in the electronic COR contract file and shall be used for CPARs and in the coase of any discrepancy to ensure accuracy is achieved in reporting.

8. SURVEILLANCE PROCESS

8.1 The COR, Government SMEs and the CO (optional) will meet with the Contractor on a bi-weekly and as-needed basis to discuss cost/price, schedule, and performance. During these meetings, the Government may raise concerns as soon as they are identified, including specific issues observed through the surveillance processes described below. Meeting materials, agendas, and minutes will be documented as part of the COR file. The surveillance processes outlined below may be utilized throughout the duration of the contract:

(a) Surveillance Methods

• 100% Inspection o All Invoices and financial management reports will have 100% inspection by the COR.

(b) Random Surveillance

• Randon inspection is scheduled for monitoring the technical performance of the following items:

o Task Management and Documentation o Meetings and Reviews o Software Functionality o Quality Software Testing o Management Effectiveness and Contract Administration o Information Disclosure and Government Furnished Equipment (GFE) Management

(c) Customer Feedback

• Customer Feedback may be necessary for certain types of tasks that do no lend themselves to random sampling of 100% inspection. Customer feedback may be obtained from the results of formal customer satisfaction surveys or from random customer complaints. The CO may use validated customer complaints as the basis for actions against the Contractor.

(d) Contractor Metrics 1.The Contractor will report status and performance. See “Deliverables” cited under Section

5 of this QASP and the PWS.

(e) Trend Analysis

FORM # USPTO-OP-014.v2 12

2.Monthly report information will be compiled quarterly to identify trends in quality, schedule and cost control. Quarterly performance data will be used to determine trends in performance throughout the contract delivery period or every 12 months if Period of Performance (POP) exceeds 12 months. In the event quality deficiencies are identified as part of the monthly surveillance, positive trend analysis that demonstrates progress towards achieving acceptable quality levels may be used by the COR to determine if additional corrective action is required or not. Negative trends in any of the performance elements will be used to facilitate discussions with the contractor to determine more suitable quality control mechanisms to correct the downward trending element and achieve the acceptable quality levels within the delivery period.

8.2 Steps in the Surveillance Process

This section provides the details on the surveillance process. Actual surveillance must be completed and this process shall implement the methods identified in Section 8.3 to include scheduling, performing, resolving performance issues discovered through surveillance, and documenting surveillance.

8.3 Scheduling Surveillance

Surveillance by the COR monthly included in QASP Attachment 1. USPTO will assess quality, schedule and cost performance measures using the deliverables, task performance standards and documentation delivered by the contractor. These measures will be documented and discussed with the contractor throughout the period of performance to ensure USPTO is receiving quality performance and deliverables within the agreed schedule and cost parameters.

8.4 Performing Surveillance

Surveillance will be performed according the methods described in this QASP and meeting the schedule established between the contractor and the organization’s personnel. The Contractor shall provide whatever assistance is required in the surveillance procedures. The Performance Requirements Summary (PRS) (QASP Attachment 1) is the list of performance objectives and standards that must be performed by the contractor. This PRS details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the perfor-mance element standards. The Acceptable Quality Level (AQL) performance Standard Table should be used to form the foundation of the COR’s inspection checklist.

FORM # USPTO-OP-014.v2 13

8.5 Resolving Performance Issues

It is not unprecedented that USPTO and the contractor may discover performance issues or deficiencies through the surveillance process. The COR will provide a list of any unresolved deficiencies to both the USPTO CO and contractor's Quality Assurance personnel to afford him/her the opportunity to show where the problem is already in the process of being resolved. If evidence is shown where the contractor’s quality program has already documented the deficiency and timely corrective action is resolving the problem, the COR will follow up with the contractor’s quality assurance personnel to ensure the deficiencies are remedied in a timely manner, in accordance with the PWS.

All deficiencies associated with performance requirements listed on the Performance Requirements Summary, whether remedied by the contractor or not, will be included in the COR/Government point-of-contacts (POCs) Status Report regardless of who identified the deficiency.

The COR and the Contractor will meet bi-weekly, monthly, quarterly or as needed to discuss contract performance. The Contractors QA Personnel will provide a copy of the meeting minutes to all attendees within 5 workdays of the meeting. Should the need arise for an emergency meeting because of unexpected performance issues, these meetings can be called by either COR or the Contractor’s quality personnel.

8.4 Documenting Surveillance

• Acceptable Performance The Government shall document acceptable performance a minimum of each Month and on an Ad-Hoc basis as required. Any report may become part of the supporting documentation for Contractor Performance Assessment Reporting System (CPARS) entries, or other actions.

• Unacceptable performance Unacceptable contractor performance can be identified by the COR as part of the routine surveillance and oversight, through customer complaints or during monthly scheduled (or Ad- Hoc) surveillance evaluations. When unacceptable performance occurs, the COR shall inform the contractor and document the discrepancy in a “Performance Spotlight”. This will normally be in writing unless circumstances necessitate verbal communication. The COR shall document the discussion and provide a copy to the CO. The Performance Spotlight entry includes a description of the issue, severity, status, and resolution. The CO will then disseminate to the contractor if the issue requires formal written communication.

The Contractor shall acknowledge receipt of the formal written communication in writing. The Performance Spotlight will specify if the contractor is required to prepare a corrective action plan to document how the Contractor shall correct the unacceptable performance and avoid a recurrence. The Performance Spotlight will also state how long after receipt the contractor has to present this corrective action plan to the CO and COR. The Government shall review the Contractor's corrective action plan to determine acceptability. The Government may withhold payment until the corrective action plan is accepted and/or corrective services are performed by the Contractor. Any restrictions on invoices will be communicated to the Contractor through the

FORM # USPTO-OP-014.v2 14

CO. Any corrective action report may become a part of the supporting documentation for CPARS or other actions deemed necessary by the CO.

• Mandatory or regulatory compliance items:

The contractor will provide initial proof of compliance with regulatory and compliance items contained in the contract. The COR and the CO will evaluate the initial submission and will conduct ad hoc reviews of the contractor's compliance throughout the remainder of the contract.

The Contractor will monitor compliance to all statutory and regulatory compliance items and report any issues to the CO.

• Surveillance Folder A surveillance folder will be developed and maintained by COR who is assigned to accomplish QA for a performance requirement. The folder is typically maintained in a computer database/file system/collaborative site provided there is adequate back up of the data to preclude accidental loss. The surveillance folder must contain at a minimum, the following documents and be set up using the QAP File Index:

• Appointments. COR nomination letters and copy of the CO’s Letter to the Contractor appointing the new COR to the contract.

• Contract Documents. The contract with all modifications, directive documents, references, pertinent terms defined by the contract, equipment listings, PWS and QASP.

• Special Reports Documentation. Minutes of the monthly performance meetings that are held with the COR, CO and the contractor should be recorded to encourage good communication and to resolve issues before they become significant challenges.

• Surveillance Paperwork. Documented surveillance inspections performed by the COR will be kept in this section. All paperwork that supports the documented surveillance should be kept in this section as well. The types of information that may be included are details of inspections or data gathering, conversations or meetings with the contractor, notes and comments that support the inspection paperwork. Quality Assurance Surveillance Reports will be kept in this section.

• Unacceptable Performance Documentation. In accordance with FAR 49.607, a section used for filing all documentation associated with contract quality assurance, e.g., Customer Complaints (both active and resolved), notices of contract deficiencies, Customer Discrepancy Report, the CO’s inputs/determinations, Cure Notices, Show Cause, and all supporting documentation.

• Final Acceptance and Funding Issues. In this area, keep paperwork for future changes to the contract, i.e., addendums, options, etc. that must be worked during the life of the contract.

8.5 Surveillance Standards and Tools

The COR will assess the Contractor’s performance to ensure the Contractor is performing up to the specified standards. The performance standards in the Performance Work Statement (PWS) are further specified in the Performance Requirements Summary (PRS) matrix (QASP Attachment 1). Performance will be documented by the contractor in their Monthly Report and documented by the COR in the VPMt at least Monthly and more frequently if performance is exceptionally good or significantly deficient.

FORM # USPTO-OP-014.v2 15

8.6 Acceptable Quality Level (AQL):

The AQLs for this product are defined in the PRS included in this document, and all deliverables must meet the quality specifications defined in the PRS.

8.7 Evaluation Method:

The COR will review documents and deliverables to ensure AQL’s have been achieved as part of the Monthly review and Monthly Vendor Performance Monitoring Tool submission. The COR will ensure that transition activities meet the required AQLs by reviewing deliverables and documentation; additionally, the COR may conduct discussions with the contractor, users, and other stakeholders, as required. The COR will ensure the activities meet required AQLs by reviewing deliverables and documentation; additionally, the COR may conduct discussions with the contractor, users, and other stakeholders, as required.

8.8 Performance Rating

In evaluating the quality of the contractor’s performance, the following performance ratings may be used (same as CPAR ratings).

Performance Rating Criteria

Exceptional

Performance meets contractual requirements and exceeds many to the government’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and contractor actions were highly effective.

Very Good

Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal

Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear marginally effective or were not fully implemented.

FORM # USPTO-OP-014.v2 16

Unsatisfactory

Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

8.9 Quality Assurance Standards

See PRS QASP Attachment 1

8.10 Tools

The Vendor Performance Management Tool will be used to routinely report performance each month. CPARS will be reported annually as required. Performance will be documented monthly and quarterly to demonstrate acceptable quality for all deliverables and services performed under this contract.

9. OTHER QA COMMUNICATION AND REPORTING

All quality issues that may arise during the execution of this task all will be managed through the resolution methods defined in the QASP.

10. RESOLVING QUALITY ASSURANCE ISSUES

All quality issues that may arise during the execution of this task will be managed through the resolution methods defined in the QASP.

10.1 Negative Incentives

If any of the performance requirements do not meet the Target Metric set in the PRS table, the COR shall document the discrepancy(ies) and shall notify the CO and Contractor promptly, on a monthly basis, for appropriate action. When the performance is below the Target Metric standard, the Government may increase surveillance and/or Contractor reporting requirements, as well as, documentation on the Contractor’s monthly performance report, interim and annual CPARS reports.

10.2 Notification

The COR will document any issues in the monthly performance report and notify the Contracting Officer, in writing, of unacceptable quality levels. The Contracting Officer will promptly provide written notification of discrepancies on a monthly basis, or immediately if the situation significantly affects the well-being of the product delivery. The Contractor will be given the opportunity to respond, in writing, to each discrepancy.

FORM # USPTO-OP-014.v2 17

10.3 Response

The Contractor shall provide a written response to the cause and the corrective action to prevent recurrence within 10 working days after receipt of Contracting Officer’s notification. The Contractor should cite specific quality assurance program procedures or new procedures instituted to prevent recurrence.

11. QASP ATTACHMENTS:

Table 1 QASP Attachments

Attachment # Title Description File

1 Performance Requirements Summary

(PRS)

The PRS is the culmination of quality planning for each specific task based on the PWS. The PRS converts the deliverables/objectives listed in the PWS into the specific requirements, expectations, and measures for this specific work product(s).

EBC Performance Metrics.docx

FORM # USPTO-OP-014.v2 18

USPTO Internal Customer Feedback Form – Sample (Tailor to your requirement)

Date and Time: 2/14/XX at 11:05 a.m.

Source of Comment:

Line Office: USPTO

Branch: OCIO

Individual: Joe XXXX

Nature of Comment: Called 3 times between 9:00 a.m. and 11:00 a.m. on Monday 2/14/2015 for Helpdesk service to my laptop and the phone was unanswered.

Contract Reference: 5, 5.4.1, and PRS Table

Validation Contract requires a one-hour response time. Complaint is valid

Date and Time Contractor Informed of Comment: 2/14/XX at 11:30 a.m.

Action Taken by Contractor: Contractor had a person out sick and did not have backup Helpdesk personnel assigned. Contractor has now developed a roster of backup Helpdesk personnel who are forwarded Helpdesk calls during normal business hours.

Received and Validated By: XXXXX/COR

FORM # USPTO-OP-014.v2 19

Contract Discrepancy Report (CDR) - Sample (Tailor to your requirement)

CONTRACT DISCREPANCY REPORT

1. Contract No.: XXXXXX 2. Report No. for this Discrepancy: XXX

3. To: (Contractor and Manager’s Name)

4. From: (Name of COR)

5. Date Prepared: XX/XX/XXXX Date Returned by Contractor: XX/XX/XXXX Date Action Completed: XX/XX/XXXX

6. Discrepancy or Problem: (Describe in detail; include reference to PWS Directive; attach continuation sheet if necessary)

[There have been 20 unscheduled Helpdesk requirements during this quarterly surveillance period (Jan – Mar). On four of the requirements, the contractor did not respond within four minutes of the agreed time as required by 5.2.2 and the PRS item 3.

On all four occasions, the service was delivered over 5 hours late. The performance requirement is 85%, therefore the requirement was performed unsatisfactorily since only 3 noncompliance events are allowed.]

7. Signature of Contracting Officer:

8. To: (Contracting Officer) From: (Contractor)

9. Contractor response as to Cause, Corrective Action, and Actions to Prevent recurrence;

Attach continuation sheet if necessary. (Cite applicable quality control program procedures or new procedures)

10. Signature of Contractor Representative:

Date:

FORM # USPTO-OP-014.v2 20

11. Government Evaluation (Acceptance, partial acceptance, rejection; attach continuation sheet if necessary)

12. Government Action (Reduced payment, cure notice, show cause, other)

Resolution

Name – Title Signature Date

Contractor Notified

FORM # USPTO-OP-014.v2 21

COR QUALITY ASSURANCE MONITORING FORM

Sample (Tailor to your requirement)

SERVICE or STANDARD:

[IT Help Desk: Synopsis of survey results and survey results (raw data). Contractor must receive average of 4 or above on scale of 1- 5.]

SURVEY PERIOD:

[1/1/2025 – 1/30/2025]

SURVEILLANCE METHOD:

Customer Input (Surveys)

FREQUENCY OF SURVEILLANCE:

Monthly

RESULTS:

[The COR reviewed all of the surveys submitted this month considering there were only 20. The COR reviewed the synopsis provided by the contractor and it matches the raw results. The contractor received a 4.5 overall. This exceeds the expectation of 4 and above.

There were two instances where the contractor went above and beyond to the benefit of the Government. 1) the Government was able to fix a computer in less than the standard time putting the person back to work earlier than expected. (see survey # XYZ) 2) There was a particularly difficult challenge that the contractor overcame and shared with the Government and the USPTO staff were able to take action so that others did not have the same issue (see survey # ABC). In both instances, the contractor personnel kept the employees informed with helpful and professional communication.

The contractor is maintaining a trend above 4 for the past 6 months.]

FORM # USPTO-OP-014.v2 22

RATING1:

Very Good - Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

PREPARED BY: [COR PIV signature with date]

1 See FAR section 42.1503 for rating definitions.

FORM # USPTO-OP-014.v2 23

Sample (Tailor to your requirement)

SME QUALITY ASSURANCE MONITORING FORM

Submit to the COR by the 15th of each month

Rate each area using the following 5 categories: Unsatisfactory, Marginal, Satisfactory, Very Good, and Exceptional. Full definitions are found in FAR section 42.1503. Comments are required for ratings of unsatisfactory, marginal, and exceptional.

Area Rating2 Comments

Technical Exceptional As always, the team exhibits high technical expertise with everything that they do.

Schedule/Timeliness Exceptional Releases were on time and defect releases were deployed promptly.

Management or Business Relations

Exceptional Members stays in touch with the tech lead and ensures that I am aware of what we need to know. They are very easy to work with and very accommodating.

Regulatory Compliance Exceptional Entire team is constantly aware of compliance requirements

2 See FAR section 42.1503 for rating definitions.

RESULTS: