1333BJ21R00151001 - PTS SOW Attachment 11 Quality Assurance Surveillance Plan (QASP).pdf

FORM # USPTO-OP-014.v1.0 (06/2015) 1

Patent Technical Services (PTS)

Attachment 11

Quality Assurance Surveillance Plan

(QASP)

FORM # USPTO-OP-014.v1.0 (06/2015) 2

REVISION HISTORY

REVISION

NUMBER DATE REVISION DESCRIPTION

DOCUMENT CONVENTIONS

This document is a template for a Quality Assurance Surveillance Plan (QASP). This guide, along with the Performance Work Statement (PWS), should be used when creating a QASP for USPTO performance based contracts. This template must be supplemented with program-specific information to produce a QASP that accurately describes the surveillance methods, specific roles and responsibilities, and performance requirements for the USPTO.

FORM # USPTO-OP-014.v1.0 (06/2015) 3

1. INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirement listed in the statement of work for Patent Techinical Srvices (PTS) contract. This plan sets forth the procedures and guidelines that Office of Patent Application Processing (OPAP) will be using to ensure the required quality performance standards or services levels are achieved by the Contractor.

2. PURPOSE

The QASP is used to inspect and monitor performance of work being performed by the Contractor. The QASP provides a means for evaluating whether the Contractor is meeting the performance standard/quality levels identified in the Statement of Work (SOW) and the Contractor’s Quality

Control Plan (QCP), and to ensure that the government pays only for the level of services received.

The QASP defines roles and responsibilities of all government staff in OPAP organization, identifies the performance objectives, and defines the methodologies used to monitor and evaluate the Contractor’s performance.

3. SCOPE

The QASP is a tool for use in Government administration of the contract and remains subject to revisions at any time by the USPTO throughout the contract performance period. USPTO will retain the right to change the surveillance methods, metrics, and Quality Assurance (QA) procedures described in the QASP, or to increase or decrease the degree of surveillance efforts at any time necessary to assure contract compliance. The contractor, and not the Government, is responsible for management and Quality Control (QC) actions to meet the terms of the contract.

4. SUMMARY OF WORK TO BE PERFORMED

The Office of Patent Application Processing (OPAP) government staff will be performing daily quality performance surveillance under the following

Tasks.

File Maintenance

(Reference Refiling, Artifacts Retrieved & Re-filed)

0002 Copying

0002AA Other

0002AB Office Action Images

0002AC PCT

FORM # USPTO-OP-014.v1.0 (06/2015) 4

Document Receipt &

Mailing

0003AA Incoming Mail

0003AB Outgoing Mail

0004 Print Working Folders

Incoming Mail Processing

(Open/Sort)

0005AA US Received

0005AB PCT Received

0005AC Faxes Received

Customer Service

Window

0006AA Processing Clerk 2 Day

0006AB

Processing Clerk 2 Night

(2 people)

0007 Fee Collection

0007AA New Applications

0007AB US Applications

0007AC PCT Applications

Fee Collection Other

Applications

0008AA US

0008AB PCT

0008AC Refunds

0009 Formality Review

0009AA Utility

0009AB Design

0009AC

Direct Hague Industrial

Design

0009AD

In-Direct Industrial

Design

0009AE Provisional

FORM # USPTO-OP-014.v1.0 (06/2015) 5

Formality Review

(Follow-on Papers)

0010AA Abandonments Processing

0010AB Missing Parts Responses

0010AC Change of Address

0010AD

Corrected Filing Receipt

Request

0010AE

Revocation/ Power of

Attorney Request

0010AF PDX Request

0010AG

SB69 /ADS Permission to

Access EPO Search

Results

0011 PCT National Stage

0011AA

PCT National Stage Bib

Data

0011AB

PCT National Stage

Mailing

0012 PCT International

0012AA

PCT International Bib

Data

0012AB PCT Printing & Mailing

Patent Application

Processing

0013AA

Secretary 3

(Commissioner's Office, 1 person)

0013AB

Secretary 2 (PCT Legal 2 people)

0013AC

Processing Clerk 2 (Patent

Academy, 2 people)

0013AD

Receptionist (Petitions 1 person)

FORM # USPTO-OP-014.v1.0 (06/2015) 6

0013AE

Processing Clerk 2

(Petitions, 3 people)

0013AF

Processing Clerk 2 (ODM, 3 people)

0013AG

Lead Processing Clerk

(ODM, 1 person)

Courier Support(Vehicle, Maintenance, Driver)

Project Management Staff

(Onsite)

5. ROLES AND RESPONSIBILITIES

5.1 Contracting Officer (CO)

The Contracting Officer (CO) is responsible for monitoring contract compliance, contract administration, cost control, and for resolving any difference between the observation documented by the Contracting Officer Representative (COR) and the Contractor. The CO will designate two full-time COR as the government authority for contract performance management. The number of additional representative serving as Task Order

Manager (TOM) or technical subject matter experts depends on the complexity of the services measured, as well as the Contractor’s performance.

5.2 Contracting Officer’s Representative (COR)

The Contracting Officer Representative (COR) is designated in writing by the CO to act as his or her authorized representative to assist in administering a contract. The COR is responsible for technical administration to ensure proper government surveillance of the Contractor’s performance. The COR is part of the Patents Management Team.

In implementing the QASP, the COR responsibilities include managing Quality Assurance (QA) activities to ensure compliance and completeness by:

• Overseeing the quality aspects of day to day operation of the PTScontract

• Serving as the technical liaison with the Contractor Quality staff

• Responsible for the final inspection and acceptance of all reports

• Other such other responsibilities as may be specified in the contract.

FORM # USPTO-OP-014.v1.0 (06/2015) 7

5.3 Contractor Personnel

Project Manager

• Oversee the performance and quality of work being performed.

• Respond to discrepancy on bi-weekly and monthly reports presented by the OPAP government staff.

• Meet with the USPTO COR on a monthly basis to review contract performance

Operations Manager

• Maintain and daily Operations of the PTS contract

• Respond to discrepancy reports or any actions needed from Director, OPAP

• Meet with the USPTO COR on a monthly basis to review contract performance

6. METHOLOGIES TO MONITOR PERFORMANCE

6.1 Surveillance Techniques

The OPAP government staff will perform quality surveillance using many of the various methods applicable which is included in the QASP as

Attachment A. OPAP government staff will use Random and Periodic surveillance methods and techniques which are appropriate for PTS contract and is incorporated into the SOW to monitor contract performance. Government will also rely on customer feedback from internal and external customers for quality assurance surveillance. The Contractor shall correct any problems or deficiencies upon notification from the Government based on specified sections of the SOW as addressed in Attachment 5 (Performance Metrics).

Random Surveillance

Evaluates random surveillance of contractor work as needed. The random sampling may be compiled into an excel spreadsheet of application numbers to determine its completeness and compliance with the patent filing requirement. This method is done to determine whether the contractor’s performance of a particular task for a given period of time meets the performance requirement based on randomly selected samples of the task. This method is the most appropriate for frequently recurring tasks

Periodic Surveillance

OPAP government staff typically performs the periodic inspections as needed and report any discrepancies as stated in SOW. It may be appropriate for tasks that occur infrequently and where 100% inspection is neither required nor practical. . The Government will also rely on customer feedback

FORM # USPTO-OP-014.v1.0 (06/2015) 8 from internal and external customers for quality assurance. The Contractor shall correct any problems or deficiencies upon notification from the Government based on specified sections of the SOW as addressed in Attachment 5 (Performance Metrics).

Acceptable Quality Levels

The acceptable quality levels (AQL) included in Attachment A, Performance Requirement Summary Table, for Contractor performance are instructed to allow the Contractor to manage how the work is being performed.

Customer Feedback

The Contractor is expected to establish and maintain professional communication between its employee and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

Performance management drives the Contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problem, but the customer always has the option to communicate complaints to the COR as opposed to the Customer.

Customer complaints to be considered valid must be set forth clearly and in writing the detailed nature of the complaint to be forwarded to the COR.

The COR will accept the customer complaint and investigate using the Quality Assurance Monitoring Form identified in Attachment A or another format mutually agreed to with the CO.

Customer feedback may also be obtained from random customer complaints.

Walkthroughs

Walkthroughs method evaluates correctness, timeliness, reliability, and productivity. Structured walkthroughs are used for orientation, examining actual output of contractor work, identifying defects or errors, and improving products at any stage in the process. OPAP’s government staff and

COR will periodically perform walkthroughs in areas of work being performed by Contractors.

6.2 Resolving Performance Issues

OPAP government staff, COR and the Contractor may discover performance issues or deficiencies through the surveillance process. The COR will provide an email or a list of any unresolved deficiencies to the Contractor to afford them the opportunity to investigate the deficiencies to determine if in fact the errors are the contractors. If evidence is shown where the contractor’s quality program has already documented the deficiency and timely corrective action is resolving the errors and/or problem, the COR will follow up with the contractor’s Key Personnel to ensure the deficiencies are remedied in a timely manner, in accordance with the SOW.

FORM # USPTO-OP-014.v1.0 (06/2015) 9

6.2.1 Reviews and Resolution

The COR will continue to review performance surveillance of contractor performance on efficiency and effectiveness of any deficiencies being presented. The COR and the Contractor will meet monthly to discuss contractor performance. Should the need arise for an emergency meeting because of unexpected performance issues, these meetings can be called by either COR or OPAP Director.

The COR will notify the Contracting Officer, in writing, of unacceptable quality levels. The Contracting Officer will promptly provide written notification of discrepancies on a monthly basis, or immediately if the situation significantly affects the well-being of the contract. The Contractor will be given the opportunity to respond in writing to each discrepancy.

The Contractor shall provide a written response to the cause and the corrective action to prevent recurrence within 10 working days after receipt of

Contracting Officer’s notification. The Contractor should cite specific quality assurance program procedures or new procedures instituted to prevent recurrence.

6.3 Documenting Surveillance

OPAP government staff will provide documentation of performance surveillance by electronic reports, paper submission, customer feedback, and contract data requirements.

All documentation resulting from surveillance must be made part of the contract file. The COR must keep the on-going documentation files of performed surveillance in the COR contract file. Monitoring Reports are also kept by OPAP Program Office for documentation purposes.

7. QUALITY ASSURANCE DOCUMENTATION

7.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the

SOW and in this QUASP. Performance standards are assessed using the performance monitoring techniques identified Attachment A. The COR or

OPAP Management will provide feedback to the Contractor’s Key Personnel of any performance issues for remediation.

7.2 Monitoring Forms

The government’s QA surveillance will be accomplished periodically by the OPAP government personnel and be reported using the monitoring forms in Attachment B or other format mutually agreed to with the CO. The forms, when completed, will document the government’s assessment of the Contractor’s performance stated in the SOW to ensure that the required results are being achieved. These forms, in additional to other documented information are used by the COR to asses performance during the life of PTS contract and inform the CO of any performance issues.

7.2.1 The OPAP Managers will retain a copy of all completed QA surveillance forms.

7.3 Reporting

FORM # USPTO-OP-014.v1.0 (06/2015) 10

OPAP Management will retain results of The COR will provide monthly reports which summarize the results of the quality assurance surveillance of the Contractor’s performance. The monthly reports will include compilation of the Government Quality inspections results from the completed quality assurance monitoring forms (Attachment B) or other format if mutually agreed to with CO, will become part of the Quality Assurance documentation.

8. PERFORMANCE REQUIREMENTS SUMMARY

See Chart below (Attachment A: Performance Requirement Summary)

ATTACHMENT A: PERFORMANCE REQUIRMENT SUMMARY

CLIN Deliverable Accuracy Timeliness Turnaround Times Method of Surveillance

0001 File Maintenance 99% 99% 4-business hours within the receipt; Pickup artifact files from Tech Center (TC); refile artifact 4-hours of receipt from

TC.

Periodically inspect artifact files room located in

Remsen Concourse 0A41, Remsen 2C79 & Remsen

5C84 to ensure appropriately refiled or rely on internal and external customer feedback.

0002AA Other Copying 98% 98% Complete all copying and deliver of the folder to the designated location within 1 business day after pickup

Periodically inspect processed work and internal requests for copying in

Randolph Building, Room

1B11 or rely on internal and external customer feedback.

Due to automation, the majority of the Copying will be done for PCT appropriately refiled or rely on internal and external customer feedback.

FORM # USPTO-OP-014.v1.0 (06/2015) 11

0002AB Office Action

Copying

98% 98% Complete all copying and deliver of the folder to the designated location within 1 business day after pickup

Randomly inspect processed work and internal requests for copying in

Randolph Building, Room

1B11. Check contractor designated location files ready for mailing and scanning to ensure accuracy and timeliness appropriately refiled or rely on internal and external customer feedback. Due to automation, the majority of the Copying will be done for PCT.

0002AC Copying (PCT) 98% 98% Complete all copying and deliver of the folder to the designated location within 1 business day after pickup

Randomly inspect processed work in

Randolph Building, Room

1B11 of copied office action in PLRC. Check contractor designated location files ready for mailing and scanning to ensure accuracy and timeliness appropriately refiled or rely on internal and external customer feedback.

FORM # USPTO-OP-014.v1.0 (06/2015) 12

0003AA DRAM Incoming

Mail

99% 99% Complete all mail delivery with 2 business hours from receipt

Check DRAM mailroom area for incoming mail.

Check scheduled deliveries of incoming mail from Mail

Center to ensure mail delivery is completed within 2 hours appropriately refiled or rely

0003AB Outgoing Mail 99% 99% Notice of Allowance within (1) business day of retrieval from pickup location. Accelerated

Mail Notice within (1) business hour. All other mailing immediately following retrieval from printers; seal envelopes available for mailroom pickup no later than

(2) business days after retrieval of work.

Randomly inspect outgoing mail before sent to drop-off location for Mail Center pick-up or rely on internal

0004 Print Working

Folder

100% 98% 1 business day Randomly inspect Print

Working Folder area for accuracy and timeliness of

PWFs appropriately refiled or rely on internal and external customer feedback.

FORM # USPTO-OP-014.v1.0 (06/2015) 13

0005,0005AA,

0005AB,

00005AC

Incoming Mail

Processing (US, PCT, Faxes)

99% 99% Perform mail processing tasks several times each workday.

Central Fax - Keep log of all documents that do not print within (1) business day of receipt. Open and Sort -

Complete opening, sorting and accurate routing of all non-fee mail within (2) business days of receipt.

Randomly sample fee/non-fee mail to ensure they are opening, sorting, and accurate routing all non-fee mail within (2) business days. Randomly sample

Central faxes to ensure contractor is following proper instructions, apply correct fee and fee code to the correct application number within (1) business day appropriately refiled or rely on internal and external customer feedback.

0006 Customer Service

Window

99% 99% Perform mail processing tasks during Window hours of 8:30 am to 12 midnight. Open and

Sort - Complete opening, sorting and accurate routing of all non-fee mail within (2) workdays of receipt.

Perform surveillance of

Customer Window located in Randolph Building, IB11. Randomly sample fee/non-fee mail to ensure they are opening, sorting, and accurate routing all non-fee mail within (2) business days or rely on internal and external

FORM # USPTO-OP-014.v1.0 (06/2015) 14

0007, 0007AA,

0007AB

Fee Collection-

New Applications

(US Applications &

PCT Applications)

100%, for posting to correct application number;

98% for accuracy for other fee collection tasks.

98% Process all fees within 2 business days of receipt

Sample RAM transactions to ensure they are properly entering correct fees, fee codes with the correct applications within (2) accounting days of their receipt. Check mail for correct posting of fees.

Also check application numbers on RAM reports for correct RAM transactions or rely on internal and external customer feedback.

0008, 0008AA,

0008AB, 0008AC

Fee Collection –

Other Applications

(US Applications, PCT Applications, Refund Processing)

100%, for posting to correct application number;

98% for accuracy for other fee collection tasks.

98% Process all fees within 2 business days of receipt

Sample RAM transactions to ensure they are properly entering correct fees, fee codes with the correct applications within (2) business days of their receipt. Check mail for correct posting of fees.

Also check application numbers on RAM reports for correct RAM transactions appropriately refiled or rely on internal

FORM # USPTO-OP-014.v1.0 (06/2015) 15

0009, 0009AA,

0009AB,

0009AC,

0009AD,

0009AE,

Formality Review –

New Applications

(Utility, Design, Direct Hague

Industrial Design, Indirect Hague

Industrial Design, Provisional)

99% 99% Complete formality review within 10 business days of receipt

Sample contractor work after mailing and report error rate and errors found or rely on appropriately refiled or rely on internal and external customer feedback. From the results of the Government

Sampling, contractor has

(5) business days to contest errors on new application formality review processing

0010, 0010AA,

0010AB,

0010AC,

0010AD,

0010AE,

0010AF, 0010AG

Formality Review –

Follow-

Ons(Abandonment

Processing, Missing

Parts Responses, Change of Address, Corrected Filing

Receipt Request, PDX Request)

99% 99% Complete all processing with

10 business days of receipt

Sample contractor work after mailing and report error rate and errors found or rely on appropriately refiled or rely on internal and external customer feedback. From the results of the Government

Sampling, contractor has

(5) business days to contest errors on new application formality review processing.

FORM # USPTO-OP-014.v1.0 (06/2015) 16

0011, 0011AA,

0011AB

PCT National Stage

– (PCT National

Stage Bib Data, PCT National Stage

Mailing

98% 98% Complete PCT NST bib data entry and mailing within 2 business days of receipt

Sample contractor’s work after mailing to ensure bib data entry is correct and validate error rate. Check official notification mailing for correctness and to ensure timely drop-off at designated outgoing mail pick-up. The Government will also rely on internal and external customer feedback.

0012, 0012AA,

0012AB

PCT International –

(PCT International

Bib Data, Printing

& Mailing)

99% 99% Complete PCT International bib data entry and mailing within 2 business days of receipt

Sample contractor’s work after mailing to ensure bib data entry is correct and validate error rate. Check official notification mailing for correctness and to ensure timely drop-off at designated outgoing mail pick-up. The Government will also rely on internal

FORM # USPTO-OP-014.v1.0 (06/2015) 17

0013, 0013AA,

0013AB,0013AC,

0013AD,

0013AE,

0013AF,

General Clerical

Support

N/A N/A Perform various clerical duties as assigned by various Program

Offices.

Verify with the Program

Office Point of Contact

(POC) that staff hours were worked and clerical support received was satisfactory.

Perform random surveillance of contract staff that support CLIN

0013 and sub CLINs or rely

FORM # USPTO-OP-014.v1.0 (06/2015) 18

FORM # USPTO-OP-014.v1.0 (06/2015) 19

ATTACHMENT B:

File Room Inspection Form

Time Inspection Started: ___________ Time Inspection Completed: __________

Location: ___________________ DATE: __________________________

Inspector Name: __________________

Application Number:

Quality: Timeliness

File incorrectly placed on shelf Files not refiled within 4 hours

Multiple artifacts for same application number not located together

Other_____________________________________________________________________________

Multiple artifacts for same application number

Multiple artifacts for same application number

Multiple artifacts for same application number

FORM # USPTO-OP-014.v1.0 (06/2015) 20

Total Inspected_______________

Total Errors__________________

PLRC Inspection

DATE: __________________________ Inspector Name: __________________

Time Inspection Started: Time Inspection Completed:

Application Number: ______________________

Incorrect international application number Mail not processed within 2 business days

Incorrect mail date Corrections made within 4 hours of notification

Incorrect attachments

Incorrect pages noted for search Report

Incorrect copies made

Other_______________________________________

Quality: Timeliness

Incorrect international application number Mail not processed within 2 business days

Incorrect mail date Corrections made within 4 hours of notification

Incorrect attachments

Incorrect pages noted for search Report

Incorrect copies made

FORM # USPTO-OP-014.v1.0 (06/2015) 21

Total Inspected___________

Total Errors______________

DOCUMENT RECEIPT AND MAILING

OUTGOING MAIL INSPECTION FORM

Date:

Time Inspection Started:

Inspector:

Time Inspection Completed:

Quality of Mailing for Office Action Code

FORM # USPTO-OP-014.v1.0 (06/2015) 22

Indicate which Quality Code was incorrectly done on application:

A

Copies of the cited references(s) not mailed to applicant

B

Pages of office action not in numerical order

C Attachment(s) not included

D

Mailing address is not visible through the envelope's address window

Total Inspected

Total Errors

FORM # USPTO-OP-014.v1.0 (06/2015) 23

Print Working Folder

INSPECTION FORM

Date: Time Inspection Started: ___________

Inspector: Time Inspection Completed: __________

Quality of Print Working Folders Code

Indicate which Quality Code was incorrectly done on PWF request:

FORM # USPTO-OP-014.v1.0 (06/2015) 24

A Missing Request: (Specification, Abstract, claims and/or drawings)

B Missing serial number and bid data label on folder

Total Inspected

Total Errors

Central Fax Inspection Form

DATE: _______________________

Inspector Name: __________________

Application Number:____________________

Incorrect mail date Fees processed within 2 accounting days

Notify Gov. Rep. of New Applications Received Non fee documents routed within 2 workdays

Incorrect application Number applied to fees Corrections made within 4 hours of notification

Other_________________________________

Application Number:____________________

FORM # USPTO-OP-014.v1.0 (06/2015) 25

Total Errors______________

Fee Collection Inspection

DATE: __________________________ Inspector Name: __________________

Time Inspection Started: Time Inspection Completed:

Application Number: New Follow On (optional)

Incorrect application number Fees not entered in RAM within 2 accounting days

Incorrect mail date Corrections made within 4 hours of notification

Incorrect fee code

FORM # USPTO-OP-014.v1.0 (06/2015) 26

Incorrect authorization to charge

Incorrect fee code

Incorrect fee code

Total Errors______________

FORM # USPTO-OP-014.v1.0 (06/2015) 27

PCT 371 NATIONAL STAGE DATA ENTRY AND MAILING

DATE: _________________________

Inspector: _______________________

QUALITY OF MAILING (Check all of the following that are applicable)

_____ PCT 371 Bib Data entered correctly

_____ Mail complete official PCT 371 Notice

_____ Correct Filing Incorrect

_____ Mailing address is not visible through the envelope’s address window

_____ Mailing date is not the current (after 1 pm next day mail date)

_____ Envelopes are not properly sealed

TIMELINESS OF PCT 371 NATIONAL STAGE BIB DATA & MAILING (Check all of the following that are applicable)

_____ Process PCT 371 National Stage Bib Data entry and mailing within (5) days of contractor’s receipt

_____ Contractor fails to correct problem and deficiencies within (4) work hours of receiving notification.

Comments

FORM # USPTO-OP-014.v1.0 (06/2015) 28

PCT INTERNATIONAL BIB DATA ENTRY

DATE: _________________________

Inspector: _______________________

QUALITY OF MAILING (Check all of the following that are applicable)

_____ PCT International Bib Data entered correctly

_____ Mail correct PCT International Notice

_____ Correct Filing Incorrect

_____ Mailing address is not visible through the envelope’s address window

_____ Mailing date is not the current (after 1 pm next day mail date)

_____ Envelopes are not properly sealed

TIMELINESS OF PCT INTERNATIONAL BIB DATA (Check all of the following that are applicable)

FORM # USPTO-OP-014.v1.0 (06/2015) 29

_____ Process PCT International Data entry within (2) days of contractor’s receipt

_____ Contractor fails to correct problem and deficiencies within (4) work hours of receiving notification.

Comments