Attachment 5 - QASP.docx

Quality Assurance Surveillance Plan

For: Department of Labor, Bureau of Labor Statistics Contract Number: < upon award, enter contract number> Contract Description: Job Openings and Labor Turnover Survey Data Collection Center Operations The contractor shall provide non-personal services for the CES program with survey data collection services at the government Data Collection Centers (DCCs) in Atlanta, GA; Dallas, TX; Kansas City, MO; and Fort Walton Beach, FL. Limited telework flexibility is available under conditions listed in section C.4.2.7. Further, the contractor shall provide Help Desk services in Suitland, MD. The contractor shall provide quality control, training, and project management to support contractor operations at the Data Collection Centers and Help Desk.

Contractor’s name: < upon award, enter contractor name > (hereafter referred to as the contractor).

Purpose This Quality Assurance Surveillance Plan (QASP) provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

· What will be monitored?

· How monitoring will take place?

· Who will conduct the monitoring?

· How monitoring efforts and results will be documented?

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. However, the Government shall coordinate changes with the contractor. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

The following FAR clauses may apply depending on contract type:

· 52.246-4 Inspection of Services – Fixed-Price,

· 52.246-6 Inspection of Services – Time-and-Material and Labor-Hour

Government Roles and Responsibilities The following personnel shall oversee and coordinate surveillance activities.

a. Contracting Officer (CO) - The CO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The CO shall also assure that the contractor receives impartial, fair, and equitable treatment under this contract. The CO is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

Assigned CO: Shayla Battle Organization or Agency: US Department of Labor/Office of the Special Procurement Executive Telephone: 202-693-7176 Email: Battle.Shayla.E@dol.gov

b. Contracting Officer’s Representative (COR) - The COR is responsible for technical administration of the contract and shall assure proper Government surveillance of the contractor’s performance. The COR shall keep a quality assurance file. At the conclusion of the contract or when requested by the CO, the COR shall provide documentation to the CO. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. The contractor shall refer any changes they deem may affect contract price, terms, or conditions to the CO for action.

Assigned COR: Robert Viégas Telephone: 202-691-5832 Email: Viegas.Robert@bls.gov

c. Other Key Government Personnel

Sally Anderson (Government Project Manager) Title: Branch Chief, CES Data Collection Branch Telephone: 202-691-5496 Email: Anderson.Sally@bls.gov

Tom Shaffer Title: Division Chief, Division of Cooperative Survey Programs Telephone: 202-691-6163 Email: Shaffer.Tom@bls.gov

Contractor Representatives The following employees of the contractor serve as the contractor’s Contract Manager and Task Manager for this contract.

a. Contract Manager - <upon award, enter name> Telephone: <enter number> Email: <enter address>

b. Task Manager - <upon award, enter name> Telephone: <enter number> Email: <enter address>

c. Other Contractor Personnel - <upon award, enter name or delete these lines if not applicable> Title: <enter title> Telephone: <enter number> Email: <enter address>

Performance Standards Performance standards define desired services. The Government performs surveillance to determine if the contractor exceeds, meets or does not meet these standards.

The Performance Requirements Summary Matrix, paragraph C.6 in the Performance Work Statement includes performance standards. The Government shall use these standards to determine contractor performance and shall compare contractor performance to the Acceptable Quality Level (AQL). Technical Exhibit 1 - Performance Requirements Summary - JOLTS

Address Refinement and Enrollment (5.1)

Multi-Modal Collection (5.2)

Item response rate – Layoffs/Discharges (1st Closing)

Item response rate – Layoffs/Discharges (2nd Closing)

Item response rate – Other Separations

Item response rate – Other Separations

Item response rate – Total Separations

Item response rate – Total Separations

Maximum rate of units reporting only total employment

Maximum rate of units reporting only total employment

Maximum refusal rate (2nd Closing)

Edit Reconciliation (5.3)

Training (5.4)

The government expects the contractor to be fully staffed to achieve the AQLs. For each managerial position that remains unfilled for 30 calendar days, a 5% penalty will be assessed against the firm-fixed-price component of the contract in the first invoice received following the 30 days and any subsequent invoice while that position has not been filled.

Incentives The Government shall use favorable contractor performance evaluations as incentives. The Government shall report unfavorable contractor performance as disincentives. If repetitive or consistent the Government shall take further corrective action or begin termination procedures. Incentives and disincentives shall be based on exceeding, meeting, or not meeting acceptable quality levels.

Methods of QA Surveillance Various methods exist to monitor performance. The COR and BLS staff shall use the surveillance methods listed below in the administration of this QASP. Regardless of the surveillance method, the COR shall always contact the contractor's task manager or on-site representative when a defect is identified and inform the manager of the specifics of the problem. The COR, with assistance from the CO, shall be responsible for monitoring the contractor’s performance in meeting a specific performance standard/AQL.

DIRECT OBSERVATION / RANDOM SAMPLING / PERIODIC INSPECTION

DCC management will periodically listen in on calls to ensure that interviewers are following the procedures correctly to ensure the integrity of collected data. DCC management will provide case review and re-interview checklists to Contractor management (See Appendix A)

MANAGEMENT INFORMATION SYSTEMS (MIS) / PERFORMANCE REPORTING

DCC management and Program Office management will rely on reports generated from CATI and other sources to identify potential performance issues. If any issues are discovered, they will be communicated to the Contractor management in an ad hoc manner.

PROGRESS OR STATUS MEETINGS / ANALYSIS OF CONTRACTOR'S PROGRESS REPORTS

BLS staff and Contractor management shall meet monthly to review performance reports and address any performance issues. If there are significant performance issues, more status meetings may be scheduled.

Ratings Metrics and methods are designed to determine if performance exceeds, meets, or does not meet a given standard and acceptable quality level. A rating scale shall be used to determine a positive, neutral, or negative outcome. The following ratings shall be used:

Documenting Performance

a. ACCEPTABLE PERFORMANCE The Government shall document positive performance in the monthly performance report (See Appendix B). A report template is attached. Any report may become a part of the supporting documentation for fixed fee payments, award fee payments, or other actions.

b. UNACCEPTABLE PERFORMANCE.

When unacceptable performance occurs, the COR shall inform the contractor in the monthly performance report. This will normally be in writing unless circumstances necessitate verbal communication. In any case the COR shall document the discussion and place it in the COR file.

When the COR determines formal written communication is required, the COR shall prepare a Contract Discrepancy Report (CDR), and present it to the contractor's task manager or on-site representative. A CDR template is attached to this QASP (See Appendix C).

The contractor shall acknowledge receipt of the CDR in writing. The CDR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CDR will also state how long after receipt the contractor has to present this corrective action plan to the COR. The Government shall review the contractor's corrective action plan to determine acceptability.

Any CDRs may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, or other actions deemed necessary by the CO.

Frequency of Measurement

a. FREQUENCY OF MEASUREMENT.

During contract/order performance, the COR shall review monthly performance reports as specified in the AQL column of the Performance Standards Summary Matrix.

b. FREQUENCY OF PERFORMANCE ASSESSMENT MEETINGS.

The COR shall meet with the contractor biweekly to address performance and operational issues. A Performance Report will be provided monthly.

Prepared by: Robert Viegas

Signature – Contracting Officer’s Representative

Appendix A

Interviewer’s Length of Service: Less Than One Year: ___ 1-3 Years: ___ More Than Three Years: ___ Note Review

Comments

Critical Element Do the Permanent Notes contain required information?

Unemployment Insurance Number Verification

Comments

Are we collecting the correct UI?

Contact Information

Comments

Is enough essential respondent information provided to ensure we can contact the correct respondent by their requested method?

Reporting Location

Comments

Are the company name, address, and location fields for the worksite information completed correctly for each unit?

Method of Collection

Comments

Is the Collection Method appropriate and correctly documented?

Scheduling

Comments

CES and QCEW Comparison

Is the TopCATI AE count comparable to the AE counts from the EDB?

Data Edits

Comments

Are edit failures in the microdata for the current and/or previous month explained adequately?

Data Item Collection

Comments

If not all data items are collected, is there an explanation?

Aggregation

JOLTS Quality Assurance Re-Interview Checklist Date

Report Number Reviewer Interviewer

Interviewer’s Length of Service Less Than One Year ____ 1 to 3 Years ____ More Than 3 Years ____ Introduction

Comments

Good afternoon Mr./Ms.______.

My name is (BLS Reviewer); I’m with the U.S. Department of Labor, Bureau of Labor Statistics. As a part of our on-going Quality Assurance for the Current Employment Statistics Program, I would like to verify some of the information that you provided.

Data Integrity

Critical Element Did you receive a call from (interviewer) regarding your data this month?

Or Did you receive your fax reminder this month from (interviewer)?

Is your name and job title:_____________ Is your mailing address:_______________ Your fax number?_____________ Your email address?________________

Unemployment Insurance Number Verification

Critical Element Our records indicate that your Unemployment Insurance Number is __.

Is that correct?

Reporting Location

Critical Element If Statewide data are collected:

The data you provided only includes employees under this UI for the state of __. Is this correct?

Or If Locations are collected:

The data you provided only includes employees under this UI for the individual locations of _,_. Is this correct?

Pay Period Verification

Are the data you provided for the pay period that includes the 12th of the month?

Data Accuracy

You provided AE=__ AEPR=__ AECM= ___AEHR=__ AEOT=____ Is this correct?

Data Items Collection

Critical Element If there are missing data items:

Were you asked to provide (missing data item(s)

Method of Collection

Can I call you back in the next two workdays to confirm your Unemployment Insurance number?

After Call Review:

b. Special arrangements of collection

c. Missing data item(s) explanation

d. Date Web was offered or declined

e. Special notes about the case

f. First month edit AEQ failure documented

g. LP Code and Pay Period

Additional Comments

Appendix B Performance Report

1. CONTRACT NUMBER: <insert number>

2. Prepared by: Robert Viegas

3. Period of Performance:

4. Observation:

Technical Exhibit 1 - Performance Requirements Summary - JOLTS

Rating

Address Refinement and Enrollment (5.1)

Multi-Modal Collection (5.2)

Item response rate – Layoffs/Discharges

93%

Item response rate – Layoffs/Discharges

93%

Item response rate – Other Separations

93%

Item response rate – Other Separations

93%

Item response rate – Total Separations

96%

Item response rate – Total Separations

96%

Maximum rate of units reporting only total employment

0.5%

Maximum rate of units reporting only total employment

0.5%

Maximum refusal rate (2nd Closing)

25%

Edit Reconciliation (5.3)

Training (5.4)

Notes:

Prepared by: Robert Viegas

Appendix C

QUALITY ASSURANCE SURVEILLANCE PLAN

Current Employment Statistics Data Collection Center Operations image1.png