CES_PWS_DRAFT.pdf

CURRENT EMPLOYMENT STATISTICS

DATA COLLECTION CENTER

PERFORMANCE WORK STATEMENT

C.1 BACKGROUND

C.1.1 CURRENT EMPLOYMENT STATISTICS

The Bureau of Labor Statistics (BLS) collects data each month on employment, hours, and earnings from a sample of nonfarm establishments through the Current Employment Statistics (CES) program. The CES survey includes about 147,000 businesses and government agencies, which cover approximately 634,000 individual worksites drawn from a sampling frame of Unemployment Insurance (UI) tax accounts covering roughly 9.7 million establishments. The active CES sample includes approximately one-third of all nonfarm payroll employees in the 50 states and the District of Columbia. From these data, a large number of employment, hours, and earnings series in considerable industry and geographic detail are prepared and published each month.

The CES survey collects the following data elements from business establishments:

• All Employees (AE)

• Women Employees (WE)

• Production and Nonsupervisory Employees (PE)

• AE Payroll

• AE Hours

• AE Commissions (paid at least once per month)

• AE Overtime Hours (manufacturing only)

• PE Payroll

• PE Hours

• PE Commissions (paid at least once per month)

• PE Overtime Hours (manufacturing only)

For more information on the data collected by CES, please see Appendix A - CES Forms and Data Elements.

The primary statistics derived from the CES survey are monthly estimates of employment, hours, and earnings for the nation, states, and major metropolitan areas. Preliminary national estimates for a given reference month are typically released on the third Friday after the conclusion of the reference period in conjunction with data derived from a separate survey of households, the Current Population Survey (CPS). The reference period for the CES survey is the pay period which includes the 12th of the month. For more information on the CES production schedule, please see Appendix B - CES Schedule.

For more information on CES, please visit https://www.bls.gov/ces/.

All glossary words are bolded the first time they appear in this document. Please see Appendix F – Glossary.

https://www.bls.gov/ces/

C.1.2 CES DATA COLLECTION

C.1.2.1 CES DATA COLLECTION BACKGROUND

Each month, the Bureau of Labor Statistics (BLS) collects data on employment, payroll, and paid hours from a sample of establishments. CES has a comprehensive program of new sample unit enrollment in four CES Data Collection Centers (DCCs). The DCCs perform initial enrollment of each establishment via telephone, collect the data for several months via Computer Assisted Telephone Interviewing (CATI), and where possible transfer respondents to web collection. In addition, the DCCs conduct an ongoing program of refusal conversion. Nearly all of the initial enrollment, and about 25 percent of ongoing monthly collection, occurs by CATI at the DCCs.

The BLS web collection tool allows respondents to complete their monthly data entry online on a secure website. The data are subject to a series of edit & screening checks before being transmitted to CES. About 17 percent of CES data are collected by web.

The BLS Electronic Data Interchange (EDI) Center enrolls and collects data from large firms, typically those with fifty or more worksites. Firms that report to the EDI Center provide an electronic file to CES each month in a prescribed file format. This file includes data for all of the firm's worksites. The file is received, processed, and edited by the EDI Center. About 47 percent of CES data are collected by EDI.

Touchtone Data Entry (TDE) was once a major collection tool for the CES but is no longer offered to new respondents. Under the TDE system, the respondent uses a touchtone telephone to call a toll-free number and activate an interview session. The respondent enters numeric responses by pressing the touchtone telephone buttons. Each answer is read back for respondent verification. About two percent of CES data are collected by TDE.

The CES DCCs collect about three percent of CES data by fax.

For the establishments that do not use the above methods, data are collected using email, mail, or other special arrangements.

For more information on the evolution of CES data collection, please visit: https://www.bls.gov/opub/mlr/2016/article/one-hundred-years-of-current-employment-statistics-data-collection.htm

Figure 1 shows the percentage of the reports collected by the various CES collection methods. Although EDI is responsible for almost half of the reports collected (47%), these reports come from processing files from fewer than 100 very large firms. The remaining 53% of reports collected by CATI, web, fax, TDE, and other methods, come from data received from approximately 147,000 businesses and government agencies.

https://www.bls.gov/opub/mlr/2016/article/one-hundred-years-of-current-employment-statistics-data-collection.htm https://www.bls.gov/opub/mlr/2016/article/one-hundred-years-of-current-employment-statistics-data-collection.htm https://www.bls.gov/web/empsit/cestn.htm#Fg1

Figure 1. Current Employment Statistics survey data collection methods by percent

C.1.2.2 CES SURVEY UNITS

The CES survey consists of certainty selected and probability selected units. All government units, units with large levels of employment, and units processed by the EDI Center are certainty selections and remain in the CES survey indefinitely. All remaining units are selected by probability and remain in the CES survey for at least two years, but typically no more than four consecutive years. All new probability selections and some certainty selections are enrolled by the DCCs.

For more information on the CES sample, please visit: https://www.bls.gov/web/empsit/cestn.htm

C.2 SCOPE AND DESCRIPTION OF SERVICES

C.2.1 OBJECTIVE

The contractor shall provide non-personal services for the CES program with survey data collection services at the government Data Collection Centers in Atlanta, GA; Dallas, TX;

Kansas City, MO; and Fort Walton Beach, FL. Further, the contractor shall provide Help Desk services in Washington, DC. The contractor shall provide quality control, training and project management to support contractor operations at the Data Collection Centers and Help Desk.

The government retains responsibility for CES survey methodology, procedures, and tools including:

o Sampling methodology and selection.

o Assignment of cases for enrollment and collection to Data Collection Centers.

o Data collection instruments, including all hardware and software.

o Policies and procedures for enrollment, collection, and refusal conversion.

https://www.bls.gov/web/empsit/cestn.htm https://www.bls.gov/web/empsit/cestn.htm

C.2.2 PERIOD OF PERFORMANCE

One TBD base period and four 12-month option periods.

C.2.3 PLACE OF PERFORMANCE

C.2.3.1 CES DATA COLLECTION CENTERS (DCCs) The DCCs are located in Federal government facilities in Atlanta, GA and Dallas, TX and in government leased space in Kansas City, MO and Fort Walton Beach, FL. All four DCCs are responsible for cases nationwide; assignments are not made on a regional basis.

The DCCs are responsible for enrolling new sample units, known as cases, and the ongoing monthly collection of these cases. This work is primarily done using the government CATI and telephone system. During the initial five months of collection by CATI, the DCCs work to transition respondents to the government web collection tool.

Respondents that are unable or unwilling to provide data by web will be assigned by the DCC to permanent CATI collection. The government will send web respondents, who fail to report reliably, back to the DCC for CATI collection.

*Hours of operation are subject to change.

The total number of workstations, which includes cubicles and private offices, available to contract staff at each DCC is located in the chart above and represents the maximum number of staff that can work at each DCC. Interviewer workstations are cubicles and supervisory workstations are typically private offices, dependent on availability.

DCC staffing requirements peak during the two weeks leading up to CES 1st Closing, which is the final day of collection for each reference month. The workload during these two weeks is about 30% higher than the rest of the month. Currently, a portion of the interviewer staff works only the last two weeks leading up to CES 1st Closing. For more information on the CES production schedule, please see Appendix B - CES Schedule.

DCC Address Hours of Operation*

Workstations

Atlanta 61 Forsyth St., S.W.

Atlanta, GA 30303

6 am – 7 pm (Eastern)

Dallas 525 Griffin Street Dallas, TX 75202

6 am – 7 pm (Central)

Kansas City

Two Pershing Square Building 2300 Main Street Kansas City, MO 64108

6 am – 7 pm

Fort Walton Beach 73 Eglin Parkway Fort Walton Beach, FL 32548

6:30 am – 5:30 pm

C.2.3.2 CES HELP DESK

The Help Desk is located in the BLS National Office in Washington, DC. The Help Desk is primarily responsible for supporting web and TDE respondents. The Help Desk also manually enters data received at the National Office and resolves data quality issues.

Help Desk Address Hours of Operation Workstations

Washington D.C.

Postal Square Building 2 Massachusetts Ave., NE Washington D.C., 20212

7:30 am – 8 pm (Eastern)

The total number of workstations available to contract staff is located in the chart above and represents the maximum number of staff that can work at the Help Desk. All workstations are located in cubicles.

C.2.4 GENERAL INFORMATION

C.2.4.1 KEY PERSONNEL

The contractor shall identify the key personnel that will be responsible for managing the contract. The Project Manager (PM) is considered key personnel for the contract. See section C.4.1 for PM requirements. The PM shall be the single point of contact for the management of this contract. The PM is responsible to the Contracting Officer’s Representative (COR) and the Contracting Officer (CO) for compliance with all the requirements of this contract. Historically, this has been a full-time position.

The PM shall supervise and maintain a comprehensive management organization over staff working on this contract. A staff management plan delineating the lines of authority shall be maintained and provided to the COR on a quarterly basis. The PM must meet with the COR and other program officials no less than once per month to discuss issues related to the administration of the contract and actual performance on respective tasks. The contractor shall be available to attend in-person at the BLS National Office in Washington, DC for these meetings at least once quarterly; other meetings can be held by conference call or other virtual method. The contractor shall not remove or divert key personnel unless written notice is provided at least 30 business days prior to the removal. Furthermore, a replacement for the key personnel must be approved by the CO at least 10 business days prior to the removal to ensure business continuity and transfer of institutional knowledge.

In the event that any of the key personnel named in the contractor’s proposal, as accepted by the government, are unable to perform because of death, illness, resignation from the contractor’s employ, or other reasons, the contractor shall submit a written notice within 48 hours to the CO. A detailed explanation of the circumstances necessitating the proposed substitutions, complete resumes for the proposed substitutes, and any other information that the CO deems pertinent to approve the substitution shall be submitted to the CO as soon as possible, but no later than 30 days after this initial written notification. No substitution is to be made without prior written approval by the CO

C.2.4.2 HOLIDAYS

The contractor shall not perform work on government holidays or non-work days (including weekends) without prior written approval of the CO. Approved work performed on holidays or non-work days shall be billable at regular rates.

The following is a list of government holidays:

• New Year’s Day

• Martin Luther King Day

• President’s Day

• Memorial Day

• Independence Day

• Labor Day

• Columbus Day

• Veterans Day

• Thanksgiving Day

• Christmas Day

• Inauguration Day (every 4 years) – DC only

C.2.4.3 UNSCHEDULED DISMISSALS

Unless requested by the Government Task Monitor (GTM) and approved by the CO, in the event that the government facility closes, on-site contractor personnel working under this contract shall be dismissed. In any situation regarding unscheduled dismissal and/or the closure of government offices, the contractor shall not be compensated for lost hours. In the event extraordinary conditions require continued contractor support when government offices are closed, at the specific request of the GTM, and with the approval of the CO, the contractor shall provide sufficient personnel to support. Such work shall be compensated at the normal contractual rate.

C.2.4.4 CONFIDENTIALITY AND SECURITY

The Contractor shall comply with the BLS Confidentiality and Security Requirements.

Please see Appendix G –BLS Confidentiality and Security Requirements.

The contractor shall safeguard all government equipment, information and property provided for contractor use. At the close of each work period, government facilities, equipment, and confidential materials shall be secured.

C.2.4.5 CONTRACTOR PERSONNEL CONDUCT

Contractor personnel’s conduct shall not reflect discredit upon the government. The contractor shall ensure that personnel present a professional appearance. The contractor’s employees shall observe and comply with all local policies and procedures posted at each site concerning fire, safety, environmental protection, sanitation, security, and possession of firearms or other lethal or illegal weapons or substances. The contractor is responsible for ensuring that any contractor employees providing services under this contract conduct themselves and perform services in a professional, safe, and responsible manner. The contractor shall ensure no contractor employees conduct political related activities or events in any government or government-leased facilities.

C.2.4.6 DATA RIGHTS

The government has unlimited rights to all documents and materials produced under this contract. All documents and materials, to include the source codes of any software, produced under this contract shall be government owned and are the property of the government with all rights and privileges of ownership/copyright belonging exclusively to the government. All materials supplied to the government will be the sole property of the government and may not be used for any other purpose. This right does not abrogate any other government rights.

C.2.4.7 WORKING REMOTELY

The contractor shall make an email request to the COR with a cc to the DCC Manager requesting permission prior to authorizing remote working (i.e. other than at the assigned BLS office) for contract staff. Interviewer staff are not permitted to work remotely. It is expected that these requests shall be seldom in nature and only covers staff who normally work in BLS provided space.

C.2.4.8 GOVERNMENT-FURNISHED MATERIALS

C.2.4.8.1 FURNISHED BY THE GOVERNMENT

The government will provide office space for the Data Collection Centers and the Help Desk that includes office furniture (desks, chairs, and office storage), information technology (computers, printers, monitors, keyboards, and internet connections), normal office supplies (pens, paper, staplers, etc.), fax machine, internet, and telephone access.

The government will not provide office space for a project management office.

The government will provide a laptop computer to the PM.

The government will retain title to all government-furnished property. All materials developed by the contractor under this contract shall become the property of the U.S. Department of Labor.

The government-furnished property and the materials developed by the contractor under this contract, shall only be used for the performance of this contract.

The government will maintain all facilities and equipment that shall be provided for use by the contractor. Additionally, the contractor shall ensure that its personnel do not misuse any government-furnished property.

The government will provide CATI and Help Desk processing software.

C.2.4.8.2 CONTRACTOR RESPONSIBILITY

If due to the default, negligence (whether of action or omission), or dishonesty of the contractor or its employees, or any combination of the prior activities, any government-owned or controlled property is lost or damaged as a result of the contractor’s performance of this contract, the contractor shall be responsible to the government for such loss or damage. The government at its option, may, in lieu of payment therefore, require the contractor to replace at its own expense, all property lost or damaged. The contractor shall not provide computer hardware or software without approval from the government.

C.2.4.9 SUPPORTING DOCUMENTS

The contractor shall abide by all applicable regulations, publications, manuals, and local policies and procedures. The contractor shall adhere to and use the versions/revisions in effect. The contractor shall comply with the policies and procedures set forth in all such documents.

C.3. TASKS

Task Sub-task Sub-task Number

Page number

ENROLLMENT ADDRESS REFINEMENT C.3.1.1

ENROLLMENT ENROLLMENT OF NEW SAMPLE C.3.1.2

ENROLLMENT REFUSAL CONVERSION C.3.1.3

ENROLLMENT ENROLLMENT PERFORMANCE

MEASURES

C.3.1.4

COLLECTION MULTI-MODAL COLLECTION C.3.2.1

COLLECTION COLLECTION PERFORMANCE MEASURES C.3.2.2

HELP DESK WEB AND TDE RESPONDENT SUPPORT C.3.3.1

HELP DESK WEB AND TDE EDIT RECONCILIATION

AND DATA QUALITY

C.3.3.2

HELP DESK DATA ENTRY C.3.3.3

HELP DESK DATA PROCESSING C.3.3.4

HELP DESK OTHER HELP DESK SUPPORT ACTIVITIES

C.3.3.5

HELP DESK HELP DESK PERFORMANCE MEASURES C.3.3.6

QUALITY CONTROL AND TRAINING GOVERNMENT QUALITY ASSURANCE C.3.4.1

QUALITY CONTROL AND TRAINING CONTRACTOR QUALITY CONTROL C.3.4.2

QUALITY CONTROL AND TRAINING TRAINING C.3.4.3

PROJECT MANAGEMENT PHASE-IN OF WORK C.3.5.1

PROJECT MANAGEMENT PROGRAM SUPPORT ACTIVITIES C.3.5.2

PROJECT MANAGEMENT PHASE-OUT OF WORK C.3.5.3

C.3.1 ENROLLMENT

C.3.1.1 ADDRESS REFINEMENT

C.3.1.1.1 The government will provide quarterly enrollment panels, containing business establishments that need to be enrolled into the CES survey over the next calendar quarter. A business establishment consists of a single Unemployment Insurance (UI) number, which can contain one or more business worksites with shared ownership in a single state. The government will cluster UIs by Federal Employer Identification Number (EIN) and load the resulting cases into the government CATI system.

The contractor shall distribute cases to contractor staff using the government CATI system. For more information, please see Appendix D – CES CATI System.

C.3.1.1.2 The government will provide case registry information that will usually contain the business name, UI, EIN, North American Industry Classification System (NAICS) code, and address. Point of contact (POC) information, including name and telephone number, is sometimes provided to the contractor.

The contractor shall verify and update case registry information, confirm that the business is still operating, and call the establishment to identify a POC who will receive a mailed survey enrollment package. The contractor shall use the government CATI system telephone auto dialer for all outgoing calls. The contractor shall enter the POC information into the government CATI system. The contractor shall use the government CATI system to print forms.

C.3.1.1.3 The government will provide brochures, pocket folders, and envelopes to be used in an enrollment package prepared for each case.

The contractor shall prepare enrollment packages, including cover letter, form, brochures, and pocket folder, for mailing by assembling materials and stuffing envelopes. The contractor shall prevent confidentiality breaches by verifying that inserts for each envelope are only for one POC.

The government will provide pre-paid postage envelopes for all enrollment packages.

The contractor shall deliver completed enrollment packages to a designated onsite location for mail pick-up.

C.3.1.2 ENROLLMENT OF NEW SAMPLE

C.3.1.2.1 The contractor shall call the POC for the new enrollment case to verify receipt of the enrollment package. The contractor shall use the government CATI system telephone auto dialer for all outgoing calls. The contractor shall solicit survey participation, which will typically include the following steps:

C.3.1.2.1.1 Explain the CES survey C.3.1.2.1.2 Detail the scope of the survey request C.3.1.2.1.3 Answer respondent questions C.3.1.2.1.4 Verify UI number C.3.1.2.1.5 Update respondent contact information C.3.1.2.1.6 Collect respondent email address C.3.1.2.1.7 Collect data for all CES data items that are available C.3.1.2.1.8 Determine if data are consistent with quarterly UI records, and C.3.1.2.1.9 Plan for future monthly collection

Survey enrollment typically requires multiple contacts to successfully convert a POC to a CES respondent. The contractor shall make the necessary number of contacts to complete enrollment. The contractor shall work to convert initial refusals to productive cases before marking initial refusals as final refusals. The contractor shall identify another POC or transfer the case to a different interviewer if necessary in order to convert an initial refusal into a productive case.

C.3.1.2.2 The contractor shall provide at least one bi-lingual (English/Spanish) staff member at each DCC to enroll and collect data from Spanish speaking respondents (note: enrollment and collection in Spanish is rare, far less than one percent of cases).

C.3.1.3 REFUSAL CONVERSION

The government will identify previous final refusal cases that should be re-attempted for enrollment. Typically, the government will wait for at least one year after initial attempted enrollment to provide these refusal cases to the contractor. The government will load these refusal conversion cases into the government CATI system.

The contractor shall distribute refusal conversion cases to contractor staff using the government CATI system. The contractor shall attempt to obtain survey participation from the cases selected for refusal conversion by following the steps for Address Refinement and Enrollment.

C.3.1.4 ENROLLMENT PERFORMANCE MEASURES

C.3.1.4.1 QUALITY STANDARDS

C.3.1.4.1.1 The contractor shall attempt to enroll all new sample cases provided by the government within a calendar quarter. Approximately 70,000 cases will be provided per year for enrollment. This number can vary plus or minus 5% per year.

C.3.1.4.1.2 The contractor shall attempt to enroll all refusal conversion cases provided by the government within a calendar quarter. Approximately 6,000 cases will be provided per year. This number can vary plus or minus 20% per year.

C.3.1.4.2 ACCEPTABLE QUALITY LEVEL (AQL)

The following rates are calculated for each panel based on the number of months since panel assignment at CES 1st Closing. The government will calculate these rates at the aggregate level and for each individual DCC. The expectation of the contractor is that the aggregate level is equal to or greater than the rates listed in the chart below and that each DCC is no more than five percentage points lower than any of the rates listed in the chart below. For more information on the calculation of these rates, please see Appendix C- CES Response Rate Calculation.

Rate 2nd month 3rd month 4th month 5th month 6th month Enrollment Response 40% 60% 75% 75% 75%

AE Hours & Earnings Response - - 55% 55% 55% PE Hours & Earnings Response - - 45% 45% 45% UI Enrollment Verification - - 95% - - Refusal Conversion - - 25% 25% 25%

C.3.1.4.3 MONITORING METHOD

Review by DCC Manager, COR, and Government Task Monitor.

C.3.2 COLLECTION

C.3.2.1 MULTI-MODAL COLLECTION

The contractor shall collect monthly CES data primarily via Computer Assisted Telephone Interviewing (CATI). The contractor shall collect data at the earliest time they are available from each respondent. The contractor shall collect data according to the CES production schedule with the goal of maximizing collection by CES 1st Closing. The contractor shall collect data that were not available by CES 1st Closing as soon as they become available, preferably before CES 2nd Closing.

The contractor shall work with respondents during the initial five-month CATI collection period (this period may be reduced to three months in the future)with the goal of preparing the respondent for a successful transition to web reporting. The contractor shall use this five-month period to effectively train the respondent on providing accurate and timely data in preparation for moving the respondent to web reporting. The contractor shall work to minimize the number of respondents that remain on permanent CATI, which should be reserved only for respondents that are unable or unwilling to submit data on the government web collection tool.

The contractor shall ensure that respondent contact information remains up-to-date. If the respondent is no longer available at the establishment, the contractor shall identify and enroll a new respondent. The contractor shall verify the UI number(s) for each case at least annually.

The contractor shall notify the government if the establishment has been part of a merger, acquisition or experienced any other major structural change.

The contractor shall notify the government if a case needs to be transferred to another DCC and assist with the transfer and re-assignment of the case. Transfers between DCCs are necessary when either the contractor or the government identifies that a case being collected at one DCC shares a common respondent with a case being collected at another DCC.

The government will provide updates, when applicable, to UI account information. These changes typically consist of updates to the UI number, change to the worksites within that UI account, change to NAICS industry code, or similar changes related to the structure or classification of the UI account. When needed, the contractor shall confirm these updates with the respondent and ensure that data reported match the updated UI account information.

C.3.2.1.1 CATI COLLECTION

C.3.2.1.1.1 The government will provide a CATI system designed specifically for CES collection. The contractor shall use the government CATI system telephone auto dialer for all outgoing calls. For more information, please see Appendix D – CES CATI System.

The government will print and mail monthly reminders to CATI respondents specifying the previously agreed upon date for collection. The government may at some point in the future transition to sending electronic monthly reminders to CATI respondents.

C.3.2.1.1.2 The contractor shall call respondents that have agreed to participate in the CES survey to collect all data items for the current month and any missing data for the previous two months. The contractor shall enter data provided by the respondent into the government CATI system.

The contractor shall review and address edit & screening errors flagged by the government CATI system. The contractor shall verify the data with the respondent and either correct it or address it by entering an appropriate comment code and documenting in the notes section.

The contractor shall schedule with the respondent a date and time for collecting data for the subsequent month and enter this date and time into the government CATI system The contractor shall attempt to schedule future collection so that data can be collected before the next month’s CES 1st Closing.

The government will provide a notes template used to record all pertinent information about all contacts with respondents. The contractor shall enter notes in the government CATI system based on this template.

C.3.2.1.1.3 During the initial five-month period of CATI collection, the contractor shall prepare respondents for a transition to reporting data via the government provided data collection website (web reporting). The contractor shall collect and enter a valid email address into the government CATI system. The contractor shall provide detailed information to the respondent on entering data on the BLS web collection tool.

Some CES respondents have experience with CES web reporting from prior requests. For these cases, the contractor shall determine if the respondent is capable of a conversion to web reporting on an accelerated schedule (i.e. prior to five months of CATI reporting).

Further, the contractor shall be responsive to those, both experienced and inexperienced in CES web reporting, who would otherwise become final refusals if not converted to web reporting. In all cases, the contractor shall always ensure that a respondent has provided data by CATI for at least one month and fully understands the survey request before being converted to web reporting. The contractor shall work to minimize these accelerated web conversions and reserve them only for situations when it is a necessity to maintain a productive respondent.

The contractor shall collect data by permanent CATI from respondents who are unable or unwilling to report by web. For these cases, the contractor shall document in the case notes why the respondent will report by permanent CATI and not by web.

C.3.2.1.2 WEB COLLECTION

C.3.2.1.2.1 The government will provide and operate a web collection tool(s). The government will provide all hardware, software and technical staff to support the web collection tool(s). The government will develop updates, both major and minor, to the web collection tool(s). For more information, please see Appendix E – CES Web Collection Tool.

The government will print and mail replacement forms, including instructions, to each web respondent annually. Note: these forms are used by web respondents as reference and not used for actual data submission.

The government will send monthly email reminders to web respondents. The current schedule for sending these emails is listed in the chart below. The government may alter this schedule in the future, based on either internal or external research, with the goal of improving web response.

*Respondents can elect to not receive email prompts.

The contractor shall place Non-Response Prompt (NRP) telephone calls to web respondents who did not submit data after receiving NRP emails. This should occur during the week leading up to CES 1st Closing and the week leading up to CES 2nd Closing. If the contractor is not able to speak with the respondent during the first call, the contractor shall place a follow-up call to the respondent to remind him or her to report by web. If the contractor is unable to speak to the respondent during a follow-up call, the contractor shall leave a voicemail reminder to report data. If the respondent offers to provide data by telephone instead of by web, the contractor shall enter these data into the government CATI system. The contractor shall use the government CATI system telephone auto dialer for all outgoing calls.

Currently, all NRP telephone calls are made at the Fort Walton Beach DCC. If the contractor wants to distribute these NRP cases differently, the government may need approximately six months to make system updates to the government CATI system before these cases can be distributed to DCCs other than Fort Walton Beach.

Sent to 1st Closing 2nd Closing

Advance Notice All web respondents*

12th of the month or next business day Not sent

Non-response prompt (NRP) Nonrespondents*

About 10 days before 1st closing

About 10 days before 2nd closing

Last chance Nonrespondents* Morning of 1st Closing Not sent

C.3.2.1.2.2 The government will provide monthly delinquent panels, containing web cases that have not provided data for two or more consecutive months. The government will identify these cases and load them into the government CATI system.

The contractor shall distribute these delinquent cases to contractor staff.

The contractor shall call these delinquent respondents to resolve any issues that are preventing successful web reporting. If the respondent is unable or unwilling to return to reporting data by web, the contractor shall collect data by permanent CATI using the government CATI system.

C.3.2.1.3 FAX COLLECTION

C.3.2.1.3.1 The contractor shall minimize the use of fax collection and shall use fax collection only to avert a refusal. The government may, at its discretion, decide to discontinue fax collection during the term of this contract.

C.3.2.1.3.2 The contractor shall fax monthly reminders to fax respondents using the government CATI system. The contractor shall pick-up faxes from fax machine(s) and distribute to assigned contractor staff. The contractor shall manually enter data into the government CATI system. The contractor shall review and address edit & screening errors flagged by the government CATI system. The contractor shall verify the data with the respondent and either correct or address the data by entering an appropriate comment code and documenting in the notes section.

C.3.2.1.4 EMAIL COLLECTION

C.3.2.1.4.1 The contractor shall minimize the use of email collection and use email collection only to avert a refusal. The contractor shall follow all established rules and guidelines on the use of email for data collection provided by the government.

C.3.2.1.4.2 The government will email monthly reminders to email respondents.

C.3.2.1.4.3 The contractor shall manually enter data into the government CATI system from emails received. The contractor shall review and address edit & screening errors flagged by the government CATI system. The contractor shall verify the data with the respondent and either correct or address the data by entering an appropriate comment code and documenting in the notes section.

C.3.2.2 COLLECTION PERFORMANCE MEASURES

C.3.2.2.1 QUALITY STANDARDS

C.3.2.2.1.1 The contractor shall attempt to collect cases before CES 1st Closing.

The contractor shall collect additional cases not available by CES 1st Closing before CES 2nd Closing.

C.3.2.2.1.2 The contractor shall attempt collection on all CATI cases provided by the government. Approximately 80,000 CATI cases will be provided for collection monthly.

C.3.2.2.1.3 The contractor shall contact all NRP telephone cases provided by the government. This includes approximately 20,000 cases monthly during the week leading up to CES 1st Closing and 3,000 cases monthly during the week leading up to CES 2nd Closing.

C.3.2.2.1.4 The contractor shall attempt collection on all delinquent cases provided by the government. This includes approximately 600 cases per month.

This number can vary plus or minus 20% per month.

C.3.2.2.2 ACCEPTABLE QUALITY LEVEL (AQL)

The following rates are calculated based on the CES closing schedule. These rates will be calculated both at the aggregate level and for each individual DCC. The expectation for the contractor is that the aggregate level is equal to or greater than the rates below and that each DCC is no more than five percentage points lower than any of these rates.

For more information on the calculation of these rates, please see Appendix C- CES Response Rate Calculation.

Rate 1st Closing 2nd Closing DCC Collection 85% 92% Total Response (DCC and web) 64% 72% DCC Collection - AE Hours & Earnings 50% - DCC Collection - PE Hours & Earnings 40% - Web/TDE Nonresponse Prompting 100% 100%

The following rates are calculated based on the number of months since assignment. For more information on the calculation of these rates, please see Appendix C- CES Response Rate Calculation.

Rate 2nd month 3rd month 4th month 5th month 6th month UI Re-verification 95% - - - - Delinquent conversion - - 65% 65% 65%

C.3.2.2.3 MONITORING METHOD

Review by DCC Manager, COR, and Government Task Monitor.

C.3.3 HELP DESK

C.3.3.1 WEB AND TDE RESPONDENT SUPPORT

C.3.3.1.1 The contractor shall provide coverage for incoming telephone calls from 7:30 a.m. to 8:00 p.m. ET on all government work days (business days excluding federal holidays).

The government will provide initial training on handling help desk inquiries. The contractor shall provide subsequent training.

The contractor shall answer incoming telephone calls from web and TDE respondents and manually enter any data received into government-provided software. The contractor shall be prepared to handle questions about how to report data and what type of data will need to be reported. The contractor shall forward any technical or otherwise complex questions to the government. The contractor shall process approximately 1,500-2,000 incoming calls per month.

C.3.3.1.2 The contractor shall, in support of web data collection, reply to respondent emails in the CESMAIL email account. The contractor shall process approximately 200 emails per day during the last 2 weeks of each month and 5 emails per day during the first two weeks of each month.

C.3.3.2 WEB AND TDE EDIT RECONCILIATION AND DATA QUALITY

C.3.3.2.1 The government will provide a list each day of reports whose microdata failed edit & screening tests for All Employee, Payroll, and/or Hours. The contractor shall call web and TDE respondents whose microdata failed these edit & screening tests for All Employee, Payroll, and/or Hours. The contractor shall obtain corrected data when possible or assign explanatory comment codes and enter these data into government-provided software. This process is referred to as Edit Reconciliation (ER). The contractor shall process approximately 1,600 cases per month.

C.3.3.2.2 The government will provide a file each month to the contractor of early microdata (data received before the 12th day of the reference month) received from TDE respondents. The contractor shall contact these TDE respondents via telephone or email to research this early microdata to determine its validity. The contractor shall process approximately 60 cases per month.

C.3.3.2.3 The government will provide a daily sync report containing a list of reports with a response code of 74. The contractor shall contact respondents via telephone or email to research TDE and web reports with a response code of 74, which indicates that a worksite-level report, or ‘child,’ has responded instead of the expected aggregate report, or ‘parent.’ The contractor shall ensure the respondent is using the correct report ID and obtain data for the parent report when possible. The contractor shall process one to five cases per business day.

C.3.3.2.4 The government will provide a daily sync report containing a list of reports with multiple pay systems. The contractor shall contact web and TDE respondents via telephone or email to research units with multiple pay systems, and ensure that these reports are categorized correctly and are accurately reporting earnings data. The contractor shall process approximately 10 cases per month.

C.3.3.2.5 CES program stakeholders within the government will generate On-Track tickets, which is an internal system used to seek clarification of collected data. The contractor shall answer these On-Track tickets for data collected by web and TDE. This will involve placing telephone calls to respondents to verify data including but not limited to: address, employment level, UI number, and EIN. The contractor shall process between 50-75 On-Track tickets per month.

C.3.3.2.6 The contractor shall create On-Track tickets for reports that have large differences in All Employees data reported to the CES program versus the QCEW program. The collection entity (DCC, state government, web, TDE) responsible for a particular report will resolve On-Track tickets for that report. In cases concerning web or TDE respondents, the contractor shall contact the respondent to verify the All Employees data being reported to CES.

C.3.3.3 DATA ENTRY

C.3.3.3.1 The contractor shall manually enter data collected by the government via transcript (postal mail) into government-provided software. The contractor shall process approximately 10 reports per month.

C.3.3.3.2 The contractor shall manually enter respondent data collected via fax at the BLS National Office into government-provided software. The contractor shall process approximately 30 reports per month.

C.3.3.4 DATA PROCESSING

C.3.3.4.1 The contractor shall run and monitor the progress of Daily Data Collection Branch (DCB) processing jobs twice per business day, once in the morning (typically around 8 a.m.) and once in the afternoon (typically around noon). This involves invoking jobs and monitoring a series of about 50 system-generated emails. The contractor shall inform government staff if the jobs fail to run or if the system-generated emails are not received.

C.3.3.4.2 The government will provide a monitoring system for the incoming and outgoing telephone traffic of the Help Desk Call Center. The contractor shall generate daily exports from this monitoring system for use by government staff.

C.3.3.5 OTHER HELP DESK SUPPORT ACTIVITIES

C.3.3.5.1 The government will provide a list of respondents whose contact information (address, fax number, and/or email address) must be updated because correspondence from the CES program has failed to reach the respondent. The contractor shall utilize internet searches and telephone inquiries to research failed faxes, failed emails, and postal mail that were returned or failed to reach the intended recipient. The contractor shall update the government’s records with corrected fax numbers, email addresses, and postal addresses of respondents. The contractor shall process approximately 200 cases per month, plus an additional 500-1,000 cases related to the annual CES schedule mailing in January.

C.3.3.5.2 The government will provide inputs for software used to send faxes to respondents. The contractor shall use these government-provided inputs to send faxes which prompt units to report data. The government will provide a production schedule that will include the days that faxes should be sent (typically six days a month). The contractor shall send these fax prompts.

C.3.3.6 HELP DESK PERFORMANCE MEASURES

C.3.3.6.1 QUALITY STANDARDS

C.3.3.6.1.1 The contractor shall provide courteous and professional service by telephone, email and in answering On-Track tickets.

C.3.3.6.1.2 The contractor shall enter microdata and registry information in a timely and accurate manner.

C.3.3.6.1.3 The contractor shall run reports and programs on schedule.

C.3.3.6.2 ACCEPTABLE QUALITY LEVEL (AQL)

C.3.3.6.2.1 Timely (within one business day) and accurate entry of microdata and registry information.

C.3.3.6.2.2 Timely (between ½ a day and one business day depending on the request) and accurate generation of requested reports.

C.3.3.6.2.3 A missed call rate of 3% or less of incoming calls per day.

C.3.3.6.2.4 95% of emails answered within two business days, multi-pay reports contacted within two business days, and edit & screening failures contacted within five business days.

C.3.3.6.2.5 85% of On-Track tickets are closed within two weeks, 100% closed by two months.

C.3.3.6.2.6 100% of ER reports are called and 80% of ER reports are resolved prior to the CES 2nd Closing for the report’s reference month.

C.3.3.6.3 MONITORING METHOD

Review by CES Registry & Data Flows Branch, COR and Government Task Monitor.

C.3.4 QUALITY CONTROL AND TRAINING

C.3.4.1 GOVERNMENT QUALITY ASSURANCE

The government will perform Quality Assurance (QA) activities to ensure data collection is in compliance with CES procedures and to ensure data integrity. The government will ensure the integrity of the data collected by confirming the interviewer contacted the respondent and by verifying the accuracy of the data collected. The government will perform case reviews and re-interviews. The government may monitor Interviewer telephone calls. The government will verify data entry, data code utilization, collection comments entered, proper use of collection procedures and systems, and employ additional QA activities as needed.

The government will use case reviews to review critical elements including:

C.3.4.1.1 Notes C.3.4.1.2 UI number verification C.3.4.1.3 Contact information C.3.4.1.4 Data edits C.3.4.1.5 Data item collection, and C.3.4.1.6 Other elements as needed

The government will use re-interviews to review critical elements including:

C.3.4.1.7 Data integrity C.3.4.1.8 UI number verification C.3.4.1.9 Reporting location C.3.4.1.10 Pay period verification C.3.4.1.11 Data accuracy C.3.4.1.12 Data items collected, and C.3.4.1.13 Other elements as needed

The government will share results of these QA activities with the contractor. The contractor shall address these issues to the satisfaction of the government within five business days of notification by the government; if this is not possible, the contractor shall provide the DCC Manager a reasonable explanation.

The government will notify the contractor of concerns or issues identified by other methods.

The contractor shall address these issues to the satisfaction of the government within five business days of notification by the government; if this is not possible, the contractor shall provide the DCC Manager a reasonable explanation. The contractor shall resolve monthly actions items provided by the government.

C.3.4.2 CONTRACTOR QUALITY CONTROL

The contractor shall investigate and resolve the following issues, but not limited to:

C.3.4.2.1 The contractor shall investigate if the CES All Employees (AE) is not similar to the QCEW Employee Database (EDB) AE. The contractor shall review the CES AE and EDB AE and determine if there is an error. The contractor shall correct errors and notify the government within five business days of any issues that the government needs to resolve.

C.3.4.2.2 The contractor shall investigate if the UI number has not been verified by the interviewer. The contractor shall verify the UI number if possible. If the respondent refuses or is unable to verify the UI number, the contractor shall verify the EIN with the respondent according to government provided guidelines. If the UI number cannot be verified, the contractor shall provide an explanation in the case notes in the government CATI system.

C.3.4.2.3 The contractor shall verify that cases have either been moved to web collection or permanent CATI after the initial five-month CATI collection period. The contractor shall verify that cases that are in permanent CATI mode have sufficient justification for not being moved to web collection. Please refer to C.3.2.1.1.3 for discussion of transitioning CATI respondents to web collection.

C.3.4.2.4 The contractor shall ensure that UI numbers are properly aggregated. The contractor shall correct the aggregation or notify the government if unable to make the correction when the UI numbers are not properly aggregated.

C.3.4.2.5 The contractor shall perform re-interviews on cases where data falsification is suspected. During re-interviews, if the respondent indicates that he or she did not talk to an interviewer on the day indicated, the contractor shall immediately notify the government DCC Manager of a potential falsification of data. During re-interviews, if the respondent indicates that he or she did not report data that is shown in the government CATI system, the contractor shall immediately notify the government DCC Manager of a potential falsification of data.

C.3.4.2.5.1 The contractor shall immediately begin an investigation, including a case review of the interviewer’s completed assignment.

C.3.4.2.5.2 The contractor shall document the cases in which falsification occurred, correct the data in the government CATI system, and provide these results to the government DCC Manager throughout the investigation.

C.3.4.2.5.3 The contractor shall evaluate the circumstances and consult with the DCC Manager, CO and COR to determine whether or not the interviewer should be removed from data collection activities.

C.3.4.2.6 The contractor shall provide the government with a Contractor Quality Control Plan that ensures data is in compliance with CES procedures and ensures data integrity utilizing the surveillance and recording functions included in the telephone system software.

A final Quality Control Plan must be provided within 90 days following award.

C.3.4.3 TRAINING

The contractor shall certify new employees for the applicable new hire training modules before the new employees can independently collect data for each module topic that is relevant to their assigned work. The contractor shall provide email documentation that certifies a new contractor hire is capable of independently collecting data that are in compliance with CES procedures and data integrity guidelines.

The government will provide new hire training materials with four separate module certifications: Address Refinement, Enrollment, Data Collection, and Non-response Prompting.

Each module involves 2-3 days of classroom training.

The contractor shall provide all training, using government provided or government approved training materials.

C.3.4.3.1 The contractor shall certify that new hires complete the Address Refinement module. The contractor shall not allow interviewers to independently perform Address Refinement without this certification. The contractor shall ensure that this training is completed within 60 calendar days of a new hire’s starting date if the new hire will be performing Address Refinement duties.

C.3.4.3.2 The contractor shall certify that new hires complete the Enrollment Module successfully and complete five scheduled enrollment interviews under the observation of a contractor mentor to receive their Enrollment Certification. The contractor shall not allow interviewers to independently enroll cases without this certification. The contractor shall ensure that this training is completed within 60 calendar days of a new hire’s starting date if the new hire will be performing enrollment duties.

C.3.4.3.3 The contractor shall certify that new hires complete the Collection Module successfully and complete fifteen data collection interviews under the observation of a contractor mentor…