PWS - PAP FY26.docx

Request for Information Participant Assistance Program – Customer Satisfaction Survey

OBJECTIVE: Collect qualitative data and compile reports on the applicability of EBSA’s Participant Assistance Program.

The Department of Labor, Employee Benefits Security Administration (EBSA) is seeking professional services to collect qualitative data and compile reports on the applicability and utility of EBSA’s Participant Assistance Program (PAP). The PAP is an integral part of EBSA’s overall mission to deter and correct violations of statutes through strong administrative, civil and criminal enforcement efforts to ensure workers receive promised benefits, develop policies and regulations that encourage the growth of employment-based benefits, assist workers in getting the information they need to exercise their benefit rights, and assist plan officials to understand the requirements of the relevant statutes in order to meet their legal responsibilities.

EBSA requires a contractor who will review previous study results and develop, as necessary, new surveys to gather additional information on areas of inefficiency and analyze the resulting data to disclose valuable findings on previous studies of the Participant Assistance Program. Previous studies were comprised of robust customer satisfaction assessments developed from gathered data. The studies provided information that could be used to improve program performance. Examples of improved performance include, but are not limited to:

· Being more attuned to inquirers’ needs - Benefits Advisors (BAs) should be more adept at identifying issues that lead to benefit recoveries and enforcement leads.

· Survey data that enabled EBSA National and Regional management to identify potential training needs.

· Improved satisfaction scores which guided EBSA leadership to determine which Regional BAs needed assistance improving customer service scores. The agency was also able to see high performing BAs, and use their techniques as best practices to foster program-wide improvement,

· The study will include data from EBSA regional offices.

SCOPE: The contractor will conduct studies to assess EBSA-provided data, which will enable Regional and National offices to determine the actions necessary to provide participant assistance services. The contractor shall be responsible for maintaining and implementing all aspects of the study:

1.) Document the sampling methodology and survey instrument (Survey must include up to 10 agency EBSA-provided questions in addition to any proprietary questions used by the contractor to measure customer satisfaction and related attributes).

2.) Prepare the Paperwork Reduction Act package for submission to the Department of Labor’s Office of the Chief Evaluation Officer (OCEO) and the Office of Management and Budget (OMB) that meets all of OMB’s requirements as published on their website (https://www.reginfo.gov/public/jsp/Utilities/index.myjsp) and incorporating any changes required by either OCEO or OMB.

3.) Conduct interviews/surveys and analyzing the results [e.g., regression, bar, line charts, etc.].

4.) Provide consulting services throughout the project, including up to two in-person presentations for executive staff which could take place in DC or in one of the cities where EBSA maintains a field office.

5.) Providing annual training to each regional and district office in person or virtually as determined by EBSA.

6.) Develop interim and final detailed reports. Such analysis needs to be done at the agency, office, and individual employee levels.

To accomplish the study, it will be necessary for the vendor to maintain a customer satisfaction survey instrument and contact the customers using contact information captured by EBSA’s tracking system. This system is used by the agency to track and record the details of its interactions with its plan participants. Other performance data such as inquiry volumes, staffing, etc. will be provided to the vendor as necessary to inform its analyses. It may be necessary for the vendor to develop other data not specified here to accomplish these studies. Services include all aspects of carrying out a large-scale customer survey. EBSA has approximately one hundred Benefits Advisors in thirteen offices who receive over 200,000 inquiries each year. Approximately 80% of the inquiry volume is from telephone calls. In order to reach a statistically valid result across the entire program, it is necessary to conduct thousands of individual surveys.

TASKS & DELIVERABLES:

The contractor shall consult with and advise EBSA on all aspects of the study including but not limited to survey instrument design, methodologies, developing actions plans, etc. The contractor shall be available to answer questions about reports and occasionally provide data and advice on customer satisfaction issues that arise during the study.

1.) Review Study Protocol, Methodology, and Survey Instrument to Maintain the Survey Instrument: The Office of Outreach, Education and Assistance (OEA) is the lead agency for all interactions with the contractor for this study. EBSA requires a contractor to update the existing study protocol and provide maintenance for the survey instrument. The contractor shall meet with OEA staff to learn about the program, available data, and required modifications and updates (if necessary), the current study protocol, and survey instrument. The EBSA will provide a list of agency-specific questions required in the survey instrument. However, EBSA is open to considering other questions which the contractor may propose. The updated version of the study protocol shall be finalized and submitted to EBSA 30 days after the contract award.

2.) Project Work Plan: The contractor shall develop a project work plan that will:

a. Provide a detailed study design plan and timeline which links goals, tasks and deliverables to specific dates, with actionable strategies for accomplishing evaluation goals.

b. Describe how work on the contract will be executed for each task and include milestone dates.

c. Provide a detailed list of information or assistance needed from DOL to perform the work.

d. Include all actionable items from kickoff meeting.

e. The contractor will submit the draft workplan within 30 days after the contract award for EBSA review and approval. Within 10 calendar days after agency comments, the contractor will submit the finalized work plan.

3.) Conduct Surveys and Studies to Evaluate the Participant Assistance Program: This task entails the contractor contacting as many of the inquirers as needed to develop data that is no less than 95% accurate at the agency and office level. The Regional offices participating in the study include Atlanta, Boston, Chicago, Cincinnati, Dallas, Kansas City, Los Angeles, New York, Philadelphia, and San Francisco. The District Offices include Miami, Seattle and Washington. Statistically valid samples will be needed for each office. Inquiries are received via telephone, mail, delivery service, facsimile or the website and we expect the contractor to differentiate into these subgroups as well as by office and final disposition (granted or denied) or to provide a recommendation as to why not. The contractor must be able to conduct interviews in the inquirers’ primary languages. The EBSA uses Spanish, French, Arabic, Simplified Chinese, Traditional Chinese, Haitian Creole, Korean, Polish, Russian, Tagalog, and Vietnamese in publications.

a. Subtask: Agency Level Study: This level of effort shall comprise a study that will provide an overall agency score which will allow the Agency to fulfill reporting requirements under the Government Performance and Results Act (GPRA). GPRA is a requirement of Federal agencies to set long-term goals and objectives as well as specific near-term performance goals. The previous contracted survey baseline performance measurement was 72% overall customer service; furthermore, it should include detailed survey data combined with agency systems data to provide specific information on EBSA’s various customers’ perception of their interactions. By combining the agency systems data with survey results the contractor can obtain improved customer data. Office level results in their customer satisfaction scores (compiled from individual phone surveys provided to the offices and used for an agency-wide satisfaction score) will be provided each quarter for each office for which enough surveys have been completed to issue a valid report. Results of the individual Benefits Advisors’ service interactions based on the survey data gathered which should give a breakdown of strengths and weaknesses will be produced at the midyear and end of year for each Benefits Advisor for whom sufficient complete surveys were gathered to issue a valid report.

b. Subtask: Office Level Study: An Office Level Study shall comprise all aspects of the Agency level study and will also require additional surveying (such as previous actions with the office) in order to provide statistically valid results at the office level. The Regional offices participating in the study include Atlanta, Boston, Chicago, Cincinnati, Dallas, Kansas City, Los Angeles, New York, Philadelphia, and San Francisco. The District Offices include Miami, Seattle and Washington. This study will also incorporate agency systems information to provide actionable data for each office as described in the Agency Level study description. There shall be a mid-year report of the weighted data on all survey questions and an executive briefing in addition to the final end-of-year report. Results for individual Benefits Advisors will be produced at the midyear and end of year for each Benefits Advisor who had sufficient completed surveys gathered to issue a valid report.

c. Subtask: Benefits Advisor (BA) Level Study: The Benefits Advisor Level study shall include all aspects of the Office Level Study and will also require additional surveying in order to provide a valid report for each BA, at the midyear and at the end of the year.

d. Subtask: Office Level Oversampling: The contractor shall conduct additional surveys for offices upon request. The purpose is to oversample offices that are struggling in order better identify areas for improvement.

e. Subtask: Office Level Training: The contractor shall provide training either in person or virtually to each of the 13 field offices on each office’s results as well as making specific recommendations for improving interactions based on the data collected and analyzed. The contractor shall consult in advance with EBSA staff on the general theme and content of the training before it has been finalized.

f. Subtask: Non-Response Bias Analysis: A non-response bias analysis will be conducted to identify potential sources of non-response bias. Non-response bias associated with estimates consists of two factors - the amount of nonresponse and the difference in the estimate between the groups of respondents and non-respondents. Bias may therefore be caused by a lower response rate and/or by significant differences in estimates between respondents and non-respondents. The goal of the non-response bias analysis will be to identify potential sources of nonresponse bias on estimates and to identify potentially biased estimates. The non-response bias analysis will compare the “Early” respondents to “Late” respondents on selected key variables of primary interest. The basic assumption in such an approach is that later respondents to a survey are more similar to non-respondents than earlier respondents. In this study, data collection will be conducted using the telephone and a respondent can receive anywhere between 1 and 10 calls to complete an interview. Respondents will be divided into two groups (Early and Late respondents) based on the number of calls received. The exact definition of these two groups will be finalized after examining the distribution of the number of calls needed to complete an interview for this study. Comparison of estimates (proportions or means of selected key variables like proportion of satisfied customers, for example) between these two groups will be carried out by testing the hypothesis of equality of proportions (or means). The analysis can be done using non-response adjusted weights but can also be done using the base weights. This process will help identify estimates that may be subject to non-response bias. The non- response bias analysis shall be included in the final report.

4.) Monthly Progress Reports: Reporting shall be done on a monthly basis. Reports will be submitted on a mutually agreed upon date and should include the number of inquirers contacted, the number of refusals, number of surveys completed, and current rating and confidence level. As statistically valid data become available over the course of the study for the various subsets (e.g., office and agency level), those customer satisfaction scores shall be included in the monthly report.

5.) Quarterly Results Reports: The contractor shall provide, in writing, quarterly and year-to-date scores for each survey question at the agency level and for each office (depending on the subtask in Task 3). The scores shall be weighted to factor probability response and non-response to minimize bias in the survey, and the report shall compare scores to the previous year and the study as well as display trends on a quarterly basis. The contractor may be requested to provide briefing training either in Washington, DC, or at the annual EBSA Managers’ Training (location determined each year). Quarterly and year-to-date agency-level scores are due within 5 business days of the close of each quarter (December 31, March 31, June 30, September 30 or as specified in the agreed-upon work plan); the contractor will need to report the agency level score for the previous quarter and year to date. Quarterly and year-to-date office-level scores are due within 5 business days of the close of each quarter (December 31, March 31, June 30, September 30 or as specified in the agreed-upon work plan). Within 15 business days of the end of the second quarter and again at the end of the year, the contractor will need to report results for individual Benefits Advisors. These reports should provide data on all questions in an easy-to-read format.

6.) Final Report: The final report for the fiscal year shall include:

a. Detailed information, including weighted score data, findings and recommendations at the agency and office level.

b. Graphically displayed data (regression, bar, and line charts) to report customer satisfaction and other data.

c. A narrative to convey thorough knowledge of the participant assistance program. Detailed data must be provided on each question asked at the agency and office level for final disposition, mode of inquiry submission and other factors determined by study findings.

d. Provide analysis of the results, possible reasons for the results whether positive or negative, the strengths and weaknesses of the program at large as well as by office or other subset that provides actionable data for future reference. Results shall be provided for each question asked.

e. A comparison of other similar private or public business operations and recommendations for improvements to the customers’ experience that are based on the data gathered and the contractor’s experience of customer satisfaction.

f. Include the final study protocol and an executive summary.

The final report is due no later than December 31 each year or as mutually agreed upon.