Attachment 01 - QA Log - PAP - Responses to Vendor Questions.xlsx
XLSX spreadsheet 29 KB Posted
- Attached to
- 0200 - PAP Customer Satisfaction Survey
- Solicitation number
- OAA-26-00200
About this file
This file is a Questions and Answers (Q&A) log documenting vendor inquiries and government responses regarding the Participant Assistance Program (PAP) Customer Satisfaction Survey Services contract opportunity. The document provides clarification on survey methodology, data collection requirements, reporting deliverables, and contract terms for the solicitation OAA-26-00200.
The contract requires the successful contractor to conduct ongoing telephone-based customer satisfaction surveys of EBSA's Participant Assistance Program inquirers, with flexibility to propose alternative collection modes. The contractor must achieve 95% confidence levels with margins of error of approximately 10% at the Benefits Advisor (BA) level, 5% at the office level, and 1% at the agency level across 13 regional offices. Survey data collection will be conducted on approximately 200,000 annual inquiries, with multilingual capability in 11 languages (less than 3% of surveys require non-English administration). The government will provide contact information (name, phone number, zip code, language preference, and occasionally email/mailing address) on a bi-weekly basis, and the contractor will host survey data on FedRAMP-authorized and ATO-compliant servers. The contract is Firm Fixed Price with an anticipated start date of September 30, 2026, and a base period of one year (9/30/2026–9/29/2027) with four optional renewal periods through September 29, 2031. Deliverables include quarterly and annual trend analysis reports, at least 100 completed surveys annually per BA for valid reporting, non-response bias analysis, annual training for 13 field offices (separate sessions), up to two executive presentations, and final reports in PDF format—all priced as a single comprehensive effort with separate CLINs for office-level oversampling. The OMB Paperwork Reduction Act clearance (Control Number 1210-0161, expiring 12/31/2027) will require review if survey methodology changes, with an anticipated OMB approval deadline of January 1, 2027. No respondent incentives are permitted, no historical baseline data or previous methodology reports will be provided, and survey response rates historically targeted 30%.
View the file
Other files
| File | Type | Posted |
|---|---|---|
| PWS - PAP FY26.docx | DOCX document | |
| RFI - PAP.doc | DOC document | |
| Attachment 01 - QA Log - PAP.xlsx | XLSX spreadsheet |
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
Sheet1
| Section | Question | Response | |
| N/A | Whose the incumbent? | Gallup | |
| PWS 5.0 Task 1 & 6 | Does EBSA require the survey to identify and report on the performance of specific, named Benefits Advisors (BAs), or is the evaluation intended as an aggregate metric for each of the 13 regional offices? | specific BAs and the regional office in the aggregate, and the BAs across the country together in the aggregate | |
| PWS 5.0 Task 1 | To achieve 95% accuracy at the individual office level, has EBSA set a minimum number of completed surveys per office, or should the contractor propose a sampling plan based on regional inquiry volumes? | the contractor proposes | |
| PWS 5 Task 3 | Will the contractor be provided direct access to the EBSA inquiry database, or will the government provide pre-scrubbed contact lists for the three inquirer categories on a recurring basis? | We will provide the data to the contractor. | |
| RFI Generatl PWS 5 | Does the DOL permit the use of nominal respondent incentives (e.g., small gift cards) to ensure high response rates among "Invalid Claim" inquirers? | No | |
| PWS 5.0 Task 4 | Will the successful offeror be provided with raw data or final reports from the previous contract cycle to establish a baseline for the required Quarterly Trend Analysis? | No | |
| PWS 5.0 Task 2 | What is the anticipated internal deadline for the OMB/PRA submission to ensure data collection commences according to the proposed Period of Performance? | The anticipated contract would start on or around 9/30/2026 and the requested OMB deadline would be 1/1/2027. | |
| Section 4.0 Instructions | How should we quote these services? As a cost-per-survey; overall services with an estimated n-size? Should we provide a costing for a set number of surveys as an example? | Provide costing for a set number of surveys as an example. Note that the contract requires more than just surveying; we need quarterly and annual reporting on survey results and regular meetings between the contractor and the agency; as well as an annual training. | |
| PWS | What is the current approved survey instrument? | The current survey was created by the current contractor with input from the agency. | |
| PWS | What is the current OMB-approved survey instrument? | The current survey was created by the current contractor with input from the agency. | |
| PWS | What is the breakdown in % of the inquiries received in different languages? | In FY 2025, a language other than English was used in less than 3% of completed surveys | |
| PWS | Are these surveys conducted online, over the phone, or mixed mode? | Telephone currently but the vendor can propose alternatives | |
| PWS | What is the timeline for conducting all data collection? | Data collection for the quarter must be completed with time left to analyze the results and share them with the agency. How much time does the potential vendor think they would need for that? | |
| PWS | What is the sample size for the subtasks (office level study; benefits advisor study; office level oversampling? | These numbers fluctuate, vendor determines. | |
| PWS | Is the vendor hosting the survey and the data collected in their servers? | yes | |
| PWS | What are the security requirements? FedRamp, ATO? | With regard to what specifically? Need example. Generally, DOL requires both FedRAMP authorization and ATO (Authorization to Operate) security requirements for cloud computing products and services. Authorizations must be documented through an ATO or, where an acceptable and current ATO is available from a third party, an ATU (Authorization to Use) | |
| Section 4.0 Instructions | Do marketing materials in the Appendix count toward the page limit? | No | |
| PWS | Is this a phone only survey? | yes, currently, but the vendor can propose alternatives | |
| PWS | What kind of contact information do you have: phone, email? | phone currently | |
| PWS | Is there a current OMB approved questionnaire that can be used, or do we have to get a new one approved? | See 45 below | |
| Section 4.0 Instructions | Can you clarify what is included in the 10-page limit? Is the 10-page limit to answer the questions or the answers to the questions is separate? | The 10 page limit incorporates Attachment 01 Market Research Questionnaire | |
| 3. | Conduct Surveys and Studies to Evaluate the Participant Assistance Program | Can the government provide estimated percentages of the number of inquiries from among the over 200,000 received which have required support for each of the languages used in EBSA publications, or alternately an estimated percentage of the number of inquiries requiring support for primary languages other than English? | In FY 2025, a language other than English was used in less than 3% of completed surveys |
| PWS – Objective & Scope; focus on data collection and reporting, but no mention of continuous decision systems | How does EBSA intend to utilize survey data beyond reporting to support ongoing program optimization, decision-making, and performance improvement? | We only use the survey data for program optimization, decision making and performance improvement. | |
| PWS – Task 3 Analysis; regression and charts mentioned, but no predictive or AI methods specified | Does EBSA have interest in incorporating advanced analytics (e.g., predictive modeling, driver analysis, or AI-assisted insights) into the study? | EBSA supports adaptive methodologies, such as skip logic, that would improve response completion rates and data quality. | |
| PWS – Scope; primarily references EBSA-provided data only | Are there opportunities to integrate external datasets (e.g., demographic, behavioral, economic, or regional data) to enrich analysis and benchmarking? | No | |
| PWS – Monthly, Quarterly, and Annual Reporting; static reporting cadence defined | Would EBSA consider implementing real-time or near real-time dashboards for continuous monitoring and decision support? | No | |
| PWS – Objective; emphasizes satisfaction and improvement, but not behavioral analytics depth | Does EBSA expect deeper behavioral analysis (e.g., service pathways, interaction patterns, or outcome drivers) beyond standard satisfaction metrics? | Yes | |
| PWS – Scope; qualitative data mentioned without defined sentiment analysis methodology | Is there interest in incorporating sentiment analysis of qualitative responses to better understand participant perceptions? | Yes | |
| PWS – Task 3; segmentation by office and inquiry type mentioned, but not advanced segmentation | Would EBSA value advanced segmentation of participants (e.g., demographics, inquiry type, engagement channel, or behavioral profiles) to generate more actionable insights? | No | |
| PWS – Task 1; survey instrument updates allowed, but modernization limits not defined | To what extent can the contractor modernize or enhance the survey instrument while maintaining longitudinal comparability with prior studies? | The agency needs to be able to compare the results across time. That is possible if we keep the sampling methodology and survey questions fairly consistent. However, the vendor can propose changes to enhance or modernize the survey, particularly if these approaches would increase response rates or reduce costs, which EBSA would then consider. | |
| PWS – Task 1; methodology described, but no adaptive or dynamic approach specified | Does EBSA expect or encourage innovative or adaptive methodologies (e.g., dynamic surveys, skip logic, or evolving instruments based on interim findings)? | EBSA supports adaptive methodologies, such as skip logic, that would improve response completion rates and data quality. | |
| PWS – Non-Response Bias Analysis; methodology referenced, but thresholds not defined | What response rate thresholds and statistical standards are required to ensure validity across reporting levels? | EBSA requires that the sample be designed to ensure representativeness of the population being sampled and the sampling precision assumptions at each reporting level balance statistical rigor with cost. If the Office and Benefits Advisor (BA) Level Study Subtasks are exercised, the most granular sampling unit for this project will be at the BA level, and there will naturally be a higher margin of error for that level than at the office and agency-levels. Historically, this requirement has a stated precision goal of sampling and COMPLETING a sufficient number of surveys to allow for 95% confidence at all levels, with margins of error of ~10% at the BA level, ~5% at the office level, and ~1% at the agency level, though the contract awardee will have to calculate the specific MOEs upon receipt of the contract based on actual inquiry data. A vendor should anticipate making sufficient survey attempts in order to achieve enough survey completions to achieve the stated precision goals. | |
| PWS – Task 1 and Non-Response Bias Analysis; requirements mentioned but not prescriptive | Are there preferred approaches for sampling, stratification, and weighting, particularly to address non-response bias? | The universe of all inquiries should be first stratified by the 13 district/regional offices. Each of those offices will be further sub-stratified by BA (Benefits Advisor) level inquiries. Presumably weights would adjust for non-response and historically the agency has required a non-response bias analysis. Vendor would need to determine their follow-up protocol for responders. | |
| PWS – Conduct Surveys and Studies; traditional methods described, innovation not specified | Is the contractor expected to propose innovative or multi-modal data collection approaches (e.g., mobile-first, digital, or mixed-mode outreach)? | The agency is open to such proposals | |
| PWS – Conduct Surveys; mentions contact attempts but limited detail on channels | Are there constraints or preferences regarding outreach channels (e.g., phone, email, SMS, web) and contact frequency? | Historically we have relied on telephone surveys but would consider alternatives | |
| PWS – Conduct Surveys; mentions contact attempts but limited detail on channels | Are there constraints or preferences regarding outreach channels (e.g., phone, email, SMS, web) and contact frequency? | this is a duplicate of the line above | |
| PWS – Task 3; regional analysis required, but no geospatial methodology specified | Would EBSA consider incorporating contextual or geospatial data collection to better understand regional or service-level differences? | No | |
| PWS – Non-Response Bias Analysis; high-level guidance provided, but operational rules not detailed | What operational rules should govern non-response handling, incomplete responses, and respondent follow-ups? | The contractor would need to develop a protocol to deal with non-responders prior to beginning the survey and adhere to that protocol. Sampling weights would need to be adjusted for nonresponse. | |
| PWS – Reporting Deliverables; static reports and presentations specified | Does EBSA expect interactive or customizable reporting tools (e.g., dashboards, drill-down analytics) in addition to static reports? | No | |
| PWS – Reporting & Training; outputs defined, but usage/application not detailed | How does EBSA envision stakeholders using data outputs beyond formal reporting (e.g., for operational decision-making or performance management)? | for program optimization, decision making and performance improvement | |
| PWS – Task 3; descriptive reporting emphasized, prescriptive expectations not defined | What level of analytical depth is expected (e.g., descriptive vs. diagnostic, root-cause, and prescriptive recommendations)? | Vendor proposals can include any/all of these | |
| PWS- Task 1 | Will the Government provide the current/previous survey instrument, study protocol, and historical data from prior contractors? If so, when will these materials be made available? | No | |
| PWS - Scope | What survey platform/technology was used previously, and is the Government expecting the new contractor to use the same platform or select their own? | New contractor is not expected to use the pre-existing platform | |
| PWS - Scope Item 2 | What is the current status of the Paperwork Reduction Act (PRA) clearance? Is there an existing OMB Control Number that would be renewed, or will this require a new submission? | The current control number is 1210-0161 with an expiration date of 12/31/2027. Changes to the survey instrument or survey methodology will likely require review by OMB. The supporting statement for the current survey is available at https://www.reginfo.gov and searching for the ICR with OMB Control number 1210-0161. | |
| PWS - Scope Item 2 & Task 2 | Given that PRA approval can take 6-9+ months, what is the expected contract start date, and how should contractors account for potential PRA delays in their pricing and work plans? | The contract start date is anticipated to be on or before 9/30/2026. The contractor would only be working on tasks requested by EBSA. Tasks 1 and 2 would likely be needed in the first year. | |
| PWS - Task 3 | The PWS states "95% accurate at the agency and office level" - can the Government clarify if this refers to a 95% confidence level, and what margin of error is acceptable at both the agency and office levels? What design effect are you currently seeing? | This is a 95% confidence level. The agency has previously used a MOE at the office and agency levels of 4 to 5 percent. | |
| PWS - Task 3c & Task 5 | For the Benefits Advisor (BA) level study, what is the minimum number of completed surveys required to produce a "valid report" for an individual BA? | At least 100 per year per benefits advisor | |
| PWS - Task 3f | What is the expected/target survey response rate based on previous studies? | Prior studies aimed for a 30% response rate | |
| PWS - Scope | Of the 200,000+ annual inquiries, approximately what percentage have usable contact information (valid phone numbers/emails) for survey follow-up? | Most will have a valid phone number | |
| PWS - Task 3 | How soon after an inquiry interaction should customers be contacted for the survey? Is there a required lag time? | As soon as possible after the contractor receives the data from the agency, measured in days. | |
| PWS - Task 3 | For the multi-language requirement (11 languages listed), what percentage of inquirers historically require non-English surveys? | In FY 2025 it was less than 3% of completed surveys | |
| PWS - Scope | What data system(s) will the contractor need to access for inquiry contact information and performance data? Are there specific security clearance or system access requirements? | The agency will provide the data to the contractor. | |
| PWS - Scope | Will the contractor receive direct access to EBSA's tracking system, or will data extracts be provided periodically? | The agency will provide the data to the contractor. | |
| PWS - Scope Item 4 | For the "up to two in-person presentations for executive staff," will the Government reimburse travel costs, or should this be included in the fixed pricing? | Include in fixed pricing | |
| PWS - Scope Item 5 | How many training sessions are anticipated annually for the 13 field offices? Should contractors assume 13 separate sessions or consolidated regional training? | 13 separate sessions | |
| PWS - Task 4, 5, 6 | What format does the Government prefer for reports (e.g., PDF, interactive dashboards, both)? | ||
| PWS - Task 6e | For the final report requirement to include "a comparison of other similar private or public business operations," does EBSA have specific benchmark organizations or industries in mind? | The agency doesn't have specific benchmark organizations in mind | |
| RFI - Section 4 | What is the anticipated contract type (Firm Fixed Price, Time & Materials, Cost-Plus)? | Firm, Fixed Price | |
| RFI - Section 2 | What is the anticipated period of performance (base year plus option years)? | The anticipated POP is as follows: |
9/30/2026 - 9/29/2027 - Base Period 9/30/2027 - 9/29/2028 - Option Period One 9/30/2028 - 9/29/2029 - Option Period Two 9/30/2029 - 9/29/2030 - Option Period Three 9/30/2030 - 9/29/2031 - Option Period Four
| PWS - Task 3d | Is Task 3d (Office Level Oversampling) expected to be priced as a separate CLIN or included in the base survey effort? | Yes, there will be separte CLINs |
| PWS - Task 3a, 3b, 3c | Should contractors price the Agency Level, Office Level, and BA Level studies as separate options, or as a single comprehensive effort? | single comprehensive effort |
| PWS - Task 3a | The PWS references a "72% overall customer service" baseline. Is this the current target, or does EBSA have an updated performance goal? | This is the current target. |
| PWS - Scope | What is the expected frequency of data extracts from EBSA's tracking system (daily, weekly, monthly)? | Every 2 weeks |
| RFI - Section 4 & Market Research Questionnaire | Are there any restrictions on subcontracting, particularly for multilingual interviewing services? | No |
| General | What is the anticipated timeline for releasing the RFP? | We are currently in the acquisition phase and don't have a timeframe at this time |
| PWS; Task 3 | Please describe the current customer survey data collection methodology, including the number and format of contacts used to invite participation, the mode(s) of data collection (e.g., web, telephone, mixed‑mode), and typical cadence or frequency of data collection (e.g., monthly, quarterly, ongoing) of the survey administration. | Currently, telephone. Data collection is on-going. I think the rest of this is proprietary to the current contractor. |
| PWS; Task 3 | Is the contractor expected to maintain continuity with the incumbent methodology, or propose improvements/changes? | Continuity is needed for longitudinal comparisons, but open to changes with that caveat. |
| PWS; Task 3 | To assist contractors in understanding the details of the current methodology, can EBSA share the most recent methodology report? | No, it's proprietary. |
| PWS; Task 3 | What are the recent response rates for the customer survey overall and, if applicable, by data collection mode? | EBSA requires that the sample be designed to ensure representativeness of the population being sampled and the sampling precision assumptions at each reporting level balance statistical rigor with cost. If the Office and Benefit Advisor (BA) Level Study Subtasks are exercised, the most granular sampling unit for this project will be at the BA level, and there will naturally be a higher margin of error for that level than at the office and agency-levels. Historically, this requirement has a stated precision goal of sampling and COMPLETING a sufficient number of surveys to allow for 95% confidence at all levels, with margins of error of ~10% at the BA level, ~5% at the office level, and ~1% at the agency level, though the contract awardee will have to calculate the specific MOEs upon receipt of the contract based on actual inquiry data. A vendor should anticipate making sufficient survey attempts in order to achieve enough survey completions to achieve the stated precision goals. |
| PWS; Task 3 | What is the expected target number of completed surveys per year to meet the 95% confidence/accuracy requirement at agency and office levels? | EBSA requires that the sample be designed to ensure representativeness of the population being sampled and the sampling precision assumptions at each reporting level balance statistical rigor with cost. If the Office and Benefit Advisor (BA) Level Study Subtasks are exercised, the most granular sampling unit for this project will be at the BA level, and there will naturally be a higher margin of error for that level than at the office and agency-levels. Historically, this requirement has a stated precision goal of sampling and COMPLETING a sufficient number of surveys to allow for 95% confidence at all levels, with margins of error of ~10% at the BA level, ~5% at the office level, and ~1% at the agency level, though the contract awardee will have to calculate the specific MOEs upon receipt of the contract based on actual inquiry data. A vendor should anticipate making sufficient survey attempts in order to achieve enough survey completions to achieve the stated precision goals. |
| PWS; Task 3 | The PWS states, "The contractor must be able to conduct interviews in the inquirers’ primary languages. The EBSA uses Spanish, French, Arabic, Simplified Chinese, Traditional Chinese, Haitian Creole, Korean, Polish, Russian, Tagalog, and Vietnamese in publications." What is the current number of surveys conducted in each of these languages? Or can EBSA provide the number customer interactions in each of these languages, including English and non-English languages? | Less than 3% of completed surveys in FY 2025 |
| PWS; Task 3 | What customer information is available in the data file that EBSA will provide? For example, customer phone number, email, mailing address, language indicator, or any other variables that may be helpful in stratifcation for sampling or reporting. | Name and phone number. Zip code. Language. Sometimes email or mailing address. |
| PWS; Task 3 | Are there prescribed sampling rules (e.g., handling repeat inquirers, multiple contacts per case) that contractors must follow? | We are not aware of a published OMB standard regarding sampling rules. We reiterate that the purpose of this survey is to provide robust estimates of BA performance at both the regional and individual level, and so the sampling approach should be consistent with generally accepted scientific methods to produce such estimates. |
| PWS; Task 3 | Are incentives permitted or prohibited for this survey, and if permitted, are there existing guidelines or precedents for incentive type or amount? | No incentives |
| General | Beyond deliverables, what are the key success metrics for the contractor (e.g., response rates, timeliness, usability of insights)? | Response rates, timeliness, usability of insights, utility of training sessions |
| SCOPE | The scope section is ambiguous as it mixes the requirements for collecting data and conducting new surveys with the analysis existing data. Can the government separate out and identify what tasks in the scope are related to 1) analyzing the EBSA-provided data and reports, as opposed to 2) conducting new surveys? | no |
| SCOPE | In what data format is the "performance data...provided"? | I don't know what this is referring to |
| SCOPE | Can the government provide examples of what "other data not specified here" might need to be developed? | no |
| TASKS AND DELIVEABLES #3 | The government has specified 200K inquiries a year, but requires sample sizes to be designed based on inquire-ers. Can the government provide an estimate of # of unique inquirers per year? | 200K |
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reginfo.gov%2F&data=05%7C02%7Cphillips.kristy%40dol.gov%7C2603e9e4f8754178a4b208de8e9a94af%7C75a6305472044e0c9126adab971d4aca%7C0%7C0%7C639104991741061475%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=us4rOXYHGsKASjTiZN1QvmSmU78KM%2FCHQIFVW9ZBjjo%3D&reserved=0
File details come from the government source that posted it. Updated .