RFP 70B06C25R00000075 - Questions and Answers.pdf

RFP 70B06C25R00000075

Questions and Answers

1. Offeror’s Question:

In Request for Proposal (RFP) 70B06C25R00000075, page 94, Item 8, the Government references the submission of an oral presentation. However, it is the only reference to an oral presentation – is this simply a generic statement and not applicable to this procurement effort?

Government’s Response:

70B06C25R00000075, page 94, 8. Oral Proposal Information, is not applicable to this solicitation and will be stricken from the final posting of this solicitation.

2. Offeror’s Question:

In RFP 70B06C25R00000075, proposal submission requirements are detailed in two spots, one starting on page 60 and one on page 89. Which submission requirements should be followed?

Government’s Response:

III.25 GENERAL INSTRUCTIONS FOR THE PREPARATION OF PROPOSALS (OCT 2008) is stricken from RFP 70B06C25R00000075. III.7 52.212-1 - INSTRUCTIONS TO OFFERORS-- COMMERCIAL PRODUCTS AND COMMERCIAL SERVICES (SEP 2023), ADDENDUM to 52.212-1 INSTRUCTIONS TO OFFERORS, PROPOSAL SUBMISSION remains the requirement for the submission of proposals.

3. Offeror’s Question:

In the Statement of Work, item 4.6.2.1, it states that testing should be near CBP ports, stations, and branches. Are the zip codes in Attachments 3a, 3b and 3c representative of the port, station and branch locations?

Government’s Response:

Yes, the zip codes in Attachments 3a, 3b and 3c are representative of the port, station and branch locations.

4. Offeror’s Question:

SOW Section 4.1.1.7 Data Migration (Page 6) – What historical test and candidate data will be provided for migration, and in what format?

Government’s Response:

If the contract is terminated, the contractor should continue to provide applicant and candidate data in the same format (e.g., test response report described in section 5.0) as requested elsewhere in the SOW until the Government is able to retrieve that data.

5. Offeror’s Question:

SOW Section 4.2.1 Assessment Volume (Page 9) – How many examinations are anticipated under this contract?

Government’s Response:

For promotional testing (the CATS portion of the contract) there has been an average of 3,900 promotional assessment batteries administered each year for the last five years.

For entry-level testing (the COACA portion of the contract), it is estimated that there could be hundreds of thousands of assessment batteries administered each year.

6. Offeror’s Question:

SOW Section 4.2.1 Exam Assembly (Page 9) – For the initial implementation, how many examinations will use fixed-form test structures?

Government’s Response:

For promotional testing (the CATS portion of the contract), all of the tests that are used (Critical Thinking Skills Test, Writing Skills Test, Job Knowledge Test, In-basket Job Simulation) are currently fixed-form/static tests. For entry-level testing (the COACA portion of the contract), both computer adaptive tests and static tests are used.

Although there are no firm plans at this point, it is possible that Linear-On-the-Fly Testing and multistage computer adaptive tests could be used.

7. Offeror’s Question:

SOW Section 4.2.1.3 Exam Assembly (Page 9) – For the initial implementation, how many examinations will utilize Linear-On-the-Fly Testing (LOFT)?

Government’s Response:

For the initial implementation, there will be no examinations using LOFT.

8. Offeror’s Question:

SOW Section 4.2.1.3 Calibration & Validation (Page 9) – Will CBP provide pre-calibrated item banks, or is the contractor expected to handle item calibration and validation?

Government’s Response:

For the initial implementation, CBP will provide pre-calibrated item banks to the contractor. The contractor should have the ability to conduct item response theory analyses (e.g., item calibration, theta score computation).

9. Offeror’s Question:

SOW Section 4.2.1.6.8 Exam Review (Page 10) – Are test takers required to review both selected and unselected response options before completing an examination?

Government’s Response:

Test-takers are not required to review both selected and unselected response options before completing an examination. The fixed-form/static tests allow test-takers to mark items as incomplete that they then can return to for review at the end of the test before submitting their test responses to the system. For entry-level static tests (the COACA portion of the contract), applicants should have the ability to review unanswered questions and to go back and review previously answered questions within a test. The computer adaptive tests do not allow this functionality.

10. Offeror’s Question:

SOW Section 4.2.1.6.21 Practice Examinations (Page 11) – Please confirm whether practice examinations are included in the required deliverables.

Government’s Response:

Yes, practice examinations will be used for both entry-level (COACA) and promotional testing (CATS) and therefore are required deliverables.

11. Offeror’s Question:

SOW Section 4.2.2 Content Import (Page 11) – What file formats will be acceptable for the delivery and import of test content?

For importing test content, Microsoft Word, Microsoft Excel, and Adobe PDF should be available for use.

12. Offeror’s Question:

Section 4.2.2.4 Examination Item Types (Page 11) – Is Computer Adaptive Testing (CAT) currently being used? If so, please specify the item response theory models and CAT algorithms preferred or required by U.S. Customs and Border Protection (CBP).

Government’s Response:

Computer adaptive testing is currently being used. The exact algorithm is considered secure test material. The contractor should have the ability to program various types of algorithms described in the computer adaptive testing literature that are customized (e.g., stopping rules, test length) to a particular test and to handle various types of item response theory models (e.g., uni- and multi-dimensional, Rasch, 1-, 2-, and 3-parameter logistic models, unfolding models, graded response models).

13. Offeror’s Question:

SOW Section 4.2.2.5 Workflow Configuration (Page 12) – Please provide examples of the types of processes that may need to be added.

Government’s Response:

For example, if a workflow consists of three tests, each administered in separate steps, then the contractor should be able to score the first test in step 1 (even if applicants do not complete the subsequent steps), the second test in step 2 (even if applicants do not complete step 3), and the third test.

14. Offeror’s Question:

SOW Section 4.4.1 Scheduling (Page 12) – Please describe the business rules governing test scheduling, rescheduling, and eligibility verification.

Government’s Response:

Separate sets of customized business rules will be provided after the contract is awarded for each testing program.

15. Offeror’s Question:

Section 4.4.1.8 Scheduling (Page 12) – Please provide examples of how scheduling and business rules differ between regularly scheduled examinations and hiring hub-specific testing events.

Government’s Response:

For example, if the Government holds an onsite hiring event at a specific location, applicants may initiate their application during the event and immediately participate in the testing process. The contractor would need to set up computer-based testing stations onsite at the specific location.

16. Offeror’s Question:

SOW Section 4.4.1.5 Service Management (Page 12) – Please clarify the expectations for managing “late tester” requests. Will this responsibility be managed by the vendor?

Government’s Response:

For various reasons (e.g., applicants who are active duty military), applicants may apply to an announcement after it has closed and have to participate in the hiring process.

The Government will handle the application process and the contractor would need to handle the testing process for these applicants.

17. Offeror’s Question:

SOW Section 4.4.1.9. Assessment Order Request Creation (Page 13) – Please clarify the expectations for this requirement. Typically, the Assessment Order Request (AOR) is created by the client’s applicant tracking system.

Government’s Response:

The contractor’s system needs to be able to accept an integration with the Government to receive information related to AORs.

18. Offeror’s Question:

SOW Section 4.5.1.2 Inquiry Handling (Page 15) – Can you please define the term “automatically” in the context of phone or email inquiry handling?

This means that if an individual contacts the Government Customer Service Center regarding an issue with the test vendor, then the service center will tell them to contact the vendor. If the vendor receives a question for the Hiring Center, then they have to tell the applicant to contact the Hiring Center.

19. Offeror’s Question:

SOW Section 4.5.1.3.1.1 (Page 15) – Will specific software or licenses be provided to the contractor, or will the contractor receive access to a CBP-managed tracking system?

Government’s Response:

No. There is no specific software or license that is needed for the contractor to perform the work. No access to any CBP-managed tracking system (e.g., applicant tracking system) is needed for the contractor to perform the work. The contractor should have its own tracking system for recording and tracking applicant and candidate inquiries and issues. The contractor should have the ability to connect to cloud-based systems managed by the Government (e.g., Salesforce).

20. Offeror’s Question:

SOW Section 4.6.2 International Testing (Page 17) – What are the required international testing locations, and what are the expected test volumes by country?

Government’s Response:

For entry-level testing (the COACA portion of the contract), testing takes place in the United States and its territories. From promotional testing (the CATS portion of the contract), there are applicants in Europe, the Middle East, Africa, Asia, Central and South America, and the Caribbean. The locations of applicants are constantly evolving, so the contractor needs to be able to accommodate any changes that come up. Every year, CBP has approximately 50 candidates every year who need to complete promotional testing.

21. Offeror’s Question:

SOW Section 4.6.2.2 Mobile Testing (Page 17) – What are the specific technical and operational requirements for mobile testing site deployments, including associated timelines?

Mobile testing site deployments are used when there are several candidates who are not located near a testing center. The government will coordinate with the vendor on mobile testing sites during the test scheduling phase.

22. Offeror’s Question:

SOW Section 4.6.2.3 Testing Capacity (Page 17) – How does CBP define “commensurate seating capacity” in the context of different testing volumes?

Government’s Response:

The contractor should ensure that there are enough seats for applicants and candidates to choose to test at a particular location and should be able to expand the number of testing seats if needed. Applicants and candidates should not have to travel more than 120 miles roundtrip to complete testing.

23. Offeror’s Question:

SOW Section 4.6.2.5 Offline Uploads to COACA (Page 17) – Please clarify what types of offline data are anticipated to be uploaded to the Centralized Online Applicant and Candidate Assessment (COACA) system. Are there specific formats, categories, or file transfer protocols required for this upload?

Government’s Response:

Although this would not take place during initial implementation, it is remotely possible paper-and-pencil tests may be administered to applicants and the item responses and scores would need to be uploaded into COACA. Data would be supplied to the contractor in the same format that the contractor’s reports stated in the SOW (see 5.0 Deliverables) are provided to the Government. Data would need to be submitted using a secure server.

24. Offeror’s Question:

SOW Section 4.6.2.9.3 Test Accommodations (Page 17-18) – What are the accommodation request requirements and expected approval workflow timelines?

Government’s Response:

The contract needs to be able to provide various accommodations in accordance with relevant U.S. laws and regulations (e.g., the Americans with Disabilities Act) such as time extensions, private testing rooms, readers, extra breaks. The Government adjudicates requests. Once an accommodation has been approved by the Government, the applicant will resume the normal workflow (with the accommodation included in the testing process). The contractor should reach out to the applicant within one business day after the request has been adjudicated.

25. Offeror’s Question:

SOW Section 4.6.2.11 – Directions for Conducting (Page 18) – Can CBP please provide a copy of the Directions for Conducting (DFC), if available?

Government’s Response:

The DFC would consist of the business rules, instructions, and time limits for each testing process and will be supplied after contract award.

26. Offeror’s Question:

SOW Section 4.6.2.24 Offline Testing (Page 20) – Can you clarify what is meant by "offline" testing? Does this refer to pencil-and-paper testing?

Offline testing refers to pencil-and-paper testing.

27. Offeror’s Question:

SOW Section 4.8 Item Bank Development (Page 22) – How many item banks will need to be imported during the implementation?

Government’s Response:

A separate item bank would be needed for each test (including separate ones for entry-level testing and promotional testing, different occupations, different tests, and different computer adaptive test item pools).

28. Offeror’s Question:

SOW Section 4.8 Item Bank Development (Page 22) – How many languages are required for each examination program?

Government’s Response:

All instructions and applicant/candidate systems should be in English. Although this would not take place during initial implementation, it is possible that the Government will include test content that is in additional languages (e.g., Spanish); however, the instructions and navigation for these tests will be in English.

29. Offeror’s Question:

SOW Section 4.8 Item Bank Development (Page 22) – What is the average number of questions per item bank?

The number of questions per item bank could range from several dozens to thousands.

30. Offeror’s Question:

SOW Section 4.8.1 Examination Item Types (Page 22) – What types of test items are used within the examination programs?

Government’s Response:

For the initial implementation, all item types would be multiple-choice with examinees providing a single response to each question. However, the vendor should have the ability to handle other item types (e.g., multiple-select items, ranking response options, free response).

31. Offeror’s Question:

SOW Section 4.10.1.15 Integration Overview (Page 24) – Is integration required with CBP-managed systems, federal platforms such as OPM systems, or other third-party systems? Please list the systems and indicate how many integrations are expected.

Government’s Response:

Yes. For the initial implementation, the contractor should be able to integrate with a CBP-managed system for handling the AOR/ASR process. The contractor should have the ability to handle integrations with Salesforce, OPM’s USA Staffing and USA Jobs system, and other systems chosen by the Government.

32. Offeror’s Question:

SOW Section 4.10.1.15 Integration Specifications (Page 24) – For each government system requiring integration (e.g., USAJobs), please specify whether the interface will be via API, file-based transfer, single sign-on (SSO), or another method.

The contractor should have the ability to handle all of the above integration methods.

33. Offeror’s Question:

SOW Section 4.11.1.7 Hiring Systems (Page 25) – Please provide a list of known hiring systems and third-party platforms that will require integration.

Please see the response to question 31.

34. Offeror’s Question:

SOW Section 4.14 Service Level Agreements (Page 27) – What are the Service Level Agreement (SLA) requirements for system uptime, response time, and technical support services?

Government’s Response:

Per section 4.14.13, the system should provide a 99% uptime functionality, except for scheduled maintenance/enhancements windows. Scheduled maintenance should be conducted during non-busy times (e.g., overnight or federal holidays). Response time for technical support issues should be immediate.

35. Offeror’s Question:

SOW Section 5.0 Workstation Compatibility (Page 31) – Please confirm whether Department of Homeland Security (DHS) workstations are still operating on Windows 8, or if all systems have been upgraded.

Government’s Response:

All systems are using Windows 10 and 11. The contractor should have the ability to work with all versions of Windows that are actively supported by Microsoft.

36. Offeror’s Question:

SOW Section 5.0 Workstation Compatibility (Page 31) – Does this requirement include the installation of the SPLUNK monitoring agent?

CATS/COACA is an external system and its not within the CBP boundary, so SPLUNK is not applicable to CATS/COACA.

37. Offeror’s Question:

SOW Section 7.2 Ownership of Content (Page 15) – Please confirm whether all test content is owned by CBP. Are there any current or future plans to incorporate commercial off-the-shelf (COTS) assessments?

Government’s Response:

For the initial implementation, all test content is owned by CBP. The vendor should have the ability to handle off-the-shelf assessments.

38. Offeror’s Question:

SOW Section 7.8 Program Oversight (Page 18) – What level of federal oversight and involvement is anticipated in daily operations under this contract?

Government’s Response:

The contractor should have the ability to execute the contract with minimal daily oversight by the Government but may need to take direction as needed.

39. Offeror’s Question:

SOW Section 7.11.1 Reporting Requirements (Page 19) – Please provide examples of the types of annual, quarterly, and monthly data that must be submitted as part of the continuous monitoring requirements. Additionally, what specific data elements are expected to be delivered within the stated three-day timeframe?

Government’s Response:

System-level metrics to be monitored: FISMA Scorecard.

Establishing monthly scans. Reporting the security and privacy status of the system to the AO/CISO, providing monthly vulnerability scans and report findings to DHS, to comply with the Federal Information Security Management Act and Agency Privacy Management

Plan Of Action And Milestones (POA&M). The DHS POA&M process is a continuous cycle which entails identifying and documenting weaknesses and assigning responsibility for remediation. Quarterly milestones are developed for all POA&Ms that are open over 3 months, developing a timeline for resolution, and recording weaknesses and the remediation plan in the POA&M for DHS visibility.

Annual assessments and/or significant changes for monitoring

40. Offeror’s Question:

SOW Section 7.12.1.9.1 Monitoring and Data Capture (Page 22) – What are the requirements for 180-day monitoring and packet capture data retention?

Government’s Response:

Data is retained within the vendor’s system for the duration of the agency contract. At the end of the contract, data is dispositioned per the terms of the contract award.

CATS/COACA Application auditable events are updated as major functionality is added.

Updates to the vendor’s application currently follows a 6-week release cycle and any necessary changes to logging are updated as required.

41. Offeror’s Question:

Can we anticipate for the final RFP to be issued around mid Q4?

Government’s Response:

Notice ID 70B06C25R00000075 on SAM.gov specifically states:

“PLEASE NOTE: This is a draft solicitation. This notice and all documents contained herein are subject to revision(s) and are part of the full timeline scheduled for this notice.

Potential offerors are highly encouraged to begin their response to this notice as soon as it is released.”

The deadline for final submission of proposals in RFP 70B06C25R00000075 is corrected to read from 5PM EDT, Wednesday, July 10, 2025, to 5PM EDT, Wednesday, July 9, 2025.