A00001_Questions and Responses re RFP 70B06C20R0096_4June.pdf

Questions and Responses for RFP 70B06C20R0096

# Document Section Question 1 Attachment 1 – SOW 4.1.3 Would the independent third party be a contracted service that is currently used by the government or the internal auditing team?

CBP Response: If contractor is a FedRAMP provider, the Independent Assessors (3PAOs) that is retained by the vendor would conduct the assessment. In addition to the assessment, the government will conduct its own review of the SA package.

2 Attachment 1 – SOW 4.2.1 Can a use case for “allow test-takers to digitally highlight or underline text on the screen” be provided?

CBP Response: Reasoning tests often provide more information than is needed to reach a conclusion. Often applicants like to be able to highlight or underline the text that they believe to be important in determining the correct conclusion/answer. We would like the applicant to be able to highlight, underline or otherwise emphasize the text they believe to be important while they are working out which response option is correct.

3 Attachment 1 – SOW 4.10.1 Can a use case be provided for the “ability to hold scores at the Government’s direction”?

CBP Response: CBP makes every effort to ensure that our tests are scored by the vendor’s system with 100 percent accuracy. As a result, when we implement a new test or a new series of a test, we will verify the scores produced by the vendor’s system until we are satisfied that the system is accurately scoring all tests. Typically we will ask the vendor not to release test scores until we have completed the verification. This often requires the vendor to hold the scores for one business day or less while we confirm the accuracy of the system generated test scores.

Once we are satisfied that the system is scoring with 100 percent accuracy, the requirement to hold test scores will be released. Generally we like to verify that a certain number of accurate scores are produced with no deviation before the requirement to hold scores is lifted. This number is typically reached within a one to two week period but varies based on testing volume.

4 Attachment 1 – SOW 5 Will the Deliverables listed in the table also be exceptions to the 55-page limit for Volume I, Section I (some Deliverables are not listed specifically included as exceptions in the Solicitation under “Page Limitations”)?

CBP Response: Yes, these items (7, 12, 37, 40, 63 and 67) will be excluded from the 55 page count. These must be labeled as such and each page must provide a footer or header note indicating “SOW Section 5 proposal deliverable”.

Attachment 1 – SOW

7 (second bullet) Please define what materials are included as part of "turn over all materials". What is the scope of materials we should be expected to deliver (code, app logs, network logs, email, backups, internal documentation, tickets, etc.)?

CBP Response: This is at the government’s discretion at that time depending on what was produced. The scope of materials expected to be delivered should be within the scope of what is reasonable and customary.

6 Attachment 1 – SOW 7.6.2 Would the annual independent assessment and verification be rolled into the Continuous Monitoring process, or would that be a separate assessment?

CBP Response: The annual independent assessment and verification is part of the Continuous Monitoring process.

7 Attachment 1 – SOW 7.6.3 How would CBP define "on a periodic basis"?

CBP Response: That is at the government’s discretion.

8 Attachment 1 – SOW 7.8 For other federal Continuous Monitoring requirements, we provide this information with Nessus and Symantec scan submissions. Will this current process address section 7.8?

CBP Response: If the current process addresses all of the minimum processes as outlined, then that should be sufficient. Notification of changes will be provided as they occur. If the requirements change, the vendor will be expected to meet them.

7.8.2 What is meant by “near-real-time security status information to DHS SOC? Please provide more detail here. What is being asked? What is expected – SIEM data feed to DHS? Log/Syslog forwarding?

CBP Response: Company would have the ability to query and run reports on audit log information quickly in case DHS SOC requested it (a proper SIEM) and that the log sources would be current (almost real-time).

7.9.1.1 Are the SOC personnel subject to the suitability requirement listed in 7.0?

CBP Response: Yes, if the SOC personnel had access to CBP sensitive information.

7.9.1.1 Would an outsourced, offshore SOC be acceptable to the government to meet this requirement?

CBP Response: No, this would not be acceptable.

7.9.1.1 What would be the time frequency of "regular reviews”?

CBP Response: The frequency of the regular reviews would be established by DHS SOC after award.

7.9.1.1 Does “24x7x365” mean that CBP requires personnel to be watching the monitor/screen during that time, or that the monitoring system should send alerts and notifications to staff?

CBP Response: The monitoring system sending alerts and notifications to staff is acceptable.

14 Attachment 1 – SOW 7.10.2; 7.11.1;

7.12.1 7.14.1

What data is to be included in the status reports, such as the analysis of firewall logs in weekly status reports? Please provide more detail here. What is being asked? What is expected in a report?

CBP Response: This is at the government’s discretion.

15 Attachment 1 – SOW 4.1 Does the government require an integration between the automated assessment system and a staffing/applicant tracking system?

CBP Response: Yes we do require an integration between the automated assessment system and the ATS. If a direct integration is not possible (e.g., ATS vendor does not allow it) then CBP expects that an indirect integration or some other method for getting the data from the automated assessment system into the ATS may be necessary, and testing vendor is expected to participate in determining this solution as needed.

16 Attachment 1 – SOW 4.1 Does the government require that historical item, scale, and/or overall score data from existing assessments be migrated into the testing platform?

CBP Response: Historical and active items along with item statistics should be migrated into item banking software that is part of the testing platform. Also we need to be able to determine whether each applicant is eligible to test. Many of our applicant apply multiple times and if they have failed the test within the previous 6 months they are not eligible to retest. If they passed the test previously they can are eligible to retest if they choose.

17 Attachment 1 – SOW 4.2.1 Can the government provide Table 6 or clarify if there is a Table 6 in the solicitation?

CBP Response: Table 6 was removed from the Solicitation. There is no Table 6.

18 Attachment 2 – Pricing Tab T2 - Sec Auth/Reauth

CLIN 0001.1-001.5 has Option Years 1 through 4 prefilled as N/A. Will there not be a reauthorization required in Option Year 2 or 3 that would require funding?

CBP Response: There will be a reauthorization required by an outside firm in Option Year 2 and 3 and the Government will assume full responsibility of the funding for the Security Authorization/Reauthorization.

19 Attachment 5 – Technical Proposal Content Requirements

Factor 3 Can you confirm that a response is not expected for tabs 8 – 13, as there are no notes in the column “Information and materials to be provided by Offeror”?

CBP Response: This is the Past Performance information and a response is required. Please see the examples for requirements of Past Performance information Factor 4 – Past Performance The vendor should provide three examples of past work of similar size and scope. The work should be ongoing or have been completed in the last three years from the release date of this solicitation.

The past performance evaluators may also consider any data available in federal procurement databases such as the Contractor Performance Assessment Reporting System (CPARS), Federal Awardee Performance and Integrity Information System (FAPIIS) or the Past Performance Information Retrieval System (PPIRS), as well as any other available sources.

20 Attachment 5 – Technical

Requirements and Solicitation/Contract/Order 70B06C20R00000096

Section III.12 of

RFP

Evaluation Criteria on pages 47-48 of the RFP and Attachment 5: COACA Technical Proposal Content Requirements, we found that the Evaluation Criteria listed in the RFP under III.12 Evaluation Criteria, do not match many of the Evaluation Criteria listed in Attachment 5. For instance, Factor I - Technical Approach, Element 1 in the RFP lists the "Offeror's ability to offer a variety of test administration methods...," while In Attachment 5, Element 1 is listed as, "Programming customizable CAT algorithms." This mismatch between the RFP and the Attachment continues through all the Elements under Factor I. There is also a mismatch under Factor 3 - Past Performance. Element 4 is listed as "Business Relations" in the RFP, while in Attachment 5 it is "Cost Control." Finally, the Element 5 listing under Factor 3 does not match, and while there is an Element 6 in the Attachment, "Customer Service," there is no Element 6 in the RFP. Should vendors follow Attachment 5 as their guide in preparing their proposals or follow the instructions under III.12 Evaluation Criteria on pages 47-48 of the RFP? Will DHS make either a revision of the RFP or Attachment 5 and post an amended version?

CBP Response: Attachment 5 of the RFP is incorrect and does indeed have some mismatched evaluation criteria regarding Past Performance and will be changed in an amendment to match the Instructions and Evaluation Criteria in the RFQ Section III.12. Section III.12 is accurate regarding Factor 3.

21 Solicitation/Contract/Order 70B06C20R00000096

II.25 Security Procedures A. Controls #5

The statement reads "“All contractor employees". Will this be limited to just the CPM or just those assigned to the work on the CBP project/contract?

CBP Response: This clause applies to all Contractor employees.

22 Solicitation/Contract/Order 70B06C20R00000096

II.25 Security Procedures A. Controls #6

Will employees assigned to the CBP contract require a BI or just the Contractor Project Manager

(CPM)?

CBP Response: This reference section falls under the main clause which applies to all Contractor employees. Anyone who has access to CBP sensitive information will require a favorable BI.

23 Solicitation/Contract/Order

II.32 TELEWORK

b. Requirements

Can the contractor's personnel who will work remotely be approved by only the CPM without CBP approval/acknowledgement?

CBP Response: Telework arrangements will be mutually agreed to on a case-by-case basis after contract award.

24 Solicitation/Contract/Order 70B06C20R00000096

C. Information Technology (IT) and Security

Can the contractor employees use the vendor’s supplied PCs (i.e., company owned) for telework and not GFE?

CBP Response: Yes, if the equipment is configured with the same or more stringent security requirements.

25 Solicitation/Contract/Order

Section III.12 Regarding the Evaluation Criteria on pages 47-48, are there any weighting factors associated with them? Will some evaluation criteria be weighted more than others in making the contract award decision?

CBP Response: Factors 1, 2, and 3 are equally important and when combined are significantly more important than Factor (see Section III.13 RELATIVE IMPORTANCE OF EVALUATION

FACTORS)

26 Solicitation/Contract/Order 70B06C20R00000096

Section III.12 Factor 1 Technical Approach, Element 1, states “Offeror’s ability to offer a variety of test administration methods including at a network of brick-and-mortar, proctored test centers in the United States, Canada, and internationally, using remote proctoring technology, and as unproctored assessments”. However, in attachment 5 of the solicitation

+COACA+TECHNICAL+PROPOSAL+CONTENT+REQUIREMENTS+70B06C20R00000096_R1)

Factor 1, Technical Approach, Element 1: The evaluation factor is described as Programming customizable CAT algorithms.

Can the government clarify which language is the appropriate evaluation factor for the COACA solicitation?

CBP Response: Attachment 5 of the RFP is correct and does indeed have some mismatched evaluation criteria regarding Factor 1 and will be changed in an amendment so that Instructions and Evaluation Criteria in the RFQ Section III.12. match what is in attachment 5

27 Solicitation/Contract/Order

Section III.12 Factor 1 Technical Approach, Element 2, states “Capability to implement appropriate controls to verify applicant identity and ensure test security across administration methods.” However, in attachment 5 of the solicitation

+COACA+TECHNICAL+PROPOSAL+CONTENT+REQUIREMENTS+70B06C20R00000096_R1)

Factor 1, Technical Approach, Element 2: The evaluation factor is described as System Capability.

Can the government clarify which language is the appropriate evaluation factor for the COACA solicitation?

CBP Response: CBP Response: Attachment 5 of the RFP is correct and does indeed have some mismatched evaluation criteria regarding Factor 1 and will be changed in an amendment so that Instructions and Evaluation Criteria in the RFQ Section III.12. match what is in attachment 5

28 Solicitation/Contract/Order 70B06C20R00000096

Section III.12 Factor 1 Technical Approach, Element 3, states “Functional user interface that is organized and intuitive for applicants and administrative personnel.”.

However, in attachment 5 of the solicitation

+COACA+TECHNICAL+PROPOSAL+CONTENT+REQUIREMENTS+70B06C20R00000096_R1)

Factor 1, Technical Approach, Element 3: The evaluation factor is described as Testing Site Network. Can the government clarify which language is the appropriate evaluation factor for the COACA solicitation?

CBP Response: CBP Response: Attachment 5 of the RFP is correct and does indeed have some

Instructions and Evaluation Criteria in the RFQ Section III.12. match what is in attachment 5

29 Solicitation/Contract/Order

Section III.12 Factor 1 Technical Approach, Element 4, states “Offeror’s ability to meet the web service and security, compliance and privacy requirements.”.

However, in attachment 5 of the solicitation

+COACA+TECHNICAL+PROPOSAL+CONTENT+REQUIREMENTS+70B06C20R00000096_R1)

Factor 1, Technical Approach, Element 4: The evaluation factor is described as Test Administration Methods.

Can the government clarify which language is the appropriate evaluation factor for the COACA solicitation?

CBP Response: CBP Response: Attachment 5 of the RFP is correct and does indeed have some

Instructions and Evaluation Criteria in the RFQ Section III.12. match what is in attachment 5

30 Attachment 5 – Technical

Requirements

Factor 1 Element 1

The evaluation factor is described as Programming customizable CAT algorithms – The government requests Provide samples/illustrative mock-ups for:

• At least 2 examples of previous customized CAT algorithms programmed

• Example CAT algorithms should demonstrate ability to program adaptive algorithms for custom CATs

• Reports or other documentation that describes the technical aspects of CAT engines that have been previously programmed for clients

Is the material requested in this single factor to be provided within the existing 55 limit page count?

CBP Response: Yes. This is not excluded.

31 Attachment 5 – Technical

Requirements

Factor 1 Element 1

The evaluation factor is described as Programming customizable CAT algorithms – The government requests Provide samples/illustrative mock-ups for:

• At least 2 examples of previous customized CAT algorithms programmed

• Example CAT algorithms should demonstrate ability to program adaptive algorithms for custom CATs

Reports or other documentation that describes the technical aspects of CAT engines that have been previously programmed for clients.

All reports that have been developed for existing or prior clients are the property of those customers and are not available for distribution. Will the government remove this requirement from this evaluation factor?

CBP Response: No we will not remove this requirement, reports and descriptions do not need to include proprietary information, nor do they have to include technical reports, but detailed descriptions of the technical aspects and processes used when programming CAT engines is requested. If you would like to include technical reports you may redact them.

32 Attachment 5 – Technical

Requirements

Factor 1 Element 1

The evaluation factor is described as Programming customizable CAT algorithms – The government requests Provide samples/illustrative mock-ups for:

• At least 2 examples of previous customized CAT algorithms programmed

• Example CAT algorithms should demonstrate ability to program adaptive algorithms for custom CATs Reports or other documentation that describes the technical aspects of CAT engines that have been previously programmed for clients.

Please clarify what is meant by a CAT algorithm?

Is the government asking for specific code level language (e.g., python, JAVA, SAS, SPSS)?

CBP Response: No the government is not requesting specific code level language. Detailed descriptions of the process used to program custom CAT algorithms is requested.

33 Attachment 5 – Technical

Requirements

Factor 1 Element 1

The evaluation factor is described as Programming customizable CAT algorithms. The government asks “ for Custom developed CAT algorithms including but not limited to:”

Can the government define the additional scope required in the language “but not limited to”?

CBP Response: The “but not limited to” language refers to any other custom CAT algorithm capabilities outside of those listed in Attachment 5 Factor 1: Element 1 with which you have experience.

34 Attachment 5 – Technical

Requirements

Factor 1 Element 2

The evaluation factor is described as System Capability and in bullet 4 requests “a capability to maintain supplemental materials”

Can the government identify what is included in supplemental materials

CBP Response: Ability to maintain supplemental materials refers to the ability of the system to maintain supplemental test materials such as simulated emails, memos, photos, video with sound, and other documents. See section 4.2 in Attachment 1, Statement of Work

35 Attachment 5 – Technical

Requirements

Factor 1 Element 2

The evaluation factor is described as System Capability and in bullet 2 requests “administering different tests within a battery to applicants”

Can the government clarify what different tests means? Normally a battery is administered to all applicants. Does the government want two batteries or manipulation of tests within a battery for different groups of applicants?

CBP Response: A test battery refers to a set of tests that are group together and usually administered to applicants at one time. The requirement refers to the systems capability to combine multiple different tests into a single battery of administration. The combination of tests in the battery will vary by the occupation to which the applicant is applying. Some tests in the battery may be hurdled and will require applicants to pass be eligible to complete the remainder of the test battery.

36 Attachment 5 – Technical

Requirements

Factor 1 Element 2

The evaluation factor is described as System Capability. The narrative calls for the submission of “Proctor Manuals” Applicant Instructions and Proctor Training Materials”

Can the government indicate that The narrative calls for the submission of “Proctor Manuals” and Proctor Training Materials” are within or outside page count?

CBP Response: Outside. See response to Question 4. The Proctor Manuals are excluded from the 55 page count. These must be labeled as such whereby each page must provide a footer or header note indicating “SOW Section 5 proposal deliverable”.

37 Attachment 5 – Technical

Requirements

Factor 1 Element 3

The evaluation factor is described as Testing Site Network. This element calls for

•Configuration of static test centers; address how the set-up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

• Configuration of mobile test centers; address how the set-up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

• Testing center protocols (e.g., procedures for documenting administration protocol, unusual incident reporting procedures, emergency evacuations, power failures, computer failures, internet failures)

• Testing center protocols (e.g., procedures for suspected and confirmed cheating and ADA accommodations)

Can the government confirm that

• Configuration of static test centers; address how the set-up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

• Configuration of mobile test centers; address how the set-up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

• Testing center protocols (e.g., procedures for documenting administration protocol, unusual incident reporting procedures, emergency evacuations, power failures, computer failures, internet failures)

• Testing center protocols (e.g., procedures for suspected and confirmed cheating and ADA accommodations)

Are included in the 55-page limits of the solicitation?

CBP Response: This information will be considered inside the 55-page count.

38 Attachment 5 – Technical

Requirements

Factor 1 Element 4

The evaluation factor is described as Test Administration Methods.

In Element 4 there are requests for proctor test protocols and proctor training plan – these are extensive document – in the inclusion of these within the 55 page count limit?

CBP Response: Outside the page count. Excluded from the 55-page limitation. These must be labeled as such whereby each page must provide a footer or header note indicating “SOW Section 5 proposal deliverable”.

39 Attachment 5 – Technical

Requirements

What is the preferred format for these “Illustrative Mock Ups”?

CBP Response: Regarding file format types, PDF files are preferred, however the content within the files the government has no particular format request

40 Attachment 5 – Technical

Requirements

Are these “Illustrative Mock Ups” included in the 55 page limit?

CBP Response: No, these are excluded. These must be labeled as such whereby each page must provide a footer or header note indicating “SOW Section 5 proposal deliverable”.

41 Given the CONUS wide impact of this contract and the potentials for cost reimbursable elements does the government require the certification of compliant accounting system under FAR Subpart 16.3?

CBP Response: There are no cost reimbursable elements. Please refer to Section II.19 CONTRACT TYPE (OCT 2008) where it explicitly states the Government anticipates a single award of an IDIQ type contract with Firm Fixed-Price Task Orders.

42 Will the government extend the due date to June 30 2020?

CBP Response: Will extend the due date to Monday June 15th.