Responses to Vendor Questions on DRAFT RFP.pdf

Response to Vendor Comments on Draft RFP

Question

Number Reference RFP Section

Paragraph

No.

Page No.(s) Questions/Comments Answers

1 DRAFT RFP

70RTAC22R00000010.pdf, B.4

Table 1-3 Can the CLIN structure be simplified to two CLINs for Public Asset and Private Asset in order to allow alternative approaches to "Crowdsourced Vulnerability Assessment Services

(CVAS)" that minimize or eliminate the risk of operating from a bounty pool?

The "Bounty Pool" should be a sub-CLIN of a Public Asset (CLIN 0002) or Private Asset (CLIN

0003) to allow for offerors of fixed-priced CVAS offerings in addition to the "Administrative

Cost + Bounty Pool" priced offerings. This best enables the government to fairly evaluate the total price of all proposals.

When DHS conducted the test in support of the Secure Technology Act at DHS OCIO the engagement was procured with a Firm Fixed Price. This is an all inclusive price including PM, platform, and bounty payments. DHS OCIO, other DHS components, and numerous Federal agencies prefer this approach so the government knows the exact price an engagement is going to cost and vendors are not coming back to procurement while testing is going on to ask for more funding because of the number/value of vulnerabilities that have been found exceeds the original bounty pool.

It is highly recommended that the government Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test, simplifying the CLIN structure and allowing bidders flexibility to structure their price models in line with their business models.

This approach provides the government access to the widest possible offering, while it creates a fair and open playing field for all offerors. Contractors should provide a list of

SKUs/schedule with ceiling prices as part of their submission. This best enables the government to fairly evaluate the total price of all proposals.

A separate Fixed-Unit-Price CLIN for Bounty Payouts would allow the Government to pay the bounty amount based on the number/severity of vulnerabilities discovered. At the task order level, the Bounty Payout CLIN can be either Fixed-Unit-Price or Firm-

Fixed-Price. Administrative Services would be inclusive of all costs incurred for the

CVAS, except for the bounty payouts. If needed, Components are allowed to utilize task order unique CLIN(s) as described in the final RFP Section B.5.

2 DRAFT RFP

70RTAC22R00000010.pdf, Section B.4

Table 7-9 Can the CLIN structure be simplified to two CLINs for Public Asset and Private Asset in order to allow alternative approaches to "Crowdsourced Vulnerability Assessment Services

(CVAS)" that minimize or eliminate the risk of operating from a bounty pool?

The "Bounty Pool" should be a sub-CLIN of a Public Asset (CLIN 0002) or Private Asset (CLIN

0003) to allow for offerors of fixed-priced CVAS offerings in addition to the "Administrative

Cost + Bounty Pool" priced offerings. This best enables the government to fairly evaluate the total price of all proposals.

When DHS conducted the test in support of the Secure Technology Act at DHS OCIO the engagement was procured with a Firm Fixed Price. This is an all inclusive price including PM, platform, and bounty payments. DHS OCIO, other DHS components, and numerous Federal agencies prefer this approach so the government knows the exact price an engagement is going to cost and vendors are not coming back to procurement while testing is going on to ask for more funding because of the number/value of vulnerabilities that have been found exceeds the original bounty pool.

It is highly recommended that the government Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test, simplifying the CLIN structure and allowing bidders flexibility to structure their price models in line with their business models.

This approach provides the government access to the widest possible offering, while it creates a fair and open playing field for all offerors. Contractors should provide a list of

SKUs/schedule with ceiling prices as part of their submission. This best enables the government to fairly evaluate the total price of all proposals.

Please see the Government's response to Question #1.

3 DRAFT RFP

70RTAC22R00000010.pdf, Section B.4

Table 7-9 Can the government provide a definition and reference for the term "Fixed-Unit-Price" as used in CLINs 0001, 1001, 2001, 3001, and 4001? Is this Cost-Reimbursable or Cost-Plus or some other type of pricing arrangement under FAR Part 16?

For a fixed-unit price CLIN (i.e. CLINs x001), payments are based on units of output (i.e.

number/severity of vulnerabilities discovered) and will be made up to the ceiling amount established.

4 DRAFT RFP

70RTAC22R00000010.pdf, Section B.5

1 10 "price and non-price factors considered" - What are the non-price factors that DHS will consider?

Non-price factors are all evaluation factors listed in Section M except for price.

5 DRAFT RFP

70RTAC22R00000010.pdf, Section B.5

2 10 Can the government clarify if this paragraph applies to all CLINs or only CLINs x001, the

(Fixed-Unit-Price) CLINs as cost overruns/underruns are inherent to Firm-Fixed Price CLINs?

B.5 has been removed.

6 DRAFT RFP

70RTAC22R00000010

Section K.1

1 55 Is the government committed to only using NAICs Code 541519 and won’t that impact competitive pool

The Government is committed to using NAICS Code 541519.

7 DRAFT RFP

70RTAC22R00000010.pdf, Section L.4.1.2

Item 4 62 Can the government replace "DUNS Number " with "SAM Unique Entity ID "? Yes, "DUNS number" has changed to "Unique Entity Identifier."

8 DRAFT RFP

70RTAC22R00000010.pdf, Section L multiple 63 Can the government correct the section numbering? Yes, the section numbering has been corrected.

9 DRAFT RFP

70RTAC22R00000010.pdf, Section L (5.1.1.6)

1 63 Can the government expand the requirements for the self-certification for Factor 4 to include the ability to comply with all requirements outlined in Section I.3 (Safeguarding of

Sensitive Information)?

Failure to properly protect exploitable vulnerability data on DHS systems could could adversely affect the national or homeland security interest, the conduct of

Federal programs, or the privacy to which individuals are entitled .

In accordance with Factor 4, all DHS sensitive information captured and stored by the

Offeror, to include vulnerabilities in Department systems, must be stored and processed within the U.S. in accordance with PWS Section 2.4.10 and DHS Special

Clause “Safeguarding of Sensitive Information,” by completing and signing the Offeror

Certification of Compliance with HSAR 3052.204-71 and Data Processing and Storage

Requirement form (RFP Attachment J.5).

10 DRAFT RFP

70RTAC22R00000010.pdf, Section L4.3.1

3 66 Can the government clarify the basis for pricing for the IDIQ tab of the Pricing Template as no description of the scope or type of assets to be tested is provided in the Draft RFP, PWS or Pricing Template? Is this a not-to-exceed or ceiling price for each sub-CLIN? Or will further scope be provided to enable complete and accurate pricing?

The Offeror is now required to propose a percentage (%) (vs. price ($)) for the

Administrative Services CLINs, compared to the total funding for Bounty Payouts.

11 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

2nd bullet 3 Researchers conducting auditable crowdsourced vulnerability discovery and disclosure activities through a secure portal on the contractor’s platform against a variety of sensitive but Internet-connected assets, as well as non-Internet connected assets. What does DHS mean by "auditable crowdsourced vulnerability discovery"? What makes vulnerability discovery auditable? What does DHS want to know about the vulnerability discovery process?

PWS has been updated. Auditable in this sense means the vendor knows the identity of the researcher and a given action (submission or exploitation attempt) is attributable back to a researcher by the vendor if given log data by DHS.

12 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

Private

Assets:

3 The platform must have a secure portal capable of continuous monitoring and auditing of researcher activities. What kind of auditing is expected to be performed? Would DHS gain additional benefit from a per-researcher or master killswitch-type feature?

See answer to question #11 for audit info. While a "master killswitch" feature would be useful for a worst case scenario, it would be advantageous for DHS to be able to have the ability to request access be suspended on a per-researcher basis in cases where scope violations need to be investigated as not to shut down the whole event.

13 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

3rd bullet 3 Testing content hosted on contractor’s infrastructure, through its secure platform in a controlled environment or repository. A contractor-hosted environment seems to conflict with the “Private Assets” title of this section. What kind of content would the contractor be expected to host?

PWS has been updated. The intent is to cover how offline/non-internet systems will be assessed and initially the expectation was to have the contractor stand up a version of their platform in that space. However, logging vulnerability finding data in an accessible/importable format such as CSV, JSON, or XML is acceptable.

14 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

4th bullet 3 Testing content hosted by DHS through a secure contractor portal where Internet Protocol

(IP) addresses are logged and/or directly provisioned by the contractor and other data, potentially including keystrokes, are captured. Is DHS referring to a full tunnel VPN solution or an actual endpoint keylogging solution? This may make the program unattractive to researchers. Is this a hardened requirement? Can you describe the reasons for this requirement?

PWS has been updated. The intent is for the vendor to have the ability to inspect traffic going through their VPN tunnel in order to provide auditable and attributable logging.

15 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0

Private Assets

5th bullet 3 Can the government define "...secure portal... "? For example: "...a secure, FedRAMP

Moderate or higher impact level, portal.. ".

The data on DHS vulnerabilities and how to exploit them is highly sensitive and should be protected at a level commensurate with this level of sensitivity. Requiring the contractor's portal to be at a minimum at the FedRAMP Moderate impact level would ensure this data is appropriately protected in accordance with SAFEGUARDING OF SENSITIVE INFORMATION

(MAR 2015) Section (e).

In order to maintain maximum flexibility on the range of scope given the sensitivity of assets, the task orders will detail the appropriate safeguards required based on the sensitivity of assets in scope for a given event.

16 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

6th bullet 3 Scoped Time-boxed and/or continuous crowdsourced efforts ranging from 25 to 200 (or less or more) total participants depending on the needs of the government. How does DHS define "continuous"? Is it one calendar year?

PWS has been updated. Continuous would be for 1 calendar year.

17 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

7th bullet 3 All activities under each task order will include commercial background check and may include geolocation verification requirements from the government as a condition of researcher participation. These checks shall be the contractor responsibility. Does geolocation verification in this context refer to the physical location of the researcher or the country of the researcher's citizenship? We have found that physical location assurance can be trivially spoofed and not provide much value. Can you help us understand what the business requirement is, and what is the value that is being derived?

Citizenship check and/or geolocation check would be required. Geolocation verification is still a check the Government is required to perform to meet our due diligence requirements for conducting bounty activities.

18 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Private Assets

7th bullet 3 Can the government change "convictions " to "felony convictions "?

This would provide clarity, as misdemeanor convictions would normally not preclude inclusion as a researcher.

PWS has been updated to remove "The contractor shall not include any researcher who turns up having convictions in a background check in any aspect of the bounty program."

19 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Public Assets

4th bullet 4 A robust public affairs and community outreach capability to conduct public outreach and sensitive coordination with researchers. Can you describe what you mean by "sensitive coordination"?

PWS has been updated to describe sensitive coordination.

20 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Overlapping Activities

2nd bullet 4 Provide comprehensive vulnerability triaging, validation, and prioritization within 48 hours of submission, and reporting to the DHS System Owner to ensure it can patch the vulnerability as soon as feasible; To what extent, if any, is the contractor responsible for assisting in the identification of DHS system owners?

None. After award of the Task Order, DHS will identify at least one point of contact for each system/application in the bounty effort and provide this information to the vendor. The vendor will be expected to ensure, in events where there are multiple systems/applications in scope, the correct point of contact is assigned to a submission once validated. In cases where a single system or application has multiple points of contact, DHS will designate a primary point of contact.

21 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Overlapping Activities

4th bullet 4 Assist the DHS System Owner in identifying and developing mitigation approaches for discovered vulnerabilities; To what extent would the contractor be expected to develop mitigation approaches?

The mitigation approach requirement would be generic to the vulnerability found instead of a step-by-step, application specific walkthrough. For a basic example, for a default credential finding, mitigation approach would be met by suggesting system owner disable the account, it would be on the system owner to understand how to do that in their environment/application. If the vendor is in compliance with 2.2.11 then this requirement is met.

22 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Overlapping Activities

5th bullet 4 Ability to provide a means to easily export vulnerability reports to other systems (JIRA, etc.)

and synchronize vulnerability remediation statuses between multiple systems through common format (deliverable format will be identified at each Task Order); Does DHS want direct integration with your systems or via an export? Or done via other means?

PWS has been updated. This can be done by either process as described in the PWS. At a minimum the vendor must be able to export vulnerability findings that include the required elements outlined in 2.2.11 in a common format (i.e. CSV, XML, or JSON)

23 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Overlapping Activities

6th bullet 4 Conduct all management and coordination with the researcher community, and project management and coordination with DHS Remediation Team; What project management duties will the contractor be expected to provide over DHS resources?

This bullet has been updated to read: "Conduct all management and coordination with the researcher community, and coordination with DHS Remediation Team;"

24 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Overlapping Activities

7th bullet 4 Can the government clarify if the "report " identified in the second sentence is intended to be a report of validated findings or a report of all findings validated and rejected?

If the latter, can the government remove the requirement for "a step-by-step process for replicating the vulnerability. ", as rejected vulnerabilities are often rejected as they can't be replicated or are duplicates of submitted vulnerabilities?

PWS has been updated to read: "At a minimum, validated reports shall contain…" because, as correctly pointed out, reports may be rejected due to lack of reproduction steps.

25 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Existing Security

Researcher Community

1 5 Can the government clarify the number of "active researchers " required?

This paragraph seems to require 1,000 active researchers, but Attachment 3 list the requirement as 1,000 researchers and 150 active.

The documents have been updated to be consistent.

26 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

2nd bullet 5 The capability to actively manage researchers on the assessment, for example, the ability to immediately remove or disable a researcher’s account. Under what circumstance would a contractor be required to disable a researcher account?

PWS has been updated. "A researcher's account" has changed to "a researcher's access to all DHS bounty events and information."

27 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

6th bullet 5 The capability to facilitate effective communication between the triage team and researchers and between the triage team and Government remediators. This may include corresponding, separately, with multiple teams. What method of correspondence will contractors be required to use to maintain effective communication with Government remediations?

After the award of a Task Order, vendors will have access to DHS remediation team email accounts and will be able to communicate through the vendor's platform. The government person(s) executing the bounty event will also provide the vendor team their official phone number so they can be reached quickly in the event of an emergency.

28 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

10th bullet 6 The capability to function as a secure portal that is capable of continuous monitoring and auditing of researcher activities, such as those logs collected through simple proxy logging, up to full Packet Capture (PCAP) as identified at the task order level. For what purpose does

DHS need PCAP logs? How often would a PCAP log be required to be procured? With what frequency would PCAP logs require delivery in a continuous program (24x7x365)?

PCAP would be requested on a case-by-case basis to support/supplement situations where investigation of out-of-scope actions by a researcher have occurred.

29 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

1 6 DHS requires the ability to continuously monitor individual researcher activity for the duration of the live-assessment, and the ability to audit researcher activity post-assessment.

At a minimum, DHS requires the ability to know which individual researchers are accessing

(or accessed) specific parts of an assessment at specific points in time. Further monitoring/auditing requirements may be indicated at the task order level. What tools does

DHS expect to use to correlate researcher activity with the contractor? Would an integration with a SIEM tool or similar prove helpful to DHS?

The PWS has been updated to read: At a minimum, DHS requires the contractor to possess the ability to continuously monitor individual researcher activity for the duration of the live-assessment, and the ability to audit researcher activity post-assessment. At a minimum, DHS requires the contractor to possess the ability to know which individual researchers are accessing (or accessed) specific parts of an assessment at specific points in time. DHS requires the contractor to furnish this information upon request if out-of-scope and/or malicious activity is suspected.

30 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

4 6 The government expects time boxed challenges and continuous challenges. Can the government please define the challenge to include the type and number of asset(s) for the time-boxed and contiguous challenges?

There is currently not sufficient information to provide an accurate cost model or ceiling for the time-boxed or continuous challenge sub-CLINs.

The type and number of asset(s) for the fixed-duration and continuous bounty challenges will be defined at the task order level. Please note that travel is not required for the fixed-duration and continuous bounty challenges.

31 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 2.0, Vulnerability Discovery and

Disclosure Platform

5 6 The government expects Live Events of 1 to 4 days. Can the government please define the

Live Events better to include the location(s), number of on-site researchers required, type of asset(s), and number of assets?

There is currently not sufficient information to provide an accurate cost model or ceiling for the live event sub-CLINs.

PWS has been updated. The type and number of asset(s) for the live event(s) will be defined at the task order level.

32 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.1.2

1 6 Conduct criminal background checks on all researchers and geolocation verification checks, if requested, on all researchers before granting them access to any DHS information. The contractor shall not include any researcher who turns up as having convictions in a background check in any aspect of the bounty program. The contractor shall not include researchers geolocated through the contractor’s geolocation verification process to countries designated by DHS at the Task Order level in any aspect of the bounty program.

The contractor must be willing to provide proof of completion of these checks for each researcher if requested by the government. What does DHS expect to be the predominant preferred geolocation of researchers?

If there is a preferred geolocation/citizenship requirement, it will be defined at the Task

Order level. For all bounty events, there will be geolocation/citizenship restrictions in place based on current legal, policy, and geopolitical considerations at the time the

Task Order award. Very few, if any, events will contain a preference for a predominant location.

33 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

1 7 Can the government change "convictions " to "felony convictions "?

This would provide clarity, as misdemeanor convictions would normally not preclude inclusion as a researcher.

Please see the Government's Response to Question #18.

34 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.1.3

1 7 Work with the DHS Bounty Team, System Owner, and other Tech Stakeholders, to develop the asset scope of the challenge and complete a pre-assessment of the intended assets.

What does a “pre-assessment” of the intended assets entail?

PWS has been updated. The vendor's role in pre-assessment would be to assist in the determination of suitability of a particular asset to take part in a given bounty effort.

35 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.1.5

1 7 Can the government remove this paragraph or reword to allow alternative approaches to security researchers "crowd" management?

Researcher and bounty payments and the associated risk should be managed by the contractor as a commercial business best practice. The government should receive testing and vulnerabilities under the firm fixed price cost provided in the contractor's proposal without variation for quantity or types of vulnerabilities or numbers of researchers (beyond a reasonable minimum) engaged on a specific assessment/test.

When DHS conducted the test in support of the Secure Technology Act at DHS OCIO the engagement was procured with a Firm Fixed Price. This is an all inclusive price including PM, platform, and bounty payments. DHS OCIO, other DHS components, and numerous Federal agencies prefer this approach so the government knows the exact price an engagement is going to cost and vendors are not coming back to procurement while testing is going on to ask for more funding because of the number of vulnerabilities that have been found exceeds the bounty pool.

It is highly recommended that the government Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test. Contractors should provide a list of

SKUs/schedule as part of their submission. This best enables the government to fairly evaluate the total price of all proposals.

Please see the Government's response to Question #1.

36 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.2.7

1 8 Complement researcher efforts with automated testing tools for source code analysis, host and application scanning, and vulnerability analysis, if applicable. To what extent is the contractor expected to supplement researcher efforts with automated tools? Is this for the researcher or DHS purposes?

PWS has been updated to clarify this as "for example" not proscribing specific technologies. The vendor and/or researcher may use automated tools if available to assess systems within scope to provide more comprehensive assessment services.

These automated tools should only supplement and not replace manual assessments expected to be performed by researchers to fulfill the TO.

37 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.3.6

1 9 Are these awards to be funded from the bounty pool CLINs, or are they to be included in the contractor price for the Administrative Costs CLINs?

Awards are to be funded from the Administrative Services CLINs.

38 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.4.6

1 10 Notify DHS within 12 hours if a researcher violates the rules of engagement restrictions and suspend researcher’s access to DHS bounty programs pending DHS response. Contractor will be required to submit additional information requested about such action. Through what channel are ROE violations expected to be reported to DHS?

Email and phone call to designated DHS contact(s) provided at the TO level.

39 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

2.4.9 and 2.4.10

1/1 10 Can the government delete section 2.4.10 and simply require all portal access to require

MFA?

The data on DHS vulnerabilities and how to exploit them is highly sensitive and should be protected at a level commensurate with this level of sensitivity. Requiring the contractor's portal to be at a minimum at the FedRAMP Moderate impact level would ensure this data is appropriately protected in accordance with SAFEGUARDING OF SENSITIVE INFORMATION

(MAR 2015) Section (e).

PWS has been updated. See updated Section 2.4.9. Section 2.4.10 that was in the draft

PWS has been removed. MFA will be required globally for anyone with access to DHS vulnerability information.

40 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 4.5

1 12 Will the government be providing further details on the location and size of teams for the

Live Events sub-CLINS in order to allow the contractors to estimate travel costs?

If not, can the government revert travel costs to cost-reimbursable?

Please see the Government's response to Question #31.

41 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section 4.9

1 14 Can the government delete the phrase "of labor hours by labor category, all direct costs"?

As a FFP contract, T&M breakdowns are not applicable.

Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test. Contractors should provide a list of SKUs/schedule as part of their submission.

"of labor hours by labor category, all direct costs" has been removed. See answer to number 1 for pricing related response.

42 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

10.0

Table 1, Row

22 Can the government reword the Performance Standard from "The contractor coordinates with researchers and the government to ensure manages the compensation, both financial and non-financial, to the researchers. " to

"The contractor coordinates with researchers to ensure compensation, both financial and non-financial, to the researchers. "?

Researcher and bounty payments and associated risks should be managed by the contractor as a commercial business best practice. The government should receive testing and vulnerabilities under the firm fixed price cost provided in the contractor's proposal without variation for quantity or types of vulnerabilities or numbers of researchers (beyond a reasonable minimum) engaged on a specific assessment/test.

Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test. Contractors should provide a list of SKUs/schedule as part of their submission.

No change has been made to PWS Section 10.0. Please see the Government's response to Question #1 for contract type questions.

43 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

10.0

Table 1, Row

22 Can the government replace the Acceptable Quality Level replace the sentence "Bounty distribution should match the metric set forth by the Government. " to "Bounty distribution should follow commercial business best practices."?

Researcher and bounty payments and the associated risk should be managed by the contractor as a commercial business best practice. The government should receive testing and vulnerabilities under the firm fixed price cost provided in the contractor's proposal without variation for quantity or types of vulnerabilities or numbers of researchers (beyond a reasonable minimum) engaged on a specific assessment/test.

Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test. Contractors should provide a list of SKUs/schedule as part of their submission.

No change has been made to this sentence. Please see the Government's response to

Question #1 for contract type questions.

44 DRAFT RFP

70RTAC22R00000010

Attachment 1 -

Performance Work

Statement.pdf, Section

10.0

Table 1, Row

22 Can the government replace the Method of Compliance from "The COR will review the pace of bounties being distributed on a continuous basis. " to "The COR will monitor, through the contractor's portal and reports, the number of researchers and hours on target of researchers to ensure the contractor is maximizing the finding of critical and relevant vulnerabilities. "?

Researcher and bounty payments and the associated risk should be managed by the contractor as a commercial business best practice. The government should receive testing and vulnerabilities under the firm fixed price cost provided in the contractor's proposal without variation for quantity or types of vulnerabilities or numbers of researchers (beyond a reasonable minimum) engaged on a specific assessment/test.

Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test. Contractors should provide a list of SKUs/schedule as part of their submission.

No change has been made to this sentence. The intent of this contract is to discover vulnerabilities in DHS systems and pay those who discover them, thus those vulnerability findings and associated bounty payments must be transparent to the government to assess performance. If the government were to accept this change, it could allow for researchers to execute methods of evaluation that would never turn up a vulnerability, yet appear to be effective when represented by a "number of researchers and hours of work" metric.

45 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, IDIQ Pricing

Tab

Price Column 1 Can the government clarify the basis for pricing for the IDIQ tab of the Pricing Template as no description of the scope or type of assets to be tested is provided in the Draft RFP, PWS or Pricing Template? Is this a not-to-exceed or ceiling price for each sub-CLIN? Or will further scope be provided to enable complete and accurate pricing?

Please see the Government's response to Question #10.

46 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, IDIQ Pricing

Tab

CLINs 1 Can the CLIN structure be simplified to two CLINs for Public Asset and Private Asset in order to allow alternative approaches to "Crowdsourced Vulnerability Assessment Services

(CVAS)" that minimize or eliminate the risk of operating from a bounty pool?

The "Bounty Pool" should be a sub-CLIN of a Public Asset (CLIN 0002) or Private Asset (CLIN

0003) to allow for offerors of fixed-priced CVAS offerings in addition to the "Administrative

Cost + Bounty Pool" priced offerings. This best enables the government to fairly evaluate the total price of all proposals.

When DHS conducted the test in support of the Secure Technology Act at DHS OCIO the engagement was procured with a Firm Fixed Price. This is an all inclusive price including PM, platform, and bounty payments. DHS OCIO, other DHS components, and numerous Federal agencies prefer this approach so the government knows the exact price an engagement is going to cost and vendors are not coming back to procurement while testing is going on to ask for more funding because of the number/value of vulnerabilities that have been found exceeds the original bounty pool.

It is highly recommended that the government Pricing should be fixed based on the type of test and the type and quantity of asset(s) under test, simplifying the CLIN structure and allowing bidders flexibility to structure their price models in line with their business models.

This approach provides the government access to the widest possible offering, while it creates a fair and open playing field for all offerors. Contractors should provide a list of

SKUs/schedule with ceiling prices as part of their submission. This best enables the government to fairly evaluate the total price of all proposals.

Please see the Government's response to Question #1.

47 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, IDIQ Pricing

Tab

IDIQ Pricing IDIQ Pricing DHS has scoped 6 time boxed and 2 continuous events in the draft RFP, both with scopes of unknown size. Is there any opportunity for contractors to present engagements of varying size? The pricing template appears to not acknowledge size variants and is focused on a

"one size fits all" approach.

Yes, the attached scenarios will allow for contractors to present pricing on engagements of varying size.

48 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Sample

Order Pricing Tab

CLIN 0001 1 Can the government please clarify CLIN 0001 Bounty Payouts? There is a Fixed Unit Price of

$125,000. Does that mean that a fixed rate of $125,000 of bounties are paid regardless of the number of vulnerabilities?

The bounty payout amount of $125,000 was established for evaluation purposes by the

Government for the scenarios provided in the pricing template. The bounty payout amount is the amount that will be funded to CLIN 0001. The contractor will get paid up to $125,000, based on the number/severity type of the vulnerabilities discovered as described in the description column for CLIN 0001 in the sample order pricing tab.

49 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Sample

Order Pricing Tab

Entire Tab 1 Can the government restructure this tab to allow the offerors to complete it in way that shows accurate total costs of alternative approaches to security researchers "crowd" management? For example to include all challenge costs to include Administrative and

Bounty in a single Firm-Fixed Price Line Item.

Please see the Government's response to Question #1.

50 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 1 1 Will researchers be provided with individual credentials for the web application? The scenario has been updated with this information.

51 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 2 1 Should the word "applies " be instead the word "appliance "?

Will the researchers be provided with credentials to any of the three (3) web portals, the email server, and/or the appliance?

The scenario has been updated with this information.

52 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 3 1 Will researchers be provided with individual credentials for the VDI environment or the

Department single sign-on?

The scenario has been updated with this information.

53 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 4 1 Will researchers be provided with individual credentials for either the Department single sign-on and/or the single-factor login/password accounts for partners?

The scenario has been updated with this information.

54 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 5 1 Is this mobile application test for both the android and iOS versions of the application?

Is there an API for this application that also needs to be tested?

The scenario has been updated with this information.

55 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

System 6 1 Will researchers be provided with individual credentials for the cloud application? The scenario has been updated with this information.

56 DRAFT RFP

70RTAC22R00000010

Attachment 6 - Pricing

Template.xlsx, Scenario

Tab

Scenario Scenario Is the scenario reflective of real possible assets? Is the scenario supposed to be a single

Challenge or 6 Challenges operating on the same time-boxed timeline?

The intent of this scenario is to evaluate proposed pricing.

57 DRAFT RFP General Can you confirm that this is full & open and will there be any small business requirements to the prime?

Please see the RFP Section B.6