RFP_Amendment_0001__Attachment_8___Questions_FINAL_080118_(002).pdf

Attachment 8_RFP HQ003418R0202

Questions and Responses.

RFP Reference RFP Descriptive Text Question(s) / Response

Performance Work

Statement (PWS) FA2

- Attachment 1

1) Can the government please confirm that it plans on requesting a live test demonstration on any technical requirements of the vendor to validate that the functionality is available on their platform prior to any vendor being selected as an awardee?

No demonstrations for contract award is necessary.

Performance Work

2) Can the government please confirm that it plans on validating all functionality requested in this RFP (in reference to 6.10 of the

PWS, i.e platform with VPN, TLS visibility, monitoring of researchers, delivering full packet capture of researcher traffic, and length of time features where available) when evaluating past performance?

The vendor is responsible for demonstrating in its proposal, to the fullest extent possible, its ability to provide any monitoring and auditing capability available on their platform. Vendor is reminded that these will be on-going contractual requirements, and failure to meet them will be a breach of contract which may result in ineligibility to compete for future task orders.

Performance Work

3) Task orders for sensitive systems will have unique testing requirements, desired objectives that may or may not relate to standard vulnerability discovery categories, environment and assessment complexities, researcher skill-set and testing time requirements. The pricing table doesn’t provide the ability to reflect that. Can the government please expand the pricing table to be able to adequately quote different types of engagements that could be requested to include a complexity factor?

Basic engagement types: A internet facing Application that can be assessed using a web browser. Infrastructure IP addresses that are internet facing and can be assessed directly without special tooling.

The pricing template was slightly modified. Offerors shall provide a price proposal to reflect the work outlined in the

Task Order PWS. Vendors should not price all types of engagements at this time.

Complex engagement types: Environments that require a higher platform assurance for data protection and disposal upon engagement completion. Systems that have dedicated browsable interfaces that cannot be assessed using a web browser and require special system infrastructure in place to interact with the system.

Attachment 1, Section

1, # 7

4) The Government intends to award the IDIQ contract(s) and task orders without discussions. Therefore, initial submissions should contain the Offeror’s best terms. Notwithstanding its intent to award without discussions, the Government reserves the exclusive right to conduct discussions with Offerors, if it deems necessary, and to permit Offerors to revise their proposals.

Unless expressly authorized by the RFP, any Offeror planning to take exception to a term or condition of the RFP should consult with the Contracting Officer (CO) in writing prior to submitting a proposal.

QUESTION: Can the government clarify conditions where offerers would be engaged by the government to revise and resubmit their proposal? If the government enters into discussion with one offeror to revise a proposal, will the government enter into a discussion with all offerors and provide each offerer with equal feedback for resubmission?

Per the RFP, the Government intends to issue the IDIQ(s) and task orders without discussions. The Offeror is encouraged to submit their best offer by the required due date. The

Government will only enter into discussions when it is determined to be in the best interest in the Government. If discussions are entered into, the Government will enter into discussions with all offerors and each offeror will have an opportunity to resubmit.

Attachment 1 - Section

3 - Paragraph 3

5) Offerors are instructed to also propose a sound process for managing strategic communications to include conducting public relations, press inquiries, developing blogs and social media posts for recruitment, responding to press and

Internet-based inquiries and, managing problematic researchers relative to FA2.

QUESTION: Does this mean that the awardee can discuss engagements before and after kickoff without approvals from the government?

No. Discussion of DoD engagements with entities beyond internal vendor discussions and DDS will require written consent.

All press-related communications requires prior written consent and must be done in coordination with DDS.

Attachment 1, Section

5: Volume III

Instructions

6) Using the format provided in RFP Attachment 6, Offerors are instructed to propose an overall Firm Fixed Price, for accomplishing the work detailed in the RFP Attachment 4_Task

Order PWS, and shall include its price build-up utilizing the table structure at the bottom of RFP Attachment 4 – Task Order

Price. Requested information includes proposing a fixed rate or price for bounty, platform service, background check, and DoD service fees commensurate with the Offeror’s assessment of work in the Task Order Price. Travel costs have been pre-populated in the price template. Offerors shall not propose separate travel fees. Additionally, for the bounties, Offerors are instructed to propose a standard rate structure matched to the commercial vulnerability severity schema, such as CVSS, market prices and prior Offeror experience.

QUESTION: Can the government provide clarification on how it expects vendors to estimate total vulnerability costs across the different assessment types (i.e. web applications, source code, infrastructure, etc) for each of their Task Order pricing proposals? Does the government have an expectation specific to the market value of the various vulnerability severities (i.e. high, medium, low)? How would the government propose aligning market values of vulnerability submission costs to alleviate budget calculation risks for proposals for offerers?

The Task Order pricing template is intended for vendors to demonstrate the relative cost breakdown between platform services and researcher payouts. The DoD relies on the industry expertise of the vendors to determine reasonable bounty amounts that will drive the quality of researcher performance balanced with the value and complexity of the asset in scope. The DoD desires the best offerings private industry can provide and will not be prescriptive. Each task order pricing will be requested per the Attachment_5 CLINS &

IDIQ Provisions and Clauses outlines the process on how future task orders will be awarded.

Attachment 1 - Section

6, # 1-

7) States the Government intends to make an award to all qualifying offerors under this solicitation.

Question: Does the government plan to invite all awardees to the scoping meetings for all task orders to determine their pricing and approach?

Attachment_5 CLINS & IDIQ Provisions and Clauses outlines the process on how future task orders will be awarded.

Attachment 4 8) To help with the government budgeting process can you please offer an option to provide an all inclusive, Firm Fixed

Price for task orders instead of a variable cost table, beginning

No. The DoD expects the vendor proposal to be tailored to the duration and complexity of each assessment.

with the Task Order proposal in Attachment 4? This way the government will know from the beginning what a task order is going to cost and budget accordingly and mitigate cost risk.

Attachment 1 9) In section 5: Volume III Instructions - sub point #3 and #4:

Volume 3 price proposals, submitted by Offerors, must contain the requested price information. Omission of the requested price data will constitute an incomplete price submission, deemed non-compliant with the RFP requirements and unawardable under this solicitation.

QUESTION: Based on this statement, in regards to Attachment

4 - Task Order Price - can the Offeror submit a Fixed Firm Price statement in lieu of filling out the required fields in Attachment 4

- Task Order Price? Will a fixed firm price submission constitute as an incomplete price submission?

The vendor shall fill out all required fields, which will result in one firm fixed price.

Attachment 1 10) In section 7: Evaluation Process for IDIQ Contract, Phase 1:

Compliance Factor (Pass/Fail) - Does the Offeror own and operate a secure proprietary platform, for crowdsourced vulnerability discovery and disclosure activities, that allows researchers to participate in challenges to report vulnerabilities, allows for report triaging, allows for centralized communications between the government, triage personnel and the researchers, and that includes a secure portal and associated tools to enable audits capable of auditing and continuous monitoring of researcher activity? Does the platform comply with FA2 platform requirements in Section 6.10 of the

IDIQ PWS?

QUESTION: How will the government verify the platform based on the list of questions for section 7 and specifically Section 6.10 of the IDIQ PWS? Will the contractor have to showcase

Vendor is expected to self-certify for all compliance factors.

Please also see the answer to #2 above.

and demonstrate all functionality specified in a functional demonstration of the platform and features? Will the government verify only once during the FA2 Award or will there be multiple verifications during every challenge?

Note: Proposals that “Fail” Phase 1 will not be further evaluated.

QUESTION: .Are there any other criteria besides the questions and related Section 6.10 that will indicate a fail for phase 1?

Performance Work

Statement (PWS) FA2

- Attachment 2

11) Attachment 2 Scope #4 Sub Point #4 - The platform must have a secure portal capable of continuous monitoring and auditing of researcher activities;

QUESTION: Can the government please clarify the granularity required for auditing of researcher activities? For example, do all network ports and protocols need to be monitored as part of the platform solution? Does the offeror need to calculate individual researcher effort statistics?

The DoD requires the ability to continuously monitor individual researcher activity for the duration of the live-assessment, and the ability to audit researcher activity post-assessment. At a minimum, DoD requires the ability to know which individual researchers are accessing (or accessed) specific parts of an assessment at specific points in time.

Further monitoring/auditing requirements may be indicated at the task order level.

Vendors are encouraged to demonstrate additional monitoring/auditing capabilities.

Attachment 2 12) Attachment 2, Scope #4 Sub Point #6 - All activities under

FA2 will include commercial background check requirements as a condition of researcher participation, which will be contractor responsibility.

QUESTION: Can the government please clarify the depth of commercial background check of researchers? What is the length of history for a criminal background check requirement? Do identifications need to confirmed and identifications need to be verified? Is there a requirement that the researchers are not on a terrorist watch list? Is there a requirement that there are ongoing checks after researchers are onboarded? Are there restrictions of researchers having dual citizenships?

Commercial criminal background checks are required to be administered for researchers prior each assessment. Citizenship verification is required, at a minimum, once per year.

Researchers who have been convicted of a felony within the past 10 years are ineligible.

Vendor must have the ability to ensure researchers are valid citizens that meet the country eligibility list defined at the task order level; are not dual citizens of a country not listed on the country eligibility list defined at the task order level; not known terrorists, or an associate of a known terrorist or terrorist organization; are not listed on the U.S. Department of

Treasury’s Specially Designated Nationals List; or otherwise ineligible to conduct work for DoD.

Vendors may also be requested to provide further information on current, additional vetting capabilities (if applicable).

Attachment 2 13) Attachment 2, Scope #4 Sub Point #9 - Ability to create customized vulnerability workflow management and track vulnerabilities throughout the remediation lifecycle;

QUESTION: Will there be any requirements for the researchers and triage team to verify patch remediation efforts by rechecking the originally discovered vulnerability? Will there be time

Patch verification during either the live-assessment or post-assessment phase is preferred, but not a requirement.

allowed during Post-Challenge Phase dedicated to patch remediation verification by the researchers and working with the

DoD asset owners?

14) Attachment 2, PRE-CHALLENGE Phase Sub Point #2 -

Conduct criminal background checks on all researchers before granting them access to any DoD information.

QUESTION: What is the government requirement for the historical length of time of the criminal background check on the researcher?

A researcher must not have been convicted of a felony within the past 10 years.

15) Attachment 2, CHALLENGE Phase, Sub Point # 7 -

Complement researcher efforts with automated testing tools for source code analysis, host and application scanning, and vulnerability analysis, if applicable.

QUESTION: During each challenge, will there be dedicated funding from the government to provide for commercially available automated testing tools?

No. However, the use of the vendor’s proprietary scanning tools is expected to be calculated as part of the overall platform service cost.

16) Attachment 2, CHALLENGE Phase, Sub Point # 8 -Ensure subcontractors and security researchers adhere to rules and restrictions as consented to prior to registration and throughout the whole challenge.

QUESTION: Is the intent from the government that the vendor is liable for all researcher activities throughout the engagement?

If each Researcher conducts research and vulnerability disclosure activities in accordance with the restrictions and guidelines set forth in each task orders Rules of Engagement,

(1) DoD will not initiate or recommend any law enforcement or civil lawsuits related to such activities, and (2) in the event of any law enforcement or civil action brought by anyone other than DoD, DoD will take steps to make known that your activities were conducted pursuant to and in compliance with this policy. The Vendor is not liable for all researcher activities, but is responsible for meeting its contractual obligations; such as notifying DoD if a research does not act in accordance with the Rules of Engagements for each task order.

17) Attachment 2, POST-CHALLENGE Phase, Sub Point #1 -

Coordinate with researchers and the designated DoD

Remediation Team to ensure open vulnerability reports are adjudicated and closed out to the level of satisfaction of DoD personnel.

QUESTION: Will there be additional time dedicated to vulnerability patch verification efforts that are variable in length for each time boxed challenge and continuous challenge?

Patch verification during either the live-assessment or post-assessment phase is preferred, but not a requirement. There is not a dedicated time for vulnerability patch verification.

18) Attachment 2, Business Relations - The contractor shall provide corrective action plans, proposal submittals, timely identification of issues, and effective management of all contractor personnel (including subcontractors).

QUESTION: Can the government elaborate on what it would consider effective management of subcontractors?

The government will not be prescriptive, but reminds all vendors that: “The contractor shall seek to ensure customer satisfaction and professional and ethical behavior of all contractor personnel.”

19) Attachment 2, Place of Performance/Events - In certain cases, the contractor may have to coordinate and host live-hacking events.

QUESTION: For live-hacking events, can the government clarify on desired venues for these events? Would these events have special hotel and travel requirements?

Specific venue requirements will be indicated at the task order level. Expected venues could include vendor offices, co-working spaces, or other corporate facilities to accommodate community hacking events and/or access to hardware components when required.

Live-hacking events may be one-off events or in conjunction with large security conferences.

20) Attachment 2, Vulnerability Discovery & Disclosure

Platform - The capability to implement continuous monitoring as

The DoD requires the ability to continuously monitor individual researcher activity for the duration of the live-well as auditing tools, to monitor and assess, researcher behavior.

QUESTION: Can the government clarify if the capability desired requires continuous monitoring of all ports and protocols with traffic inspection?

assessment, and the ability to audit researcher activity post-assessment. At a minimum, DoD requires the ability to know which individual researchers are accessing (or accessed) specific parts of an assessment at specific points in time.

Further monitoring/auditing requirements will be indicated at the task order level.

Vendors are encouraged to demonstrate additional monitoring/auditing capabilities.

21) Attachment 2, 7.3 Security - QUESTION: Can the government confirm its intent to sponsor the vendors bidding on this IDIQ and associated Task Orders for a facilities clearance and personnel clearances? In the section, it specifies a program manager to be cleared to the secret level and up to 5 employees. Does this mean up to 6 employees can be sponsored for clearance for the duration of the contract?

Defense Digital Services, in conjunction with Washington

Headquarters Services, will serve as the Government Sponsor.

The vendor is expected to pursue the process to get a facilities clearance (with NO storage requirements). The DoD will sponsor a maximum of 5 personnel security clearances, including the designated Program Manager, for up to the entire duration of the contract.

PERFORMANCE

WORK

STATEMENT

CROWDSOURCED

VULNERABILITY

DISCOVERY AND

DISCLOSURE

SERVICES

Functional Area

(FA) 2 Task Order 1

22) Attachment 3, Scope - Sub Point # 3. The assessment will include a static source code analysis of approximately 200,000 lines of code.

QUESTION: Due to the time constraint of a 4 week

CHALLENGE Phase, can the researchers begin analysis during the PRE-CHALLENGE phase for the source code analysis?

If the source code is publicly available at the time of task order award, the source code can be shared with researchers prior to the commencement of the live-assessment for reconnaissance purposes.

If the source code remains internal at the time of task order award, the source code will be released at the commencement of the live assessment phase.

23) The task order states that the Government will ensure any access is remotely available to the contractor to make the challenge successful.

QUESTION: Will the Government only allow the 200,000 lines of code to be accessed remotely? Will the government provide a secure portal for source code repository and access for

Yes, access to the source code will be shared via the Github repository or through the vendor platform.

researchers to download the source code?

Facility Clearance 24) Who is the Government sponsor for Facility Clearance? Defense Digital Services, in conjunction with Washington

25) The DD254 indicates that WHS is the Government

Contracting Activity (GCA) – does that mean WHS will sign the sponsorship letter?

Defense Digital Services, in conjunction with Washington

26) The DD254 (11.b) says “receive and store” classified material. Does that mean the Government plans to provide us with SIPRNET, and/or a safe to store the classified material (hard drives, hard copies, etc.)?

No; there is NO STORAGE requirement. Vendors will not be required to have access to SIPRNet or store classified materials. All access to classified information will be in-person and hosted in government facilities.

27) The DD254 (page 3) says that personnel security clearance must be annotated in JPAS. Is that something DDS or WHS intends to do, or would the contractor need access to JPAS?

The contractor will require access to JPAS.

Security Personnel

Requirement

28) Is it acceptable if the Program Manager does not work out of the facility with the facility clearance?

Yes. However, all access to classified information will be in-person and hosted in government facilities.

29) What is the process for understanding how / when the

Government would “defray” the cost associated with obtaining a secret clearance?

The current process is that DoD pays for clearances and it is not reimbursed through the contractor. DDS is willing to pay to clear up to five individuals not to exceed $10,000 per qualified vendor. Defense Security Service is the agency that clears vendors for a facility clearance. The link below is a resource that outlines the process for obtaining a facility clearance and frequently asked questions.

http://www.dss.mil/isp/fac_clear/fac_clear.html

TIP: Due to review and coordination timeframe, this may require a quick turnaround to complete e-QIPs. Search the

Standard Form 86 PDF on any search engine to start collecting the information that you will need to upload into the e-QIP prior to it being released to you.

General Contracting

Questions

30) Is there a scenario in which the Government could choose to award the entire IDIQ to a single contractor? Or is the intent to award the IDIQ to all who qualify?

Per the RFP, the Government intends to make an award to all qualifying offerors under this solicitation.

http://www.dss.mil/isp/fac_clear/fac_clear.html